Information processing method and related equipment
By defining the basic interaction between AI entities and terminal devices in the wireless access network, updating the AI model and performing inference and training, the efficiency of AI technology application in the wireless access network is solved, and processing capabilities are improved and guaranteed. Privacy protection.
Patent Information
- Application Number
- CN202410963145.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2020-05-30
- Publication Date
- 2024-11-12
AI Technical Summary
The existing technology has failed to effectively apply artificial intelligence (AI) technology to the wireless access network, resulting in the inability to efficiently realize the basic interaction between AI entities and user equipment, limiting the processing capabilities of the wireless access network.
An information processing method is proposed, which receives the AI model information sent by the terminal device through the AI entity, updates its own AI model, and sends the updated model back to the terminal device, defining the basic interaction method between the AI entity and the terminal device. Ensure that user data is not included in the interaction process to protect privacy, and update AI models through AI inference and training data to improve processing power.
It realizes efficient application of AI technology in wireless access networks, improves the processing capabilities of AI entities and terminal devices, and ensures data security through privacy protection mechanisms.
Smart Images

Figure CN118945689A_ABST
Abstract
Description
[0001] This application is a divisional application. The application number of the original application is 202010480881.7, and the original application date is May 30, 2020. The entire contents of the original application are incorporated into this application by reference. Technical Field
[0002] The present application relates to the field of communication technology, and in particular to an information processing method and related equipment. Background Art
[0003] Artificial intelligence (AI) technology has been very successfully applied in the fields of image processing and natural language processing. For example, AI technology is applied to the network layer (such as network optimization, mobility management, resource allocation, etc.), or to the physical layer (such as channel coding, channel prediction, receiver, etc.). AI entities can be deployed in access networks to improve the processing capabilities of access networks (such as improving resource allocation efficiency, etc.), but the basic interaction method between AI entities in access networks and user equipment (UE) is not defined at present, and AI technology cannot be efficiently applied to wireless access networks. Summary of the invention
[0004] The embodiments of the present application provide an information processing method and related devices. The information processing method can apply AI technology to a wireless access network, which is beneficial to improving the processing capability of the wireless access network.
[0005] In a first aspect, an embodiment of the present application provides an information processing method, which can be applied to a first AI entity in an access network. The first AI entity can receive second AI model information sent by a terminal device, and the second AI model information does not include user data of the terminal device. The first AI entity updates the first AI model information according to the second AI model information. The first AI entity sends the updated first AI model information to the terminal device.
[0006] It can be seen that the above method flow defines a basic interaction mode between the first AI entity and the terminal device. Among them, the first AI entity and the terminal device both have AI training capabilities, so the first AI entity can train and update the first AI model based on the second AI model sent by the terminal device, and send the updated first AI model to the terminal device.
[0007] Among them, the second AI model information sent by the terminal device does not include the user data of the terminal device, which is conducive to the privacy protection of the terminal device. The above training interaction can update the first AI model of the first AI entity, which is conducive to improving the processing capabilities of the first AI entity and the terminal device.
[0008] In a possible design, the first AI entity may also receive a request message sent by the terminal device, where the request message is used to request the first AI model information. The first AI entity sends the first AI model information to the terminal device.
[0009] It can be seen that the above method flow defines another basic interaction mode between the first AI entity and the terminal device. Among them, when the terminal device has AI reasoning capability, the first AI entity receives the request message of the terminal device and sends the first AI model information to the terminal device. Correspondingly, after the terminal device receives the first AI model information, it can reason based on the data to be decided and the AI model to obtain AI decision information.
[0010] In one possible design, before the first AI entity receives the request message sent by the terminal device, the first AI entity may also receive AI information of the terminal device, where the AI information includes an AI capability parameter, wherein the AI capability parameter is used to indicate whether the terminal device has AI reasoning capability and / or AI training capability.
[0011] In one possible design, if the AI capability parameter indicates that the terminal device has AI reasoning capability, the first AI entity receives AI decision information and status information sent by the terminal device, the AI decision information is obtained by the terminal device inputting the status information into the second AI model for reasoning, and the status information is obtained by the terminal device based on observation information.
[0012] It can be seen that when the terminal device has AI reasoning capability, the terminal device can obtain AI decision information and send the AI decision information to the first AI entity, so that the first AI entity obtains the AI decision information of the terminal device, which is beneficial for the first AI entity to update the AI model.
[0013] In one possible design, the first AI entity receives AI information of the terminal device, where the AI information includes an AI update parameter;
[0014] If the AI update parameter indicates a scheduled AI update or an event-triggered AI update, the first AI entity receives feedback information, where the feedback information is used to indicate data used for AI training.
[0015] In one possible design, the first AI entity receives AI information from a terminal device, the AI information including an AI update parameter. If the AI update parameter indicates a scheduled AI update or an event-triggered AI update, the first AI entity receives feedback information indicating data used for AI training.
[0016] It can be seen that the AI update parameter in the AI information of the terminal device can instruct the terminal device to perform an AI update. Correspondingly, the first AI entity can receive feedback information sent by the terminal device, and the feedback information can be used for training and updating of the first AI entity, which is conducive to improving the processing capability of the first AI entity.
[0017] In one possible design, the first AI entity updates the first AI model according to AI training data, wherein the AI training data includes one or more of AI decision information, state information, or feedback information.
[0018] In one possible design, the feedback information includes reward information; the reward information is used to update the first AI model.
[0019] In one possible design, the reward information is determined according to a reward function. The reward function is determined according to a target parameter θ and a weight value φ of the target parameter. The target parameter is performance data obtained by executing the AI decision information by the terminal device, and the weight value of the target parameter is determined by the first AI entity according to the performance data of one or more terminal devices.
[0020] It can be seen that the embodiment of the present application expands a deep reinforcement learning process, and the first AI entity can monitor the performance indicators of the system, which is conducive to updating the first AI model.
[0021] In a second aspect, an embodiment of the present application provides an information processing method, which is applied to a terminal device. The terminal device sends a request message to a first AI entity, and the request message is used to request first AI model information. The terminal device receives the first AI model information sent by the first AI entity. The terminal device inputs the state information into the first AI model for reasoning to obtain AI decision information of the terminal device. The state information is determined based on observation information, and the observation information indicates data used for making AI decisions.
[0022] It can be seen that when the terminal device itself has AI reasoning capabilities, the terminal device can obtain the first AI model information from the first AI entity and determine the second AI model of the terminal device based on the first AI model information. The terminal device can input the data used for AI decision-making into the second AI model for reasoning, thereby obtaining AI decision information. Through the interaction between the terminal device and the first AI entity, the process of the terminal device realizing the AI reasoning function is completed, which is conducive to improving the processing capability of the terminal device.
[0023] In one possible design, before the terminal device sends the request message to the first AI entity, the terminal device may also send AI information of the terminal device to the first AI entity, where the AI information includes an AI capability parameter, wherein the AI capability parameter indicates that the terminal device has AI reasoning capability.
[0024] It can be seen that when the terminal device itself has AI reasoning capability, the terminal device can notify the first AI entity through interaction with the first AI entity.
[0025] In one possible design, the terminal device may also send AI decision information and status information to the first AI entity.
[0026] It can be seen that when the terminal device itself has AI reasoning capabilities, the terminal device can send the AI decision information obtained by reasoning to the first AI entity through interaction with the first AI entity.
[0027] In one possible design, the AI information of the terminal device includes an AI capability parameter and / or an AI update parameter. If the AI update parameter indicates a scheduled AI update or an event-triggered AI update, the terminal device may send feedback information to the first AI entity, where the feedback information is used to indicate data used for AI training.
[0028] It can be seen that when the AI update parameter of the terminal device indicates that AI update is required, the terminal device can notify the first AI entity to also perform AI training update data through interaction with the first AI entity.
[0029] In one possible design, if the AI capability parameter indicates that the terminal device has AI training capability, the terminal device obtains the second AI model according to the AI training data, wherein the AI training data includes one or more of AI decision information, state information, or feedback information.
[0030] It can be seen that when the terminal device has AI training capabilities, the terminal device can update the local second AI model through its own training.
[0031] In one possible design, the terminal device sends the second AI model information to the first AI entity. The terminal device receives the updated first AI model information sent by the first AI entity, where the updated first AI model information is determined by the first AI entity based on the second AI model information.
[0032] It can be seen that when the terminal device has AI training capabilities, the terminal device can send local second AI model information to the first AI entity through interaction with the first AI entity, so that the first AI entity updates the first AI model information according to the second AI model information. In addition, the second AI model information sent by the terminal device to the first AI entity has nothing to do with the data of the terminal device itself, which is conducive to the privacy protection of the terminal device.
[0033] In one possible design, the feedback information includes reward information; the reward information is used to update the first AI model.
[0034] In one possible design, the reward information is determined according to a reward function. The reward function is determined according to a target parameter θ and a weight value φ of the target parameter. The target parameter is performance data obtained by executing the AI decision information by the terminal device, and the weight value of the target parameter is determined by the first AI entity according to the performance data of one or more terminal devices.
[0035] It can be seen that the embodiment of the present application expands a deep reinforcement learning process. If the terminal device has AI training capabilities, it can monitor the performance indicators of the system, which is conducive to updating the local second AI model.
[0036] In a third aspect, an embodiment of the present application provides an information processing method that can be applied to a first AI entity in an access network. The first AI entity can receive observation information sent by a terminal device, where the observation information indicates data used for AI decision making. The first AI entity determines the AI decision information of the terminal device based on the observation information and the first AI model, and sends the AI decision information to the terminal device.
[0037] It can be seen that the above method flow defines another basic interaction mode between the first AI entity and the terminal device. Among them, the first AI entity has AI reasoning ability, and can determine the AI decision information of the terminal device based on the data sent by the terminal device for AI decision-making and its own first AI model. In other words, the first AI entity in the access network realizes the application of AI technology to the wireless access network, which is conducive to improving the processing capability of the wireless access network.
[0038] In one possible design, before receiving the observation information sent by the terminal device, the first AI entity may also receive AI information of the terminal device, where the AI information includes an AI capability parameter, wherein the AI capability parameter is used to indicate whether the terminal device has AI reasoning capability and / or AI training capability.
[0039] In a possible design, if the AI capability parameter of the terminal device indicates that the terminal device has no AI capability, the first AI entity receives observation information sent by the terminal device.
[0040] It can be seen that if the terminal device does not have AI reasoning capabilities, the terminal device can implement related AI functions through the first AI entity.
[0041] In one possible design, the first AI entity may pre-process the observation information to obtain corresponding state information, and then input the state information into the first AI model for reasoning to obtain AI decision information of the terminal device.
[0042] It can be seen that in the process of obtaining the AI decision information of the terminal device, the first AI entity must first convert the observation information into state information that can be processed by the AI model before obtaining the AI decision information.
[0043] In a fourth aspect, an embodiment of the present application provides an information processing method that can be applied to a terminal device. The terminal device sends observation information to a first AI entity, where the observation information indicates data used for AI decision making. The terminal device receives the AI decision information of the terminal device sent by the first AI entity, and executes a decision based on the AI decision information.
[0044] It can be seen that the terminal device can obtain the AI decision information of the terminal device and implement the corresponding AI function through interaction with the first AI entity.
[0045] In a possible design, before the terminal device sends the observation information to the first AI entity, the terminal device may also send AI information of the terminal device to the first AI entity, where the AI information includes an AI capability parameter, wherein the AI capability parameter indicates that the terminal device has no AI capability.
[0046] In one possible design, the AI decision information of the terminal device is obtained by the first AI entity inputting the state information into the first AI model for reasoning; the state information is obtained by the first AI entity based on the observation information.
[0047] It can be seen that when the terminal device has no AI capability, the AI decision information of the terminal device can be obtained through interaction with the first AI entity.
[0048] In a fifth aspect, an embodiment of the present application provides a first AI entity, which includes an intelligent decision-making module. The intelligent decision-making module is used to receive second AI model information sent by a terminal device, and the second AI model information does not include user data of the terminal device. The intelligent decision-making module is also used to update the first AI model information according to the second AI model information. The first AI model information is the AI model information of the first AI entity. The intelligent decision-making module is also used to send the updated first AI model information to the terminal device.
[0049] In one possible design, the intelligent decision module is further used to receive a request message sent by the terminal device, where the request message is used to request the first AI model information. After receiving the request message, the intelligent decision module can send the first AI model information to the terminal device.
[0050] In a possible design, the first AI entity further includes a preprocessing module, wherein the preprocessing module is used to receive AI information of the terminal device, where the AI information includes AI capability parameters.
[0051] In a possible design, if the AI capability parameter indicates that the terminal device has AI reasoning capability, the intelligent decision module is also used to receive AI decision information and status information sent by the terminal device. The AI decision information is obtained by the terminal device inputting the status information into the second AI model for reasoning, and the status information is obtained by the terminal device based on the observation information, and the observation information indicates the data used for AI decision making.
[0052] In one possible design, the preprocessing module is further used to receive AI information of the terminal device, where the AI information includes AI update parameters. The first AI entity may also include a data collection and training module. If the AI update parameters indicate a scheduled AI update or an event-triggered AI update, the data collection and training module is used to receive feedback information, where the feedback information is used to indicate data used for AI training.
[0053] In one possible design, the intelligent decision module is further used to update the first AI model according to AI training data, wherein the AI training data includes one or more of AI decision information, state information or feedback information.
[0054] In one possible design, the feedback information includes reward information, and the reward information is used to update the first AI model.
[0055] In one possible design, the reward information is determined according to a reward function. The reward function is determined according to a target parameter θ and a weight value φ of the target parameter. The target parameter is performance data obtained by executing the AI decision information by the terminal device, and the weight value of the target parameter is determined by the first AI entity according to the performance data of one or more terminal devices.
[0056] In a sixth aspect, an embodiment of the present application provides a terminal device, which includes a transceiver module and a processing module. The transceiver module is used to send a request message to the first AI entity, and the request message is used to request the first AI model information. The transceiver module is also used to receive the first AI model information sent by the first AI entity. The processing module is used to input the state information into the second AI model for reasoning to obtain the AI decision information of the terminal device; wherein the state information is determined based on the observation information; the observation information indicates the data used for AI decision-making; the second AI model is determined by the terminal device based on the first AI model information.
[0057] In one possible design, the transceiver module is further used to send AI information of the terminal device to the first AI entity, where the AI information includes an AI capability parameter, wherein the AI capability parameter indicates that the terminal device has AI reasoning capability.
[0058] In one possible design, the transceiver module is also used to send AI decision information and status information to the first AI entity.
[0059] In one possible design, the AI information of the terminal device includes an AI capability parameter and / or an AI update parameter. If the AI update parameter indicates a scheduled AI update or an event-triggered AI update, the transceiver module is further configured to send feedback information to the first AI entity, where the feedback information is used to indicate data used for AI training.
[0060] In one possible design, if the AI capability parameter indicates that the terminal device has AI training capability, the processing module is further configured to obtain a second AI model based on AI training data, wherein the AI training data includes one or more of AI decision information, state information, or feedback information.
[0061] In one possible design, the transceiver module is further configured to send the second AI model information to the first AI entity. The transceiver module may also receive updated first AI model information sent by the first AI entity, where the updated first AI model information is determined by the first AI entity based on the second AI model information.
[0062] In one possible design, the feedback information includes reward information, wherein the reward information is used to update the first AI model.
[0063] In one possible design, the reward information is determined according to a reward function. The reward function is determined according to a target parameter θ and a weight value φ of the target parameter. The target parameter is performance data obtained by executing the AI decision information by the terminal device, and the weight value of the target parameter is determined by the first AI entity according to the performance data of one or more terminal devices.
[0064] In a seventh aspect, an embodiment of the present application provides a first AI entity, which includes a preprocessing module and an intelligent decision-making module. The preprocessing module is used to receive observation information sent by a terminal device, and the observation information indicates data used for AI decision-making. The intelligent decision-making module is used to determine the AI decision information of the terminal device based on the observation information and the first AI model. The intelligent decision-making module is also used to send AI decision information to the terminal device.
[0065] In one possible design, the preprocessing module is also used to receive AI information of the terminal device, where the AI information includes AI capability parameters.
[0066] In one possible design, if the AI capability parameter of the terminal device indicates that the terminal device has no AI capability, the preprocessing module is used to receive observation information sent by the terminal device.
[0067] In one possible design, the preprocessing module is further used to preprocess the observation information to obtain corresponding state information. The intelligent decision module is further used to input the state information into the first AI model for reasoning to obtain AI decision information of the terminal device.
[0068] In an eighth aspect, an embodiment of the present application provides a terminal device, the terminal device comprising a transceiver module and a processing module. The transceiver module is used to send observation information to the first AI entity, the observation information indicating data used for AI decision making. The transceiver module is also used to receive AI decision information of the terminal device sent by the first AI entity. The processing module is used to execute a decision according to the AI decision information.
[0069] In one possible design, the transceiver module is further configured to send AI information of the terminal device to the first AI entity, where the AI information includes an AI capability parameter, wherein the AI capability parameter indicates that the terminal device has no AI capability.
[0070] In one possible design, the AI decision information of the terminal device is obtained by the first AI entity inputting the state information into the first AI model for reasoning; the state information is obtained by the first AI entity based on the observation information.
[0071] In a ninth aspect, an embodiment of the present application provides a first AI entity, which has the function of implementing the information processing method provided in the first aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.
[0072] In a tenth aspect, an embodiment of the present application provides a terminal device, which has the function of implementing the information processing method provided in the second aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.
[0073] In the eleventh aspect, the embodiment of the present application provides a first AI entity, which has the function of implementing the information processing method provided in the third aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.
[0074] In a twelfth aspect, an embodiment of the present application provides a terminal device, which has the function of implementing the information processing method provided in the fourth aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.
[0075] In the thirteenth aspect, an embodiment of the present application provides a communication system, which includes the first AI entity provided by the fifth, seventh, ninth or eleventh aspect, and the terminal device provided by the sixth, eighth, tenth or twelfth aspect.
[0076] In the fourteenth aspect, an embodiment of the present application provides a computer-readable storage medium, which includes a program or instructions. When the program or instructions are run on a computer, the computer executes the method in the first aspect or any possible implementation of the first aspect.
[0077] In the fifteenth aspect, an embodiment of the present application provides a computer-readable storage medium, which includes a program or instructions. When the program or instructions are run on a computer, the computer executes the method in the second aspect or any possible implementation of the second aspect.
[0078] In the sixteenth aspect, an embodiment of the present application provides a computer-readable storage medium, which includes a program or instructions. When the program or instructions are run on a computer, the computer executes the method in the third aspect or any possible implementation of the third aspect.
[0079] In the seventeenth aspect, an embodiment of the present application provides a computer-readable storage medium, which includes a program or instructions. When the program or instructions are run on a computer, the computer executes the method in the fourth aspect or any possible implementation of the fourth aspect.
[0080] In the eighteenth aspect, an embodiment of the present application provides a chip or a chip system, which includes at least one processor and an interface, the interface and at least one processor are interconnected through lines, and the at least one processor is used to run computer programs or instructions to perform the method described in the first aspect or any possible implementation of the first aspect.
[0081] In the nineteenth aspect, an embodiment of the present application provides a chip or a chip system, which includes at least one processor and an interface, the interface and at least one processor are interconnected by lines, and the at least one processor is used to run computer programs or instructions to perform the method described in the second aspect or any possible implementation of the second aspect.
[0082] In the twentieth aspect, an embodiment of the present application provides a chip or a chip system, which includes at least one processor and an interface, the interface and at least one processor are interconnected by lines, and the at least one processor is used to run computer programs or instructions to perform the method described in the third aspect or any possible implementation of the third aspect.
[0083] In the twenty-first aspect, an embodiment of the present application provides a chip or a chip system, which includes at least one processor and an interface, the interface and at least one processor are interconnected by lines, and the at least one processor is used to run computer programs or instructions to perform the method described in the fourth aspect or any possible implementation of the fourth aspect.
[0084] The interface in the chip may be an input / output interface, a pin or a circuit, etc.
[0085] The chip system in the above aspects may be a system on chip (SOC) or a baseband chip, etc., wherein the baseband chip may include a processor, a channel encoder, a digital signal processor, a modem and an interface module, etc.
[0086] In a possible implementation, the chip or chip system described above in the present application further includes at least one memory, in which instructions are stored. The memory may be a storage module inside the chip, such as a register, a cache, etc., or a storage module of the chip (e.g., a read-only memory, a random access memory, etc.).
[0087] In the twenty-second aspect, an embodiment of the present application provides a computer program or a computer program product, including codes or instructions. When the codes or instructions are run on a computer, the computer executes the method in the first aspect or any possible implementation of the first aspect.
[0088] In the twenty-third aspect, an embodiment of the present application provides a computer program or a computer program product, including codes or instructions. When the codes or instructions are executed on a computer, the computer executes the method in the second aspect or any possible implementation of the second aspect.
[0089] In the twenty-fourth aspect, an embodiment of the present application provides a computer program or a computer program product, including codes or instructions. When the codes or instructions are run on a computer, the computer executes the method in the third aspect or any possible implementation of the third aspect.
[0090] In the twenty-fifth aspect, an embodiment of the present application provides a computer program or a computer program product, including codes or instructions. When the codes or instructions are run on a computer, the computer executes the method in the fourth aspect or any possible implementation of the fourth aspect. BRIEF DESCRIPTION OF THE DRAWINGS
[0091] Figure 1 A schematic diagram of the interaction between an agent and the environment;
[0092] Figure 2 is a schematic diagram of a Markov decision process;
[0093] Figure 3a A schematic diagram of a network architecture provided in an embodiment of the present application;
[0094] Figure 3b A schematic diagram of a 5G RAN architecture provided in an embodiment of the present application;
[0095] Figure 4 A schematic diagram of a RAN architecture provided in an embodiment of the present application;
[0096] Figure 5 A flowchart of an information processing method provided in an embodiment of the present application;
[0097] Figure 6 A flowchart of information processing when a terminal device has no AI capability provided in an embodiment of the present application;
[0098] Figure 7 A flowchart of another information processing method provided in an embodiment of the present application;
[0099] Figure 8 A flowchart of information processing when a terminal device has AI reasoning capability provided in an embodiment of the present application;
[0100] Fig. 9 A flowchart of another information processing method provided in an embodiment of the present application;
[0101] Fig.10 A schematic diagram of a federated learning process provided for an embodiment of the present application;
[0102] Fig.11 A flowchart of an AI training provided in an embodiment of the present application;
[0103] Fig.12 A schematic diagram of a DRL online learning process provided in an embodiment of the present application;
[0104] Fig.13 A flowchart of a decision-making early stopping technology provided in an embodiment of the present application;
[0105] Fig.14 A schematic diagram of an application of a DRL algorithm provided in an embodiment of the present application deployed in a cell;
[0106] Fig.15 A schematic diagram of a virtual cell assisted training provided in an embodiment of the present application;
[0107] Fig.16 A schematic diagram of a training terminal deployed in a real cell provided in an embodiment of the present application;
[0108] Fig.17 A schematic diagram of the structure of a first AI entity provided in an embodiment of the present application;
[0109] Fig.18 A schematic diagram of the structure of another first AI entity provided in an embodiment of the present application;
[0110] Fig.19 A schematic diagram of the structure of another first AI entity provided in an embodiment of the present application;
[0111] Fig. 20 A schematic diagram of the structure of another first AI entity provided in an embodiment of the present application;
[0112] Fig.21 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application;
[0113] Fig. 22 A schematic diagram of the structure of another terminal device provided in an embodiment of the present application;
[0114] Fig.23 A schematic diagram of the structure of another terminal device provided in an embodiment of the present application;
[0115] Fig.24 A schematic diagram of the structure of another terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0116] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.
[0117] Before describing the embodiments of the present application, related concepts are first explained.
[0118] Artificial intelligence (AI) technology has been very successfully applied in the fields of image processing and natural language processing. At present, the academic community has conducted a lot of research on applying AI technology to the network layer (such as network optimization, mobility management, resource allocation, etc.) and the physical layer (such as channel coding, channel prediction, receivers, etc.).
[0119] Artificial intelligence (AI) technology has been very successfully applied in the fields of image processing and natural language processing. For example, AI technology is applied to the network layer (such as network optimization, mobility management, resource allocation, etc.), or to the physical layer (such as channel coding, channel prediction, receiver, etc.). Among them, the more commonly used AI technologies include supervised learning and reinforcement learning.
[0120] Among them, supervised learning refers to the process of adjusting the parameters of the classifier using a set of samples of known categories to achieve the required performance, also known as supervised training. The goal of supervised learning is to learn the mapping relationship between input and output in a training set given a training set. Among them, the training set is a set of correct input-output mapping relationships. Supervised learning methods are currently a type of machine learning method that is widely studied. For example, supervised learning methods include neural network propagation algorithms, decision tree learning algorithms, etc.
[0121] Among them, reinforcement learning is the learning of the agent and the environment in an interactive way. Figure 1 , Figure 1 This is a schematic diagram of the interaction between an agent and the environment. The agent can take actions on the environment based on the state of the environment feedback, thereby obtaining a reward and the state of the next moment, so that the agent can accumulate the maximum reward within a period of time.
[0122] Reinforcement learning is different from supervised learning, mainly in that it does not require a training set. In reinforcement learning, the reinforcement signal provided by the environment evaluates the quality of the action (usually using a scalar signal), rather than telling the reinforcement learning system how to generate the correct action. Since the external environment provides little information, the agent needs to learn from its own experience. In this way, the agent gains knowledge in an action-evaluation environment and improves the action plan to adapt to the environment.
[0123] Common reinforcement learning algorithms include Q-learning, policy gradient, actor-critic, etc. For example, the currently commonly used reinforcement learning algorithm is deep reinforcement learning (DRL), which mainly combines reinforcement learning with deep learning and uses neural networks to model the policy / value function to adapt to larger input / output dimensions.
[0124] In AI technology, a variety of mathematical models can be used for reasoning to obtain AI decisions. Mathematical models may include but are not limited to Markov decision process, neural network and other models. For example, see Figure 2 , Figure 2 The figure is a schematic diagram of a Markov decision process (MDP). The Markov decision process is a mathematical model for analyzing decision problems. It assumes that the environment has the Markov property (the conditional probability distribution of the future state of the environment depends only on the current state). The decision maker observes the state of the environment periodically (e.g. Figure 2 s0, s1, etc.), and make decisions based on the current state of the environment (such as Figure 2 a0, a1, etc.), and obtain new states and rewards after interacting with the environment (such as Figure 2 r0, r1, etc.), such as Figure 2 shown.
[0125] With the evolution of future mobile communication network technology, new radio access technology (NR) has redefined the architecture of access networks. Figure 3a , Figure 3a A schematic diagram of a network system provided for an embodiment of the present application. The network system includes a core network (5GC), an access network (NG-RAN) and a terminal device. The 5GC and the NG-RAN exchange information through the NG interface; the access network devices (such as gNB) in the NG-RAN can exchange information through the Xn interface. The terminal device can be connected to the access network device through a wireless link to realize information exchange between the terminal device and the access network device.
[0126] Among them, the network system may include but is not limited to: global system for mobile communications (GSM), wideband code division multiple access system (WCDMA), long term evolution system (LTE), enhanced mobile broadband (eMBB) scenarios in the new generation of radio access technology (NR), ultra-reliable low latency communications (uRLLC) scenarios and massive machine type communications (mMTC) scenarios, narrowband Internet of Things (NB-IoT) systems, etc.
[0127] Among them, the access network equipment can be any device with wireless transceiver function, providing wireless communication services for terminal devices within the coverage area. The access network equipment may include but is not limited to: the evolved base station (NodeB or eNB or e-NodeB, evolutional NodeB) in the long term evolution (LTE) system, the base station (gNodeB or gNB) or the transmission receiving point / transmission reception point (TRP) in the new generation of wireless access technology (NR), the base station of the subsequent evolution of 3GPP, the access node in the WiFi system, the wireless relay node, the wireless backhaul node, the equipment that assumes the base station function in the Internet of Vehicles, D2D communication, and machine communication, and the satellite, etc.
[0128] The terminal device may be a device with wireless transceiver function, or the terminal device may be a chip. The terminal device may be user equipment (UE), mobile phone, tablet computer (Pad), computer with wireless transceiver function, virtual reality (VR) terminal device, augmented reality (AR) terminal device, vehicle-mounted terminal device, wireless terminal in remote medical, wireless terminal in smart grid, wearable terminal device, Internet of Vehicles, D2D communication, terminal in machine communication, etc.
[0129] Optional, see Figure 3b , Figure 3b A schematic diagram of the architecture of a 5G RAN provided in an embodiment of the present application. Among them, the access network equipment (such as gNB) in the NG RAN may include a centralized module (central unit, CU) and a distributed module (distribute unit). The CU and DU may exchange information through the F1 interface, such as Figure 3b shown.
[0130] It can be seen that the increasingly mature AI technology will play an important role in promoting the evolution of future mobile communication network technology. For example, AI entities can be deployed in access networks to improve the processing capabilities of access networks (such as improving resource allocation efficiency, etc.), but the basic interaction method between AI entities in access networks and user equipment (UE) is not defined at present, and AI technology cannot be efficiently applied to wireless access networks.
[0131] An embodiment of the present application provides an information processing method, which can apply AI technology to a wireless access network, thereby improving the processing capability of the wireless access network.
[0132] The information processing method can be applied to a RAN architecture provided in an embodiment of the present application. Figure 4 , Figure 4 A RAN architecture is provided in an embodiment of the present application. In the RAN architecture, a first AI entity (AI module) is added, and it is defined that the first AI entity and the gNB can exchange information through the A1 interface, such as Figure 4 It should be noted that the first AI entity described in this embodiment may be located in the edge / cloud access network, which is conducive to realizing the corresponding AI function through edge computing / cloud computing.
[0133] Optionally, the first AI entity may be further divided into a first AI entity-centralized module (AM-CU) and a first AI entity-distributed module (AM-DU). The gNB may also be physically divided into a gNB-CU and a gNB-DU. The AM-CU and the gNB-CU define information exchange through an A1-C interface, and the AM-DU and the gNB-DU define information exchange through an A1-D interface, such as Figure 4 shown.
[0134] Among them, the communication content of the A1 interface may include but is not limited to uploading / downloading of AI models, uploading / downloading of data, information interaction between the gNB and the first AI entity (such as the performance tracking module in the first AI entity can monitor the performance data of the gNB), etc. Optionally, the A1 interface is divided into A1-C and A1-D interfaces according to function, which can correspond to the functional division of gNB-CU and gNB-DU, and the communication content of each interface is also different. For example, the A1-D interface transmits messages involving the physical layer (physical, PHY), media access control (media access control, MAC) layer and radio link control (radio link control, RLC) layer; the A1-C interface transmits messages involving higher layers (such as the packet data convergence protocol (packet data convergence protocol, PDCP) layer).
[0135] The following will describe it in conjunction with specific embodiments.
[0136] See also Figure 5 , Figure 5 A flowchart of an information processing method provided in an embodiment of the present application. Figure 5The information processing method flow is implemented by the interaction between the first AI entity and the terminal device, and may include the following steps:
[0137] S501, the terminal device sends observation information to the first AI entity; correspondingly, the first AI entity receives the observation information sent by the terminal device;
[0138] S502, the first AI entity determines AI decision information of the terminal device according to the observation information and the first AI model;
[0139] S503: The first AI entity sends AI decision information to the terminal device; correspondingly, the terminal device receives the AI decision information sent by the first AI entity.
[0140] This embodiment defines a basic interaction mode between the first AI entity and the terminal device when the terminal device does not have AI capability. Among them, whether the terminal device has AI capability can be indicated by the AI information of the terminal device. The AI information of the terminal device may include but is not limited to the following parameters: AI capability parameter (AICapabilityClass), AI update parameter (AIUpdateType) and AI interaction parameter (AIInteractionType).
[0141] The AI capability parameter is used to indicate whether the terminal device has AI capability. Specifically, the AI capability parameter can indicate whether the terminal device has AI capability through a specific parameter value.
[0142] For example, when the parameter value of AICapabilityClass is class 0, it means that the terminal device has no AI capability. In other words, the terminal device does not have AI reasoning and / or AI training capabilities, that is, the terminal device cannot implement AI functions.
[0143] For another example, when the parameter value of AICapabilityClass is class 1, it means that the terminal device has AI reasoning capability. In other words, the terminal device can implement some AI functions, such as obtaining AI decisions.
[0144] For another example, when the parameter value of AICapabilityClass is class 2, it means that the terminal device has AI training capability. In other words, the terminal device can implement some AI functions, such as training the AI model to obtain a better AI model.
[0145] For another example, when the parameter value of AICapabilityClass is class 3, it means that the terminal device has AI reasoning capability and AI training capability. In other words, the terminal device can implement AI functions, such as training AI models to obtain better AI models, thereby obtaining better AI decisions.
[0146] It should be noted that the above parameter value of AICapabilityClass is only an example, and the parameter value of AICapabilityClass may also be in other forms, such as represented by binary numbers, which is not limited in this embodiment.
[0147] Among them, the AI update parameter is used to indicate whether the terminal device performs an AI update. AI update refers to updating data. For example, if the AI algorithm used is a reinforcement learning algorithm, the terminal device can send feedback information to the first AI entity to enable the first AI entity to update data. Specifically, the AI update parameter can also indicate whether to perform an AI update through a specific parameter value.
[0148] For example, when the parameter value of AIUpdateType is type 0, it means that no AI update is performed.
[0149] For another example, when the parameter value of AIUpdateType is type 1, it means that AI update is triggered by events. That is, when there is an external event trigger, for example, the AI model is not suitable due to environmental changes, the AI update can be triggered by a long-term KPI deterioration event.
[0150] For another example, when the parameter value of AIUpdateType is type 2, it indicates that the AI update is triggered on a timed basis. For example, the system may set a time parameter, which may indicate that the system will trigger the AI update every preset time period.
[0151] It should be noted that the above parameter value of AIUpdateType is only an example, and the parameter value of AIUpdateType may also be in other forms, such as represented by binary numbers, which is not limited in this embodiment.
[0152] The AI interaction parameter is used to indicate the interaction content between the terminal device and the first AI entity. The interaction content between the terminal device and the first AI entity in this embodiment may include but is not limited to data, models, etc.
[0153] The data exchanged between the terminal device and the first AI entity refers to data used for AI reasoning and / or AI training, and may include but is not limited to state information, observation information, etc.
[0154] For example, when the first AI entity uses a reinforcement learning algorithm, the state information may be as follows: Figure 1 The state of environmental feedback in the reinforcement learning algorithm shown. The model for interaction between the terminal device and the first AI entity refers to a model used for AI reasoning and / or AI training. Different AI models correspond to different AI algorithms used by the first AI entity, and this embodiment does not limit this.
[0155] Specifically, the AI interaction parameter may indicate the interaction content between the terminal device and the first AI entity through a specific parameter value.
[0156] For example, when the parameter value of AIInteractionType is type 0, it indicates that the interaction content between the terminal device and the first AI entity includes uploading data and / or downloading data.
[0157] For another example, when the parameter value of AIInteractionType is type 1, it indicates that the interaction content between the terminal device and the first AI entity includes uploading data and / or downloading a model.
[0158] For another example, when the parameter value of AIInteractionType is type 2, it indicates that the interaction content between the terminal device and the first AI entity includes uploading a model and / or downloading a model.
[0159] It should be noted that the above parameter value of AIInteractionType is only an example, and the parameter value of AIInteractionType may also be in other forms, such as represented by binary numbers, which is not limited in this embodiment.
[0160] Optionally, before the terminal device sends the observation information to the first AI entity, the terminal device may send AI information of the terminal device to the first AI entity. The AI information of the terminal device may include one or more of the AI capability parameters, AI update parameters, or AI interaction parameters described in the above embodiments.
[0161] For example, after the terminal device establishes a communication connection with the first AI entity, the terminal device can send a service request message (such as a resource allocation request message) to the first AI entity. The service request message can carry AI information of the terminal device to enable the first AI entity to know whether the terminal device has AI capabilities.
[0162] Among them, the first AI entity is a new entity in the access network, and the first AI entity has AI functions such as AI reasoning and AI training. Specifically, the first AI entity can be divided into multiple functional modules according to its functions, including an intelligent policy function (IPF), a data collection and training function (DTF), a pre-processing function (PPF), a performance tracking function (PMF) and other modules, and each module is used to perform a corresponding function.
[0163] Optionally, in S501 of this embodiment, the terminal device sends the observation information to the first AI entity, which may be that the terminal device sends the observation information to a preprocessing module of the first AI entity.
[0164] The observation information indicates the data used for AI decision making. That is, the observation information is the data provided for AI decision making. For example, when a terminal device requests resource scheduling from an access network, the observation information sent by the terminal device to the preprocessing module may include data such as the throughput of the terminal device.
[0165] Optionally, in S502 of this embodiment, the first AI entity determines the AI decision information of the terminal device according to the observation information and the first AI model, which may be performed by an intelligent decision module of the first AI entity.
[0166] The first AI model is a model for AI reasoning and / or AI training in the first AI entity, that is, the first AI model is an AI model in the edge / cloud. Depending on the AI algorithm used, the first AI model may include multiple types. For example, when the AI algorithm used is deep reinforcement learning, the first AI model may be a fully connected neural network model.
[0167] Optionally, S502 in this embodiment may also be performed by the preprocessing module and the intelligent decision-making module of the first AI entity respectively, and includes the following two steps:
[0168] The preprocessing module preprocesses the observation information to obtain the corresponding status information;
[0169] The intelligent decision-making module inputs the state information into the first AI model for reasoning to obtain AI decision information of the terminal device.
[0170] Among them, since the first AI model cannot directly process the observation information, the preprocessing module can first preprocess the observation information (for example, normalize the data) to obtain the status information.
[0171] The state information is data that can be directly used when using the AI model for reasoning. For example, the state information can refer to Figure 2 The system state (such as s0, s1, etc.) in the Markov decision process shown can also refer to preprocessed data (the state in the hidden Markov model cannot be directly obtained).
[0172] The intelligent decision module can input the state information into the first AI model for reasoning. For example, when the first AI entity adopts a reinforcement learning algorithm, the intelligent decision module can be as follows: Figure 1 The intelligent agent in the reinforcement learning algorithm shown can take actions on the environment, that is, obtain AI decision information of the terminal device.
[0173] The AI decision information of the terminal device is the result obtained by the first AI entity performing AI reasoning based on the data used for making AI decisions.
[0174] For example, when the first AI entity uses a reinforcement learning algorithm, the AI decision information is the action output by the intelligent agent. Specifically, when the terminal device requests resource scheduling from the access network, the resource allocation result obtained by the first AI entity through AI reasoning is the AI decision information of the terminal device.
[0175] It should be noted that compared with traditional resource allocation methods, the first AI entity applies AI technology to resource allocation in the access network, which can allocate resources to corresponding terminal devices more specifically, thereby helping to optimize the overall network performance.
[0176] Optionally, in S503 of this embodiment, the first AI entity sends the AI decision information to the terminal device, which may be that the intelligent decision module of the first AI entity sends the AI decision information to the terminal device.
[0177] Optional, see Figure 6 , Figure 6 A flowchart of information processing when a terminal device has no AI capability is provided in an embodiment of the present application. Since the terminal device has no AI capability, the terminal device can choose to request an AI decision from the first AI entity of the edge / cloud.
[0178] It should be noted that the terminal equipment uses Figure 6 The information processing method shown has a large delay in obtaining AI decisions. This method is suitable for services that are not sensitive to delay.
[0179] S601, the terminal device sends observation information to the preprocessing module, where the observation information indicates data used for AI decision-making; correspondingly, the preprocessing module receives the observation information sent by the terminal device;
[0180] S602, a preprocessing module preprocesses the observation information to obtain corresponding status information;
[0181] S603, the preprocessing module sends status information to the intelligent decision-making module; correspondingly, the intelligent decision-making module receives the status information sent by the preprocessing module;
[0182] S604, the intelligent decision module inputs the state information into the first AI model for reasoning to obtain AI decision information of the terminal device;
[0183] S605, the intelligent decision module sends AI decision information to the terminal device; correspondingly, the terminal device receives the AI decision information sent by the intelligent decision module;
[0184] S606, the terminal device executes the decision according to the AI decision information.
[0185] S601 to S606 are the overall processing flow when the AIUpdateType parameter value of the terminal device is type 0. Optionally, when the AIUpdateType parameter value is type 1 or type 2, Figure 6 The information processing flow shown also includes the process of AI training data collection, including the following steps:
[0186] S607, the intelligent decision-making module sends the status information and AI decision information to the data collection and training module;
[0187] S608, the data collection and training module receives feedback information.
[0188] The feedback information is used to indicate the data used for AI training. Depending on the AI algorithm, the feedback information received by the data collection and training module is also different.
[0189] For example, if the AI algorithm used by the first AI entity is reinforcement learning, then the data collection and training module receives the reward information sent by the terminal device or the performance tracking module. Then S608 may include two parallel steps S608a and S608b. Among them, S608a is the terminal device sending feedback information to the data collection and training module; S608b is the performance tracking module sending feedback information to the data collection and training module.
[0190] For another example, if the AI algorithm adopted by the first AI entity is supervised learning, then the data collection and training module receives the label information sent by the terminal device.
[0191] It should be noted that there is no order in which S607 and S606 are executed, that is, S606 and S607 can be executed at the same time.
[0192] The embodiment of the present application provides an information processing method, which defines a basic interaction mode between a first AI entity and a terminal device. When the terminal device does not have AI capability, the terminal device can implement the AI function through the first AI entity in the access network to obtain the AI decision information of the terminal device. In other words, the first AI entity in the access network implements the application of AI technology to the wireless access network, which is conducive to improving the processing capability of the wireless access network.
[0193] See also Figure 7 , Figure 7 A flowchart of another information processing method provided in an embodiment of the present application. Figure 7 The information processing method flow is implemented by the interaction between the first AI entity and the terminal device, and may include the following steps:
[0194] S701, the terminal device sends a request message to the first AI entity; correspondingly, the first AI entity receives the request message sent by the terminal device;
[0195] S702, the first AI entity sends first AI model information to the terminal device; correspondingly, the terminal device receives the first AI model information sent by the first AI entity;
[0196] S703, the terminal device inputs the state information into the first AI model for reasoning to obtain AI decision information of the terminal device.
[0197] This embodiment defines a basic interaction mode between the first AI entity and the terminal device when the terminal device has AI reasoning capability. That is, when the AICapabilityClass parameter value of the terminal device is class 1, the terminal device can implement AI reasoning and obtain AI decisions.
[0198] Among them, since the terminal device only has AI reasoning capability but not AI training capability, the terminal device needs to send a request message to the first AI entity to obtain the AI model. Among them, the request message is used to request the first AI entity for the first AI model information. The first AI model information can be the first AI model or related parameters of the first AI model.
[0199] For example, when the first AI model is a neural network, the first AI model information may be an entire neural network or relevant parameters of the neural network (such as the number of layers of the neural network, the number of neurons, etc.).
[0200] Optionally, the execution of the above S701 and S702 may be determined according to the AI update parameter of the terminal device. That is, if the parameter value of AIUpdateType is type 0, S701 and S702 are only executed once during initialization.
[0201] If the parameter value of AIUpdateType is type1, S701 and S702 are executed according to event triggers. For example, if the performance tracking module monitors that the system performance has deteriorated, an update is triggered.
[0202] If the parameter value of AIUpdateType is type2, S701 and S702 are executed periodically.
[0203] Optionally, the terminal device in this embodiment has AI reasoning capability, so the terminal device may include multiple AI functional modules. For example, the terminal device includes a preprocessing module and an intelligent decision-making module, which are used to implement the AI reasoning process.
[0204] It should be noted that the terminal device can also implement the AI reasoning function through a local second AI entity. The local second AI entity is an AI entity that is physically close to the terminal device and can be an external device of the terminal device, which is not limited in this embodiment.
[0205] The following description takes the example that the terminal device itself includes multiple AI function modules.
[0206] See also Figure 8 , Figure 8 A flowchart of information processing when a terminal device has AI reasoning capability is provided for an embodiment of the present application. The terminal device has AI reasoning capability, that is, the terminal device includes at least an intelligent decision-making module. The terminal device can request first AI model information from the first AI entity of the edge / cloud, and complete AI reasoning locally to obtain AI decision information.
[0207] It should be noted that the terminal equipment uses Figure 8 The information processing method shown has a small delay in obtaining AI decision information, and this method is suitable for services that are sensitive to delay.
[0208] For ease of description, Figure 8 The AI functional modules included in the terminal device in the illustrated embodiment are respectively referred to as second modules. For example, the intelligent decision-making module of the terminal device is referred to as a second intelligent decision-making module. Figure 8 The AI function modules included in the first AI entity in the embodiment shown are respectively referred to as first modules. For example, the intelligent decision module of the first AI entity is referred to as the first intelligent decision module. The information processing process may include the following steps:
[0209] S801, the second intelligent decision module sends a request message to the first intelligent decision module; correspondingly, the first intelligent decision module receives the request message sent by the second intelligent decision module;
[0210] S802, the first intelligent decision-making module sends first AI model information to the second intelligent decision-making module; correspondingly, the second intelligent decision-making module receives the first AI model information sent by the first intelligent decision-making module;
[0211] S803, a second preprocessing module obtains observation information;
[0212] S804, a second preprocessing module preprocesses the observation information to obtain corresponding status information;
[0213] S805, the second preprocessing module sends status information to the second intelligent decision-making module; correspondingly, the second intelligent decision-making module receives the status information sent by the second preprocessing module;
[0214] S806, the second intelligent decision module inputs the state information into the first AI model for reasoning to obtain AI decision information of the terminal device;
[0215] S807, the terminal device executes the decision according to the AI decision information.
[0216] The above S801 to S807 are the overall processing flow when the AIUpdateType parameter value of the terminal device is type 0. Optionally, when the AIUpdateType parameter value is type 1 or type 2, Figure 8 The information processing flow shown also includes the process of AI training data collection, including the following steps:
[0217] S808, the second intelligent decision-making module sends the state information and AI decision information to the first data collection and training module;
[0218] S809, the first data collection and training module receives feedback information.
[0219] The feedback information is used to indicate the data used for AI training. Depending on the AI algorithm, the feedback information received by the data collection and training module is also different.
[0220] For example, if the AI algorithm adopted by the first AI entity is reinforcement learning, then the first data collection and training module receives the reward information sent by the terminal device or the performance tracking module. Then S809 may include two parallel steps S809a and S809b. Among them, S809a is the terminal device sending feedback information to the first data collection and training module; S809b is the first performance tracking module sending feedback information to the first data collection and training module.
[0221] For another example, if the AI algorithm used by the first AI entity is supervised learning, then the first data collection and training module receives the label information sent by the terminal device. Then S809 is for the first data collection and training module to receive the label information sent by the terminal device.
[0222] It should be noted that there is no order in which S807 and S808 are executed, that is, S807 and S808 can be executed at the same time.
[0223] Optionally, when the local second AI entity is an external device of the terminal device, after S806, the above processing flow may further include the following steps:
[0224] S8071, the second intelligent decision module sends AI decision information of the terminal device to the terminal device;
[0225] S808a, the terminal device executes the decision according to the AI decision information.
[0226] Among them, S8071 indicates that this step is executed after S806, replacing the original S807. S808a indicates that this step has no execution order with S808, that is, S808a can be executed at the same time as S808.
[0227] The following is a specific example Figure 8 The process of interaction between the terminal device and the first AI entity and the information of interaction are described in detail. Among them, the AI function implemented by the terminal device in this example is to use AI for channel decoding. Then the above S801 to S809 may specifically include the following steps:
[0228] The second intelligent decision module sends a request message to the first intelligent decision module, where the request message is used to request a channel decoding model;
[0229] The first intelligent decision module sends channel decoding model information to the second intelligent decision module;
[0230] The second intelligent decision module determines the channel decoding model of the terminal device according to the channel decoding model information;
[0231] The second preprocessing module receives a signal, which is data to be decoded;
[0232] The second preprocessing module preprocesses the signal to obtain a log-likelihood ratio of the signal;
[0233] The second preprocessing module sends the log-likelihood ratio of the signal to the second intelligent decision-making module;
[0234] The second intelligent decision module inputs the log-likelihood ratio of the signal into the channel decoding model of the terminal device for inference to obtain the decoding data of the signal;
[0235] The terminal device uses the decoded data of the signal.
[0236] Optionally, if the AIUpdateType parameter value of the terminal device is type 1 or type 2, the following steps are also included:
[0237] The second intelligent decision module sends the log-likelihood ratio of the signal and the decoded data of the signal to the first data collection and training module;
[0238] The first data collection and training module receives label information, which includes correct decoding data; or, the first data collection and training module receives reward information, which is 1 when the decoding is correct and 0 when the decoding fails.
[0239] The embodiment of the present application provides an information processing method, which defines another basic interaction mode between a first AI entity and a terminal device. When the terminal device has AI reasoning capability, the terminal device can perform reasoning based on the first AI model to obtain AI decision information of the terminal device, thereby realizing the corresponding AI function.
[0240] See also Fig. 9 , Fig. 9 A flowchart of another information processing method provided in an embodiment of the present application. Fig. 9 The information processing method flow is implemented by the interaction between the first AI entity and the terminal device, and may include the following steps:
[0241] S901, the terminal device sends second AI model information to the first AI entity; correspondingly, the first AI entity receives the second AI model information sent by the terminal device;
[0242] S902, the first AI entity updates the first AI model information according to the second AI model information;
[0243] S903, the first AI entity sends updated first AI model information to the terminal device; correspondingly, the terminal device receives the updated first AI model information sent by the first AI entity.
[0244] This embodiment defines a basic interaction mode between the first AI entity and the terminal device when the terminal device has AI training capability. That is, when the AICapabilityClass parameter value of the terminal device is class 2, the terminal device can train the AI model.
[0245] The second AI model information is the AI model information in the terminal device or the second AI entity. Similar to the first AI model information, the second AI model information may be the second AI model or the related parameters of the second AI model, which is not limited in this embodiment.
[0246] The first AI model and / or the second AI model are obtained by training the corresponding first data collection and training module and / or the second data collection and training module. For example, when the first AI model and / or the second AI model adopts a neural network, the first AI model and / or the second AI model can be trained by using a neural network training method.
[0247] For example, the data collection and training module can randomly initialize a neural network. Each training is the process of using existing data to obtain a new neural network from the weight matrix and bias vector of random neurons. During the training process, the output of the neural network can be evaluated using a loss function, and the error can be back-propagated. The gradient descent method can be used to iteratively optimize until the loss function reaches the minimum value. In other words, the data collection and training module can train the AI model through the above iterative optimization process to obtain a better AI model.
[0248] Optionally, the second AI model information does not include user data of the terminal device. In other words, the second AI model information sent by the terminal device to the first AI entity has nothing to do with the data of the terminal device itself, which is beneficial to the privacy protection of the terminal device.
[0249] Optionally, the second AI model information may also include user data of the terminal device to make the trained AI model better and facilitate obtaining more applicable AI decision-making information.
[0250] In one example, see Fig.10 , Fig.10 A schematic diagram of a federated learning process provided in an embodiment of the present application. Fig.10 The federated learning process shown is an example of a specific application of a basic interaction method between the first AI entity and the terminal device when the terminal device has AI training capabilities. The federated learning process includes the following steps:
[0251] S1001, the second intelligent decision-making module sends an AI training data request message to the second data collection and training module;
[0252] S1002, the second data collection and training module sends second AI training data to the second intelligent decision-making module;
[0253] S1003, the second intelligent decision-making module trains a second AI model according to the second AI training data;
[0254] S1004, the second intelligent decision-making module sends second AI model information to the first data collection and training module;
[0255] S1005, the first intelligent decision-making module sends an AI training data request message to the first data collection and training module;
[0256] S1006, the first data collection and training module sends the first AI training data to the first intelligent decision-making module;
[0257] S1007, the first intelligent decision-making module trains a first AI model according to the first AI training data;
[0258] S1008, the first intelligent decision-making module sends the trained first AI model information to the second intelligent decision-making module.
[0259] The first AI training data refers to the AI training data in the first AI entity, and the second AI training data refers to the AI training data in the terminal device. The first AI model refers to the AI model in the first AI entity, and the second AI model refers to the AI model in the terminal device.
[0260] Among them, the step of the second intelligent decision module sending the second AI model information to the first data collection and training module can be triggered on a timed basis. That is, one or more local terminal devices can upload one or more second AI model information to the cloud on a regular basis, and the cloud can save the locally uploaded second AI model information.
[0261] After the first AI entity in the cloud trains and updates the first AI model, the first AI entity can send the trained first AI model information to the local. The local will then train and update the first AI model, and the cycle will continue. The cycle process can be an infinite loop, or a threshold (such as a loss function) can be set. When the loss function is less than the threshold, the loop stops and the federated learning process ends.
[0262] In one example, when the terminal device has AI training capability, the terminal device can also perform local AI training. That is, the above AI training interaction process can be an interaction between modules inside the terminal device, and the second AI model information is obtained through AI training.
[0263] Similarly, since the first AI entity has AI training capabilities, the above-mentioned AI training interaction process can also be the interaction between modules within the first AI entity, and the first AI model information is obtained through AI training.
[0264] The following is a detailed example of local training of a terminal device or cloud training of the first AI entity.
[0265] See also Fig.11 , Fig.11 A flowchart of an AI training process provided in an embodiment of the present application. For ease of description, Fig.10 The intelligent decision-making module and / or the data collection and training module may refer to the first / second intelligent decision-making module, and / or the first / second data collection and training module.
[0266] akin, Fig.11 The AI training data and / or AI model in the description may refer to the first / second AI training data, and / or, the first / second AI model.
[0267] S1101, the intelligent decision-making module sends an AI training data request message to the data collection and training module;
[0268] S1102, the data collection and training module sends AI training data to the intelligent decision-making module;
[0269] S1103, the intelligent decision-making module trains the AI model based on the AI training data.
[0270] Among them, the AI training data may include but is not limited to AI decision information, status information or feedback information, etc. For example, when the above-mentioned AI training process is an AI training process inside the terminal device, the second intelligent decision module can update the second AI model according to the status information. For another example, when the above-mentioned AI training process is an AI training process inside the first AI entity, the first intelligent decision module can update the first AI model according to the AI decision information.
[0271] In one example, when the terminal device has AI reasoning and AI training capabilities, the terminal device can implement the AI reasoning and AI training process through the internal module. That is, when the AICapabilityClass parameter value of the terminal device is class 3, the terminal device can train the AI model and perform AI reasoning to obtain AI decision information.
[0272] The process of AI reasoning and AI training performed by the terminal device is obtained by combining the process of AI reasoning performed by the terminal device in the previous embodiment with the process of AI training performed by the terminal device. For details, please refer to the previous embodiment. Figure 8 and Fig.11 The detailed description of the illustrated embodiment will not be repeated here.
[0273] The embodiment of the present application provides an information processing method, which defines another basic interaction mode between a first AI entity and a terminal device. When the terminal device has AI training capability, the terminal device can train and update the local first AI model, or interact with the first AI entity in the cloud to train and update the first AI model, so that the AI model is more suitable for different application scenarios.
[0274] Based on the description in the above embodiments, the specific implementation method of the information processing method described in the embodiments of the present application when applied to different scenarios is described in detail below.
[0275] In one example, it is assumed that the AI algorithm adopted by the terminal device or the first AI entity is DRL. In DRL, the reward function of the system can be used as a performance indicator indicating the final convergence of the algorithm. Among them, the DRL online learning process can be implemented by the interaction between the terminal device and the first AI entity, or it can be implemented by an internal module of the terminal device with AI reasoning capability and / or AI training capability.
[0276] The following is a detailed description taking the interaction between the terminal device and the first AI entity as an example.
[0277] See also Fig.12 , Fig.12 A schematic diagram of a DRL online learning process provided in an embodiment of the present application. The DRL online learning process includes the following steps:
[0278] S1201, the first data collection and training module sends a reward function request message to the first performance tracking module;
[0279] S1202, the first performance tracking module sends a reward function to the first data collection and training module;
[0280] S1203a, the terminal device sends reward information to the first data collection and training module;
[0281] S1203b, the first performance tracking module sends reward function update indication information to the first data collection and training module;
[0282] S1204: The first data collection and training module updates the reward function according to the reward information.
[0283] Among them, the first performance tracking module can monitor the long-term key performance indicators (KPI) of the system, and the KPI can be used to guide the first data collection and training module to generate a reward function R (θ, φ). Among them, R represents the reward; the target parameter θ is the performance data obtained by the terminal device executing the AI decision information, such as throughput, packet loss rate, etc. Among them, the weight value φ of the target parameter is determined by the first AI entity based on the performance data of one or more terminal devices, and is used to indicate the weights of different short-term KPIs. In other words, the weight value φ of the target parameter can be obtained by the first performance tracking module in the first AI entity to monitor the performance of all terminal devices in the system for a long time.
[0284] There is no execution order for S1203a and S1203b, that is, S1203a can be executed simultaneously with S1203b.
[0285] Optionally, S1203b may occur regularly, or factors such as environmental changes may cause the AI model to become unsuitable, thereby triggering the update of the reward function. For example, in the process of long-term KPI deterioration triggering the adaptive process of the reward function, the first data collection and training module may send a reward function request message to the first performance tracking module to request an update of the reward function.
[0286] The following is a specific example to illustrate how the system adaptively adjusts the reward function during DRL scheduling.
[0287] Assume that the reward function is R(θ,φ)=α×thp+β×jfi+γ×pdr, where the target parameter θ={thp,jfi,pdr} includes three types of performance data, which represent throughput, fairness parameter and packet loss rate respectively. φ={α,β,γ} includes the weights of the above three types of performance data, assuming the initial value φ={1,1,1}. If the system runs for a period of time and PMF monitors fairness deterioration due to an emergency, the reward function is triggered to update so that the weights of the above three types of performance data are updated to φ={1,2,1}.
[0288] In one example, it is assumed that the AI algorithm used by the terminal device or the first AI entity is DRL. In order to avoid the catastrophic impact of exploration on the system in DRL online learning, an embodiment of the present application provides a decision early stopping technology, which can predict the system performance through a performance tracking module and determine whether catastrophic performance loss will occur, thereby avoiding the catastrophic impact of exploration on the system as early as possible.
[0289] See also Fig.13 , Fig.13A flowchart of a decision early stopping technology provided in an embodiment of the present application. The decision early stopping technology process can be implemented by the interaction between the terminal device and the first AI entity, or it can be implemented by an internal module of the terminal device with AI reasoning capability and / or AI training capability.
[0290] That is to say, Fig.13 The multiple AI functional modules in the may be functional modules in a first AI entity in the cloud, or may be functional modules in a second AI entity inside or external to a local terminal device, which is not limited in this embodiment.
[0291] S1301, the terminal device sends observation information to the preprocessing module;
[0292] S1302, a preprocessing module preprocesses the observation information to obtain corresponding status information;
[0293] S1303, the preprocessing module sends status information to the intelligent decision-making module;
[0294] S1304, the intelligent decision module performs model reasoning to obtain AI decision information of the terminal device;
[0295] S1305, the performance tracking module predicts the system performance and obtains decision mask information and / or penalty information;
[0296] S1306, the performance tracking module sends decision mask information to the intelligent decision module;
[0297] S1307, the intelligent decision module performs mask processing on the AI decision information according to the decision mask information to obtain the masked AI decision information;
[0298] S1308a, the performance tracking module sends one or more of status information, decision mask information and penalty information to the data collection and training module;
[0299] S1308b, the intelligent decision module sends the masked AI decision information to the terminal device;
[0300] S1309a, the terminal device executes a decision according to the AI decision information after mask processing;
[0301] S1309b, the intelligent decision module sends the status information and the AI decision information after mask processing to the data collection and training module;
[0302] S1310, the terminal device sends feedback information to the data collection and training module.
[0303] The performance tracking module needs to have the ability to predict long-term performance. For example, the performance tracking module needs to determine whether catastrophic performance loss will occur based on the current state of the system and the decisions made by the model.
[0304] Optionally, the decision early stopping technology described in this embodiment may also include a model synchronization step. That is, before the terminal device sends the observation information to the preprocessing module, the following steps may also be included: the intelligent decision module sends the AI model information to the performance tracking module.
[0305] Whether the model synchronization step and step S1308a are necessary depends on the prediction capability of the performance tracking module. That is, if the prediction capability of the performance tracking module is strong, then the above-mentioned model synchronization and step S1308a are both optional steps.
[0306] The decision mask information is used to mask the AI decision information so that the part that reduces system performance is processed. For example, if one or more users accessing the system will significantly reduce system performance, the performance tracking module can reduce the weight of the AI decision of the one or more users to the minimum, and the one or more users will no longer execute the corresponding AI decision. The decision mask information can be obtained directly based on the prediction results, or it can be obtained through the backup algorithm in the performance tracking module.
[0307] Optionally, after the performance tracking module obtains the decision mask information and / or penalty information through prediction, the decision mask information and / or penalty information may be used as a training sample and sent to the data collection and training module.
[0308] It can be seen that compared with the solution without early stopping decision, Fig.13 In the decision early stopping scheme, two sets of training samples will be obtained by sampling once, which improves the sampling efficiency of DRL.
[0309] The following takes the DRL scheduling process as an example to illustrate Fig.13 The decision-making early stopping scheme described.
[0310] For example, for the scheduling decision of the system for 5 users, the decision weights generated by DRL may be {1.5, 1.1, 1.2, 0.2, 0}. However, in one possible case, the expected throughput of user 0 and user 4 is 0, so scheduling user 0 and / or user 4 in this case will inevitably lead to a waste of system resources.
[0311] After predicting this situation, the performance tracking module can generate a decision mask. For example, the decision masks of the five users are {0, 1, 1, 1, 0}. Based on the above decision masks, the performance tracking module can obtain the decision weights after masking {0, 1.1, 1.2, 0.2, 0}. Then, based on the decision weight information, the system will schedule user 2. It can be seen that this scheduling is conducive to reducing the waste of system resources and optimizing the overall performance of the system.
[0312] In one example, the embodiment of the present application provides an application example of a DRL algorithm deployed in a cell. After the DRL algorithm is deployed online in each cell, the DRL algorithm can be divided into two stages: the imitation learning stage of the intelligent agent and the online reinforcement learning stage of the intelligent agent. Fig.14 The intelligent agent described in this example may be a first AI entity or a local second AI entity with AI reasoning capability and / or AI training capability.
[0313] Among them, the imitation learning stage of the intelligent agent is the first stage. In the first stage, the intelligent agent needs training data to initialize the training of the intelligent agent. For example, the base station uses the traditional scheduling algorithm for initialization training and saves the trajectory information of the entire scheduling process, so that the base station can perform supervised learning based on the saved trajectory information, thereby realizing the initialization training of the base station.
[0314] Optionally, in order to solve the problem that reinforcement learning requires a large amount of interactive data, the embodiment of the present application proposes that the imitation learning stage of the intelligent agent can be a virtual cell (vCell) assisted training process. Among them, the first AI entity can obtain the basic real information of the cell for training and generating vCell. Among them, the basic real information of the cell may include but is not limited to the location information, mobility information, service information, channel information and other related information of the terminal equipment in the cell. vCell is generally composed of a neural network.
[0315] For example, in the imitation learning stage of the intelligent agent, the first AI entity can use the generative adversarial network (GAN) algorithm. The principle of the GAN training process is to first fix the generative network and train the discriminative network so that it can distinguish between real data and virtual data; then fix the discriminative network and train the generative network so that the virtual data generated by the generative network is as similar as possible to the real data, and then alternate until convergence.
[0316] Based on the above principle, the first AI entity can obtain real data and virtual data obtained from the generated network, and alternately train the identification network and the generated network. For example, the first AI entity can obtain relevant information such as location information, mobility information, service information, channel information, etc. of the terminal device in the cell, and input the above relevant information into the generated network to obtain virtual data. The data collection and training module in the first AI entity can train the virtual data, that is, alternately train the identification network and the generated network according to the virtual data, so as to generate a vCell, such as Fig.15 shown.
[0317] Optionally, vCell can be further decomposed into multiple virtual user equipment (virtual UE, vUE) and virtual environment (virtual environment, vEnv). Among them, vUE is used to model the UE, and vEnv is used to model the environment. For example, vUE can use the multi-agentGAN algorithm to determine the location information, mobility information, and service information of the UE. For another example, vEnv can use the conditional GAN algorithm to generate a corresponding transmission channel based on the UE location information, terrain information, weather information, etc.
[0318] The second stage is the online reinforcement learning stage of the agent. In this second stage, the agent can interact with the trained vCell. Since the cost and risk of interacting with vCell are much lower than interacting with the real cell, the introduction of vCell will greatly improve the convergence speed of the agent.
[0319] Optionally, the agent can also be trained online. That is, the agent in this example can be trained based on Fig.12 and 13 The deep reinforcement learning process shown in the figure is used for online training. For the specific implementation process, please refer to Fig.12 and Fig.13 The description in the illustrated embodiment will not be repeated here.
[0320] Optionally, the present application embodiment further provides a training terminal for assisting in the online training of the DRL algorithm. For example, see Fig.16 , Fig.16 A schematic diagram of a training terminal deployed in a real cell provided in an embodiment of the present application. In the real cell, multiple and / or multiple training terminals (training UE, tUE) can be deployed, and each tUE can interact with the intelligent agent. The interaction method may include but is not limited to interaction through imitation learning, interaction through reinforcement learning, etc. Fig.16 shown.
[0321] Among them, tUE has the following characteristics: directly interacting with the reinforcement learning algorithm; being able to obtain a large number of training samples when idle; being able to collect non-communication-perceivable data within the cell; being able to provide enhanced coverage services; being a fixed-location device or a mobile device. In other words, tUE can be any one or more of the types of terminal devices described in the embodiments of the present application.
[0322] For example, tUE has the characteristic of acquiring a large number of training samples when idle, so tUE can collect a large number of training samples at night. For another example, tUE has the characteristic of collecting non-communication-perceivable data in the cell, so tUE can collect weather information, terrain information, obstacle information, etc. that can be used as training sample data and used for vCell modeling. For another example, tUE has the characteristic of providing enhanced coverage services, so tUE can also be small base stations, drones and other equipment.
[0323] It can be seen that tUE can effectively obtain training data without affecting actual business, significantly improving training efficiency.
[0324] The following combination Figures 17 to 24 The relevant equipment of the embodiments of the present application is described in detail.
[0325] The present application embodiment provides a first AI entity, such as Fig.17 The first AI entity is used to implement the method executed by the first AI entity in the above method embodiment, and specifically includes a preprocessing module 1701 and an intelligent decision module 1702.
[0326] Among them, the preprocessing module 1701 is used to receive observation information sent by the terminal device, and the observation information indicates the data used for AI decision-making. The intelligent decision module 1702 is used to determine the AI decision information of the terminal device based on the observation information and the first AI model. The intelligent decision module 1702 is also used to send the AI decision information to the terminal device.
[0327] In one implementation, the preprocessing module 1701 is further used to receive AI information of the terminal device, where the AI information includes AI capability parameters.
[0328] In one implementation, if the AI capability parameter of the terminal device indicates that the terminal device has no AI capability, the preprocessing module 1701 is used to receive observation information sent by the terminal device.
[0329] In one implementation, the preprocessing module 1701 is further used to preprocess the observation information to obtain corresponding state information. The intelligent decision module 1702 is further used to input the state information into the first AI model for reasoning to obtain AI decision information of the terminal device.
[0330] Exemplarily, the preprocessing module 1701 may be used to perform Figure 5 S501 and Figure 6 From S601 to S603, the intelligent decision module 1702 is used to execute Figure 5 S502 and S503 in, and Figure 6 S604, S605 and S607 in.
[0331] It can be understood that the methods executed by the various modules shown above are only examples, and the specific steps executed by each module can refer to the methods introduced above.
[0332] In one implementation, Fig.17 The related functions implemented by each module in the can be realized by the transceiver and the processor. Fig.18 , Fig.18 is a structural diagram of a first AI entity provided in an embodiment of the present application. The first AI entity may be a device (eg, a chip) capable of executing the information processing function described in the embodiment of the present application.
[0333] The first AI entity may include a transceiver 1801, at least one processor 1802, and a memory 1803. The transceiver 1801, the processor 1802, and the memory 1803 may be interconnected via one or more communication buses, or may be connected via other means.
[0334] Among them, the transceiver 1801 can be used to send information or receive information. It can be understood that the transceiver 1801 is a general term and can include a receiver and a transmitter. For example, the receiver is used to receive observation information sent by the terminal device. For another example, the transmitter is used to send AI decision information to the terminal device.
[0335] In one implementation, the transceiver 1801 may be used to implement Fig.17 Part or all of the functions of the preprocessing module and intelligent decision-making module shown.
[0336] The processor 1802 may be used to process the information. For example, the processor 1802 may call the program code stored in the memory 1803 to determine the AI decision information of the terminal device according to the observed information and the first AI model.
[0337] The processor 1802 may include one or more processors, for example, the processor 1802 may be one or more central processing units (CPUs), network processors (NPs), hardware chips, or any combination thereof. When the processor 1802 is a CPU, the CPU may be a single-core CPU or a multi-core CPU.
[0338] In one implementation, the processor 1802 may be configured to implement Fig.17 Part or all of the functions of the preprocessing module and intelligent decision-making module shown.
[0339] The memory 1803 is used to store program codes, etc. The memory 1803 may include a volatile memory, such as a random access memory (RAM); the memory 1803 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); the memory 1803 may also include a combination of the above-mentioned types of memories.
[0340] The processor 1802 and the memory 1803 may be coupled via an interface or may be integrated together, which is not limited in this embodiment.
[0341] The above transceiver 1801 and processor 1802 can be used to implement the information processing method in the embodiment of the present application, wherein the specific implementation method is as follows:
[0342] The transceiver 1801 is used to receive observation information sent by the terminal device, where the observation information indicates data used for AI decision making. The processor 1802 is used to determine the AI decision information of the terminal device based on the observation information and the first AI model. The transceiver 1801 is also used to send the AI decision information to the terminal device.
[0343] In one implementation, the transceiver 1801 is further configured to receive AI information of a terminal device, where the AI information includes AI capability parameters.
[0344] In one implementation, if the AI capability parameter of the terminal device indicates that the terminal device has no AI capability, the transceiver 1801 is used to receive the observation information sent by the terminal device.
[0345] In one implementation, the processor 1802 is also used to pre-process the observation information to obtain corresponding status information, and then input the status information into the first AI model for reasoning to obtain AI decision information of the terminal device.
[0346] Exemplarily, the transceiver 1801 may be used to perform Figure 5 S501 and S503 in , and Figure 6 In S601, S603 and S605, the processor 1802 is used to execute Figure 5 S502 in, and Figure 6 S602 and S604 in.
[0347] It can be understood that the methods executed by the various modules shown above are only examples, and the specific steps executed by each module can refer to the methods introduced above.
[0348] The present application embodiment provides another first AI entity, such as Fig.19 As shown. The first AI entity is used to implement the method executed by the first AI entity in the above method embodiment, specifically including an intelligent decision-making module 1901, a preprocessing module 1902, a data collection and training module 1903 and a performance tracking module 1904. Among them, the intelligent decision-making module 1901 is used to receive the second AI model information sent by the terminal device, and the second AI model information does not include the user data of the terminal device. The intelligent decision-making module 1901 is also used to update the first AI model information according to the second AI model information; the first AI model information is the AI model information of the first AI entity. The intelligent decision-making module 1901 is also used to send the updated first AI model information to the terminal device.
[0349] In one implementation, the intelligent decision module 1901 is further configured to receive a request message sent by a terminal device, the request message being used to request the first AI model information. The intelligent decision module 1901 is further configured to send the first AI model information to the terminal device.
[0350] In one implementation, the preprocessing module 1902 is used to receive AI information of a terminal device, where the AI information includes AI capability parameters.
[0351] In one implementation, if the AI capability parameter indicates that the terminal device has AI reasoning capability, the intelligent decision-making module 1901 is also used to receive AI decision information and status information sent by the terminal device; wherein the AI decision information is obtained by the terminal device inputting the status information into the second AI model for reasoning, and the status information is obtained by the terminal device based on the observation information; the observation information indicates the data used for AI decision-making.
[0352] In one implementation, the preprocessing module 1902 is further configured to receive AI information from a terminal device, the AI information including an AI update parameter. If the AI update parameter indicates a scheduled AI update or an event-triggered AI update, the data collection and training module 1903 is configured to receive feedback information indicating data used for AI training.
[0353] In one implementation, the intelligent decision module 1901 is also used to update the first AI model based on AI training data; wherein the AI training data includes one or more of AI decision information, status information or feedback information.
[0354] In one implementation, the feedback information includes reward information; the reward information is used to update the first AI model.
[0355] In one implementation, the reward information is determined according to a reward function. The reward function is determined according to a target parameter θ and a weight value φ of the target parameter. The target parameter is performance data obtained by executing the AI decision information by the terminal device, and the weight value of the target parameter is determined by the first AI entity according to the performance data of one or more terminal devices.
[0356] In one implementation, the performance tracking module 1904 is used to send reward information to the data collection and training module 1903 .
[0357] Exemplarily, the intelligent decision module 1901 can be used to execute Fig. 9 S901 to S903 in , and Fig.10 S1005 to S1008 in the above, the pre-processing module 1902 is used to perform the step of receiving the AI information of the terminal device in the above embodiment, and the data collection and training module 1903 is used to perform Figure 8 S809a and S809b in Fig.12 S1203a, S1203b and S1204 in Fig.13 S1309b and S1310 in.
[0358] It can be understood that the methods executed by the various modules shown above are only examples, and the specific steps executed by each module can refer to the methods introduced above.
[0359] In one implementation, Fig.19 The related functions implemented by each module in the can be realized by the transceiver and the processor. Fig. 20 , Fig. 20 is a structural diagram of a first AI entity provided in an embodiment of the present application. The first AI entity may be a device (eg, a chip) capable of executing the information processing function described in the embodiment of the present application.
[0360] The first AI entity may include a transceiver 2001, at least one processor 2002, and a memory 2003. The transceiver 2001, the processor 2002, and the memory 2003 may be interconnected via one or more communication buses, or may be connected via other means.
[0361] Among them, the transceiver 2001 can be used to send information or receive information. It can be understood that the transceiver 2001 is a general term and can include a receiver and a transmitter. For example, the receiver is used to receive the second AI model information sent by the terminal device. For another example, the transmitter is used to send the updated first AI model information to the terminal device.
[0362] In one implementation, the transceiver 2001 may be used to implement Fig.19 Part or all of the functions of the intelligent decision-making module 1901, preprocessing module 1902, data collection and training module 1903 and performance tracking module 1904 shown.
[0363] The processor 2002 may be used to process the information. For example, the processor 2002 may call the program code stored in the memory 2003 to update the first AI model information according to the second AI model information.
[0364] The processor 2002 may include one or more processors, for example, the processor 2002 may be one or more central processing units (CPUs), network processors (NPs), hardware chips, or any combination thereof. When the processor 2002 is a CPU, the CPU may be a single-core CPU or a multi-core CPU.
[0365] In one implementation, the processor 2002 may be configured to implement Fig.19 Part or all of the functions of the intelligent decision-making module 1901, preprocessing module 1902, data collection and training module 1903 and performance tracking module 1904 shown.
[0366] The memory 2003 is used to store program codes, etc. The memory 2003 may include a volatile memory, such as a random access memory (RAM); the memory 2003 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); the memory 2003 may also include a combination of the above-mentioned types of memories.
[0367] The processor 2002 and the memory 2003 may be coupled via an interface or may be integrated together, which is not limited in this embodiment.
[0368] The above transceiver 2001 and processor 2002 may be used to implement the information processing method in the embodiment of the present application, wherein the specific implementation is as follows:
[0369] The transceiver 2001 is configured to receive second AI model information sent by a terminal device, where the second AI model information does not include user data of the terminal device;
[0370] Processor 2002, used to update first AI model information according to second AI model information; the first AI model information is AI model information of the first AI entity;
[0371] The transceiver 2001 is also used to send the updated first AI model information to the terminal device.
[0372] In one implementation, the transceiver 2001 is further configured to:
[0373] Receive a request message sent by a terminal device, where the request message is used to request first AI model information;
[0374] The first AI model information is sent to the terminal device.
[0375] In one implementation, the transceiver 2001 is further configured to:
[0376] AI information of a terminal device is received, where the AI information includes AI capability parameters.
[0377] In one implementation, if the AI capability parameter indicates that the terminal device has AI reasoning capability, the transceiver 2001 is further configured to:
[0378] Receive AI decision information and status information sent by the terminal device; wherein the AI decision information is obtained by the terminal device inputting the status information into the second AI model for inference, and the status information is obtained by the terminal device based on the observation information; the observation information indicates the data used for AI decision making.
[0379] In one implementation, the transceiver 2001 is further configured to receive AI information of a terminal device, where the AI information includes an AI update parameter;
[0380] If the AI update parameter indicates a timed AI update or an event-triggered AI update, the transceiver 2001 is further configured to receive feedback information, where the feedback information is used to indicate data used for AI training.
[0381] In one implementation, the processor 2002 is further configured to:
[0382] Update the first AI model according to the AI training data; wherein the AI training data includes one or more of AI decision information, state information or feedback information.
[0383] In one implementation, the feedback information includes reward information; the reward information is used to update the first AI model.
[0384] In one implementation, the reward information is determined according to a reward function. The reward function is determined according to a target parameter θ and a weight value φ of the target parameter. The target parameter is performance data obtained by executing the AI decision information by the terminal device, and the weight value of the target parameter is determined by the first AI entity according to the performance data of one or more terminal devices.
[0385] Exemplarily, the transceiver 1801 may be used to perform Fig. 9 S901 and S903, Fig.10 S1004 in Fig.12 S1201 to S1203a in, and Fig.13 S1301 and S1308b in the embodiment. The processor 1802 is used to execute Fig. 9 S902 in Fig.10 S1005 to S1007 in, and Fig.12 Medium S1204.
[0386] It can be understood that the methods executed by the various modules shown above are only examples, and the specific steps executed by each module can refer to the methods introduced above.
[0387] The present application embodiment provides a terminal device, such as Fig.21As shown. The terminal device is used to implement the method executed by the terminal device in the above method embodiment, and specifically includes a transceiver module 2101 and a processing module 2102. Among them, the transceiver module 2101 is used to send observation information to the first AI entity, and the observation information indicates the data used for AI decision-making. The transceiver module 2101 is also used to receive AI decision information of the terminal device sent by the first AI entity. The processing module 2102 is used to execute a decision according to the AI decision information.
[0388] In one implementation, the transceiver module 2101 is further configured to send AI information of the terminal device to the first AI entity, where the AI information includes an AI capability parameter, wherein the AI capability parameter indicates that the terminal device has no AI capability.
[0389] In one implementation, the AI decision information of the terminal device is obtained by the first AI entity inputting the state information into the first AI model for reasoning; the state information is obtained by the first AI entity based on the observation information.
[0390] Exemplarily, the transceiver module 2101 may be used to execute Figure 5 S501 and S503 in Figure 6 S601 and S605 in the processing module 2102 are used to execute Figure 6 S606 in.
[0391] It can be understood that the methods executed by the various modules shown above are only examples, and the specific steps executed by each module can refer to the methods introduced above.
[0392] In one implementation, Fig.21 The related functions implemented by each module in the can be realized by the transceiver and the processor. Fig. 22 , Fig. 22 It is a structural diagram of a terminal device provided in an embodiment of the present application. The terminal device may be a device (such as a chip) capable of executing the information processing function described in the embodiment of the present application.
[0393] The terminal device may include a transceiver 2201, at least one processor 2202, and a memory 2203. The transceiver 2201, the processor 2202, and the memory 2203 may be interconnected via one or more communication buses, or may be connected via other means.
[0394] Among them, the transceiver 2201 can be used to send information or receive information. It can be understood that the transceiver 2201 is a general term and can include a receiver and a transmitter. For example, the receiver is used to receive AI decision information of the terminal device sent by the first AI entity. For another example, the transmitter is used to send observation information to the first AI entity.
[0395] In one implementation, the transceiver 2201 may be used to implement Fig.21 Part or all of the functions of the transceiver module 2101 shown.
[0396] The processor 2202 may be used to process information. For example, the processor 2202 may call program codes stored in the memory 2203 to implement decision making based on AI decision information.
[0397] The processor 2202 may include one or more processors, for example, the processor 2202 may be one or more central processing units (CPUs), network processors (NPs), hardware chips, or any combination thereof. When the processor 2202 is a CPU, the CPU may be a single-core CPU or a multi-core CPU.
[0398] In one implementation, the processor 2202 may be configured to implement Fig.21 Some or all of the functionality of processing module 2102 as shown.
[0399] The memory 2203 is used to store program codes, etc. The memory 2203 may include a volatile memory, such as a random access memory (RAM); the memory 2203 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); the memory 2203 may also include a combination of the above-mentioned types of memories.
[0400] The processor 2202 and the memory 2203 may be coupled via an interface or may be integrated together, which is not limited in this embodiment.
[0401] The above transceiver 2201 and processor 2202 can be used to implement the information processing method in the embodiment of the present application, wherein the specific implementation method is as follows:
[0402] The transceiver 2201 is used to send observation information to the first AI entity, where the observation information indicates data used for AI decision making. The transceiver 2201 is also used to receive AI decision information of a terminal device sent by the first AI entity. The processor 2202 is used to execute a decision according to the AI decision information.
[0403] In one implementation, the transceiver 2201 is further configured to send AI information of the terminal device to the first AI entity, where the AI information includes an AI capability parameter, wherein the AI capability parameter indicates that the terminal device has no AI capability.
[0404] In one implementation, the AI decision information of the terminal device is obtained by the first AI entity inputting the state information into the first AI model for reasoning; the state information is obtained by the first AI entity based on the observation information.
[0405] Exemplarily, the transceiver 2201 may be used to perform Figure 5 S501 and S503 in Figure 6 S601 and S605 in the embodiment. The processor 2202 is used to execute Figure 6 S606 in.
[0406] It can be understood that the methods executed by the various modules shown above are only examples, and the specific steps executed by each module can refer to the methods introduced above.
[0407] The present application embodiment provides another terminal device, such as Fig.23 As shown. The terminal device is used to implement the method executed by the terminal device in the above method embodiment, and specifically includes a transceiver module 2301 and a processing module 2302. Among them, the transceiver module 2301 is used to send a request message to the first AI entity, and the request message is used to request the first AI model information. The transceiver module 2301 is also used to receive the first AI model information sent by the first AI entity. The processing module 2302 is used to input the state information into the second AI model for reasoning to obtain the AI decision information of the terminal device; wherein the state information is determined based on the observation information; the observation information indicates the data used for AI decision-making; the second AI model is determined by the terminal device according to the first AI model information.
[0408] In one implementation, the transceiver module 2301 is further used to send AI information of the terminal device to the first AI entity, where the AI information includes an AI capability parameter, wherein the AI capability parameter indicates that the terminal device has AI reasoning capability.
[0409] In one implementation, the transceiver module 2301 is further configured to send AI decision information and status information to the first AI entity.
[0410] In one implementation, the AI information of the terminal device includes AI capability parameters and / or AI update parameters; the transceiver module 2301 is also used to send feedback information to the first AI entity if the AI update parameter indicates a scheduled AI update or an event-triggered AI update, and the feedback information is used to indicate the data used for AI training.
[0411] In one implementation, the processing module 2302 is also used to obtain a second AI model based on AI training data if the AI capability parameter indicates that the terminal device has AI training capability; wherein the AI training data includes one or more of AI decision information, status information or feedback information.
[0412] In one implementation, the transceiver module 2301 is further configured to send the second AI model information to the first AI entity. The transceiver module 2301 is further configured to receive the updated first AI model information sent by the first AI entity, where the updated first AI model information is determined by the first AI entity based on the second AI model information.
[0413] In one implementation, the feedback information includes reward information; the reward information is used to update the first AI model.
[0414] In one implementation, the reward information is determined according to a reward function. The reward function is determined according to a target parameter θ and a weight value φ of the target parameter. The target parameter is performance data obtained by executing the AI decision information by the terminal device, and the weight value of the target parameter is determined by the first AI entity according to the performance data of one or more terminal devices.
[0415] Exemplarily, the transceiver module 2301 may be used to execute Figure 7 S701 and S702 in Figure 8 S801 and S802 in Fig. 9 S901 and S903, Fig.10 S1004 and S1008 in Fig.12 S1203a in Fig.13 S1301 and S1308b in. Processing module 2302 is used to execute Figure 7 S703 in Figure 8 S803, S804 and S808 in Fig.10 S1003 in Fig.13 S1309a in.
[0416] It can be understood that the methods executed by the various modules shown above are only examples, and the specific steps executed by each module can refer to the methods introduced above.
[0417] In one implementation, Fig.23 The related functions implemented by each module in the can be realized by the transceiver and the processor. Fig.24 , Fig.24 It is a structural diagram of a terminal device provided in an embodiment of the present application. The terminal device may be a device (such as a chip) capable of executing the information processing function described in the embodiment of the present application.
[0418] The terminal device may include a transceiver 2401, at least one processor 2402, and a memory 2403. The transceiver 2401, the processor 2402, and the memory 2403 may be interconnected via one or more communication buses, or may be connected via other means.
[0419] Among them, the transceiver 2401 can be used to send information or receive information. It can be understood that the transceiver 2401 is a general term and can include a receiver and a transmitter. For example, the receiver is used to receive the first AI model information sent by the first AI entity. For another example, the transmitter is used to send a request message to the first AI entity.
[0420] In one implementation, the transceiver 2401 may be used to implement Fig.23 Part or all of the functions of the transceiver module 2301 shown.
[0421] The processor 2402 may be used to process information. For example, the processor 2402 may call program codes stored in the memory 2403 to implement decision making based on AI decision information.
[0422] The processor 2402 may include one or more processors, for example, the processor 2402 may be one or more central processing units (CPUs), network processors (NPs), hardware chips, or any combination thereof. When the processor 2402 is a CPU, the CPU may be a single-core CPU or a multi-core CPU.
[0423] In one implementation, the processor 2402 may be used to implement Fig.23 Some or all of the functionality of processing module 2302 as shown.
[0424] The memory 2403 is used to store program codes, etc. The memory 2403 may include a volatile memory, such as a random access memory (RAM); the memory 2403 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); the memory 2403 may also include a combination of the above-mentioned types of memories.
[0425] The processor 2402 and the memory 2403 may be coupled via an interface or may be integrated together, which is not limited in this embodiment.
[0426] The above transceiver 2401 and processor 2402 can be used to implement the information processing method in the embodiment of the present application, wherein the specific implementation method is as follows:
[0427] The transceiver 2401 is used to send a request message to the first AI entity, where the request message is used to request the first AI model information. The transceiver 2401 is also used to receive the first AI model information sent by the first AI entity. The processor 2402 is used to input the state information into the second AI model for reasoning to obtain AI decision information of the terminal device; wherein the state information is determined based on the observation information; the observation information indicates the data used for making the AI decision; and the second AI model is determined by the terminal device based on the first AI model information.
[0428] In one implementation, the transceiver 2401 is further configured to send AI information of the terminal device to the first AI entity, where the AI information includes an AI capability parameter, wherein the AI capability parameter indicates that the terminal device has AI reasoning capability.
[0429] In one implementation, the transceiver 2401 is further configured to send AI decision information and status information to the first AI entity.
[0430] In one implementation, the AI information of the terminal device includes an AI capability parameter and / or an AI update parameter. If the AI update parameter indicates a scheduled AI update or an event-triggered AI update, the transceiver 2401 is further configured to send feedback information to the first AI entity, where the feedback information is used to indicate data used for AI training.
[0431] In one implementation, the transceiver 2401 is also used to obtain a second AI model based on AI training data if the AI capability parameter indicates that the terminal device has AI training capability; wherein the AI training data includes one or more of AI decision information, status information or feedback information.
[0432] In one implementation, the transceiver 2401 is further configured to send the second AI model information to the first AI entity. The transceiver 2401 is further configured to receive updated first AI model information sent by the first AI entity, where the updated first AI model information is determined by the first AI entity based on the second AI model information.
[0433] In one implementation, the feedback information includes reward information; the reward information is used to update the first AI model.
[0434] In one implementation, the reward information is determined according to a reward function. The reward function is determined according to a target parameter θ and a weight value φ of the target parameter. The target parameter is performance data obtained by executing the AI decision information by the terminal device, and the weight value of the target parameter is determined by the first AI entity according to the performance data of one or more terminal devices.
[0435] Exemplarily, the transceiver 2401 may be used to perform Figure 7 S701 and S702 in Figure 8 S801 and S802 in Fig. 9 S901 and S903, Fig.10 S1004 and S1008 in Fig.12 S1203a in Fig.13 S1301 and S1308b in. Processor 2402 is used to execute Figure 7 S703 in Figure 8 S803, S804 and S808 in Fig.10 S1003 in Fig.13 S1309a in.
[0436] It can be understood that the methods executed by the various modules shown above are only examples, and the specific steps executed by each module can refer to the methods introduced above.
[0437] An embodiment of the present application provides a communication system, which includes the terminal device described in the above embodiment and a first AI entity.
[0438] An embodiment of the present application provides a computer-readable storage medium, which stores a program or instruction. When the program or instruction is executed on a computer, the computer executes the information processing method in the embodiment of the present application.
[0439] An embodiment of the present application provides a chip or a chip system, which includes at least one processor and an interface, the interface and the at least one processor are interconnected through lines, and the at least one processor is used to run a computer program or instruction to perform the information processing method in the embodiment of the present application.
[0440] The interface in the chip may be an input / output interface, a pin or a circuit, etc.
[0441] The chip system in the above aspects may be a system on chip (SOC) or a baseband chip, etc., wherein the baseband chip may include a processor, a channel encoder, a digital signal processor, a modem and an interface module, etc.
[0442] In one implementation, the chip or chip system described above in the present application further includes at least one memory, in which instructions are stored. The memory may be a storage module inside the chip, such as a register, a cache, etc., or a storage module of the chip (e.g., a read-only memory, a random access memory, etc.).
[0443] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.
[0444] The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from a computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server, or data center.
[0445] The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a high-density digital video disc (DVD)), or a semiconductor medium (e.g., a solid state drive (SSD)).
[0446] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0447] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. An information processing method, characterized in that: Applied to an intelligent decision-making module associated with a terminal device or an access network device, the method comprises: The intelligent decision-making module obtains state information, where the state information is determined based on observation information of the terminal device, where the observation information indicates data used for making AI decisions; The intelligent decision-making module inputs the status information into the AI model for reasoning to obtain AI decision information of the terminal device.
2. The method according to claim 1, characterized in that Also includes: The intelligent decision module sends the status information and / or the AI decision information.
3. The method according to claim 1, characterized in that The intelligent decision module performs mask processing on the AI decision information according to the decision mask information to obtain the masked AI decision information; The intelligent decision module sends the AI decision information after mask processing.
4. The method according to any one of claims 1 to 3, characterized in that: Applied to an intelligent decision-making module associated with the terminal device, the method further comprises: Sending a request message, where the request message is used to request first AI model information; receiving the first AI model information, where the first AI model information includes the first AI model and / or related parameters of the first AI model; The intelligent decision-making module inputs the state information into the AI model for reasoning to obtain the AI decision information of the terminal device, including: the intelligent decision-making module inputs the state information into the first AI model for reasoning to obtain the AI decision information of the terminal device.
5. The method according to any one of claims 1 to 4, characterized in that: Applied to an intelligent decision-making module associated with the terminal device, the method further comprises: Sending second AI model information, where the second AI model information is used to update the first AI model information, wherein the second AI model information does not include user data of the terminal device; Receive updated first AI model information.
6. The method according to any one of claims 1 to 5, characterized in that: The intelligent decision-making module sends feedback information to the data collection and training module, and the feedback information is used to indicate the data used for AI training.
7. The method according to claim 6, characterized in that The feedback information includes reward information, and the reward information is used to update the AI model.
8. The method according to claim 7, characterized in that The reward information is determined based on a reward function; the reward function is determined based on a target parameter θ and a weight value φ of the target parameter; the target parameter is performance data obtained when the terminal device executes the AI decision information, and the weight value of the target parameter is determined based on performance data of one or more terminal devices.
9. The method according to any one of claims 1 to 8, characterized in that: The method further comprises: The intelligent decision module updates the AI model according to AI training data; the AI training data includes at least one of the following: AI decision information, status information or feedback information.
10. The method according to claim 9, characterized in that The method further comprises: Receive the AI training data sent by the data collection and training module.
11. An information processing method, characterized in that: A data collection and training module associated with a terminal device or an access network device, characterized in that: Determining AI training data, wherein the AI training data includes at least one of the following: AI decision information, state information, or feedback information; The AI model is trained according to the AI training data, or the AI training data is sent, wherein the AI training data is used to train the AI model, and the AI model is used to infer based on the state information to obtain AI decision information.
12. The method according to claim 11, characterized in that Also includes: Receive the feedback information sent by the terminal device and / or the performance tracking module.
13. The method according to claim 11 or 12, characterized in that: Also includes: Receive the status information and / or the AI decision information sent by the intelligent decision module.
14. The method according to claim 13, characterized in that The AI decision information is AI decision information after masking according to the decision mask information; the method further includes: The decision mask information sent by the receiving performance tracking module.
15. The method according to any one of claims 11 to 14, characterized in that Before sending the AI training data, the method further includes: receiving an AI training data request message.
16. An information processing method, characterized in that: Applied to a performance tracking module associated with a terminal device or an access network device, the method comprises: Determine feedback information, where the feedback information is used to indicate AI training data, where the AI training data is used to train an AI model, where the AI model is used to infer based on the state information to obtain AI decision information; The feedback information is sent to the data collection and training module.
17. The method according to claim 16, characterized in that The feedback information includes reward information, and the reward information is used to update the AI model.
18. The method according to claim 17, characterized in that The reward information is determined based on a reward function; the reward function is determined based on a target parameter θ and a weight value φ of the target parameter; the target parameter is performance data obtained when the terminal device executes the AI decision information, and the weight value of the target parameter is determined by the performance tracking module monitoring the performance data of one or more terminal devices.
19. The method according to any one of claims 16 to 18, characterized in that Also includes: Predicting system performance to obtain decision mask information and / or penalty information, wherein the decision mask information is used to mask the AI decision information; as well as The decision mask information and / or penalty information is sent.
20. An information processing device, characterized in that: include: A module or unit for executing the method according to any one of claims 1 to 19.
21. An information processing device, characterized in that: The invention comprises a processor coupled to a memory for storing instructions, wherein when the processor executes the instructions, the method of any one of claims 1 to 10 is executed; or, the method of any one of claims 11 to 15 is executed, or the method of any one of claims 16 to 19 is executed.
22. A computer-readable storage medium, characterized in that: The method comprises a program or an instruction, which, when executed on a computer, enables the method of any one of claims 1 to 10 to be executed; or enables the method of any one of claims 11 to 15 to be executed, or enables the method of any one of claims 16 to 19 to be executed.
23. A computer program product comprising instructions, which, when executed on a computer, enables the method of any one of claims 1 to 10 to be executed; or enables the method of any one of claims 11 to 15 to be executed, or enables the method of any one of claims 16 to 19 to be executed.
