A network data processing method, device, equipment, storage medium and product

Abstract

This application discloses a network data processing method, apparatus, device, storage medium, and product. The method includes: acquiring status information and a network policy, whereby the status information indicates the current network status, and the network policy indicates actions corresponding to different network states; determining a target action corresponding to the current network status based on the network policy and status information; and executing the target action through the core network to update the current network status. Therefore, by determining the action to be performed for the current network status through the network policy, the target device can make timely adjustments when the surrounding environment changes, thereby ensuring the network (communication) quality of the target device.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to a network data processing method, a network data processing device, a computer equipment, a computer-readable storage medium, and a network data processing product. Background Technology

[0002] With advancements in scientific research, network-based mobile communication technologies (such as 5G) have developed rapidly. Mobile communication has become an indispensable part of daily life. Research has found that the environment surrounding a device often changes; for example, the environment changes continuously as the device moves. These environmental changes alter the device's network state, leading to fluctuations in network (communication) quality. How to ensure network (communication) quality for a device when the surrounding environment changes has become a hot research topic. Summary of the Invention

[0003] This application provides a network data processing method, apparatus, device, computer-readable storage medium, and product that can ensure the network (communication) quality of the target device when the surrounding environment changes.

[0004] On one hand, embodiments of this application provide a network data processing method, including:

[0005] Acquire status information and network policies. Status information is used to indicate the current network status, and network policies are used to indicate the actions corresponding to different network statuses.

[0006] Based on network policies and status information, determine the target action corresponding to the current network state;

[0007] The target action is executed through the core network to update the current network state.

[0008] On one hand, embodiments of this application provide a network data processing apparatus, which includes:

[0009] The acquisition unit is used to acquire status information and network policies. The status information is used to indicate the current network status, and the network policies are used to indicate the actions corresponding to different network statuses.

[0010] The processing unit is used to determine the target action corresponding to the current network state based on network policies and state information.

[0011] And used to perform target actions through the core network to update the current network state.

[0012] In one embodiment, the processing unit is further configured to:

[0013] Obtain feedback information for the target action, which is generated based on the network state after the target action is executed;

[0014] The network strategy is adjusted based on feedback information from the target action.

[0015] In one implementation, the network policy is associated with a state analysis model, and the feedback information includes the reward corresponding to the target action; the processing unit is used to adjust the network policy based on the feedback information of the target action, specifically for:

[0016] Construct a loss function based on the difference between the expected reward of the target action and the reward corresponding to the target action;

[0017] The state analysis model is trained based on the loss function to obtain the trained state analysis model.

[0018] In one implementation, the network strategy includes mappings corresponding to N network states. Each mapping corresponds to at least one candidate action and the probability of selecting each candidate action in that network state, where N is a positive integer. The processing unit is used to adjust the network strategy based on feedback information of the target action, specifically by:

[0019] The reward function is updated based on the feedback information of the target action. The reward function is obtained based on the expected reward of the target action.

[0020] By updating the reward function, the selection probability of each candidate action in the target network state is adjusted, and the state matching degree between the target network state and the current network state is higher than the matching degree threshold.

[0021] In one implementation, the network policy is associated with a state analysis model; the processing unit is used to determine the target action corresponding to the current network state based on the network policy and state information, specifically for:

[0022] The state analysis model is invoked to generate the target action corresponding to the current network state based on the state information.

[0023] In one implementation, the processing unit is configured to determine the target action corresponding to the current network state based on network policies and state information, specifically:

[0024] Based on network policy and state information, determine the mapping relationship of the current network state. The mapping relationship includes at least one candidate action and the selection probability of each candidate action in the current network state.

[0025] The candidate action with the highest probability is selected as the target action corresponding to the current network state, or a candidate action is randomly selected as the target action corresponding to the current network state according to the selection probability of each candidate action in the current network state.

[0026] In one implementation, the network policy includes mapping relationships corresponding to N network states, where N is a positive integer; the processing unit is used to determine the mapping relationship of the current network state based on the network policy and state information, specifically for:

[0027] Based on the state matching degree, the mapping relationship of the target network state is selected as the mapping relationship of the current network state, and the state matching degree between the target network state and the current network state is higher than the matching degree threshold.

[0028] In one implementation, the network policy is associated with a prediction model; the processing unit is used to determine the mapping relationship of the current network state based on the network policy and state information, specifically for:

[0029] The relationship prediction model is invoked to generate a mapping relationship of the current network state based on the state information.

[0030] In one implementation, the acquisition unit is used to acquire status information, specifically for:

[0031] Acquire network analysis data and device network data. Network analysis data is obtained by the network data analysis function in the core network analyzing the current network status, while device network data is collected from the target device under the current network status.

[0032] The network analysis data and device network data are merged to obtain status information.

[0033] In one implementation, the processing unit is configured to perform a target action through the core network to update the current network state, specifically for:

[0034] Send an execution request for the target action to the core network so that the core network can predict the reward corresponding to the target action. When the reward corresponding to the target action meets the execution conditions, execute the corresponding operation according to the instructions of the target action to update the current network state.

[0035] In one implementation, the acquisition unit is used to acquire network policies, specifically for:

[0036] The network policy is obtained from the application server. The network policy is obtained by the application server optimizing the network policy to be optimized based on historical data. The historical data includes historical state information, historical actions, and feedback information corresponding to the historical actions.

[0037] Accordingly, this application provides a computer device comprising:

[0038] Memory, which stores computer programs;

[0039] A processor is used to load computer programs to implement the aforementioned network data processing methods.

[0040] Accordingly, this application provides a computer-readable storage medium storing a computer program adapted to be loaded by a processor and executed by the above-described network data processing method.

[0041] Accordingly, this application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned network data processing method.

[0042] In this embodiment, status information and network policies are acquired. The status information indicates the current network status, and the network policies indicate the actions corresponding to different network statuses. Based on the network policies and status information, the target action corresponding to the current network status is determined, and the target action is executed through the core network to update the current network status. Therefore, determining the action to be performed for the current network status through network policies enables the target device to make timely adjustments when the surrounding environment changes, thereby ensuring the network (communication) quality of the target device. Attached Figure Description

[0043] Figure 1 A network data processing scenario diagram provided in an embodiment of this application;

[0044] Figure 2 A flowchart illustrating a network data processing method provided in this application embodiment;

[0045] Figure 3a This is a schematic diagram illustrating a network policy acquisition method provided in an embodiment of this application;

[0046] Figure 3b This is a schematic diagram illustrating another method for obtaining network policies provided in an embodiment of this application;

[0047] Figure 3c A schematic diagram of a mapping relationship provided for an embodiment of this application;

[0048] Figure 4 A flowchart illustrating another network data processing method provided in this application embodiment;

[0049] Figure 5a This application provides a schematic diagram illustrating the principle of network policy adjustment.

[0050] Figure 5b This application provides a data interaction flowchart corresponding to a network data processing method in an embodiment of the present application.

[0051] Figure 6 This is a schematic diagram of the structure of a network data processing device provided in an embodiment of this application;

[0052] Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0053] It should be noted in advance that, in order to enable those skilled in the art to better understand the technical solutions proposed in the embodiments of this application, the embodiments of this application will be described clearly and completely in conjunction with one or more accompanying drawings. Furthermore, the various drawings shown in the embodiments of this application are merely illustrative examples; for example, the execution order of each step in the drawings can be adaptively adjusted according to the actual application scenario. In addition, in the embodiments of this application, the block diagrams shown in the various drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, or in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0054] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0055] It should be noted that "multiple" in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0056] This application relates to technologies related to core networks and reinforcement learning. A brief introduction to these technologies is provided below:

[0057] Access and Mobility Management Function (AMF) network elements: These can be used to manage terminal access to the core network, such as terminal location updates, network registration, access control, terminal mobility management, and terminal attachment and detachment.

[0058] Session Management Function (SMF) network elements can be used to select user plane network elements for terminals, redirect user plane network elements for terminals, establish bearers (also known as sessions) between terminals and User Plane Function (UPF) network elements, modify and release sessions, and control Quality of Service (QoS).

[0059] The Unified Data Management (UDM) network element, also known as a network storage network element, handles 3GPP AKA authentication, user identification, access authorization, registration, mobility, subscription, and SMS management, similar to the Home Subscriber Server (HSS) in 4G. UDM includes functions such as generating and storing object subscription data and managing authentication data.

[0060] The Policy Control Function (PCF) network element provides a unified policy framework and policy rules for control plane functions, similar to the Policy and Charging Rules Function (PCRF) network element in 4G. The PCF network element is used to provide policies to AMF and SMF network elements, such as QoS policies and slice selection policies.

[0061] A data network (DN) can provide data services to users, such as IP multi-media service (IMS) networks and the Internet. A DN can contain various application servers (AS) that provide different application services, such as carrier services and Internet services.

[0062] User Plane Function (UPF) network elements are primarily responsible for user data transmission. Other network elements, referred to as Control Plane Function (CFD) network elements, are mainly responsible for authentication, authorization, registration management, session management, mobility management, and policy control to ensure reliable and stable user data transmission. UPF network elements can be used to forward and receive terminal data. For example, a UPF network element can receive service data from the data network and transmit it to the terminal through access network equipment; a UPF network element can also receive user data from the terminal through access network equipment and forward it to the data network. The transmission resources allocated and scheduled by the UPF network element for the terminal are managed and controlled by the SMF network element. The bearer between the terminal and the UPF network element can include: the user plane connection between the UPF network element and the access network equipment, and the establishment of a channel between the access network equipment and the terminal. The user plane connection enables the establishment of a QoS flow for data transmission between the UPF network element and the access network equipment.

[0063] Application Function (AF) network elements interact with core network elements to support routing of application-influenced data, access to network exposure functions, and interaction with PCF network elements for policy control. AF network elements can implement control plane functions of third-party application servers, interacting via AF-NEF-PCF or AF-PCF. AS network elements can implement user plane functions of third-party application servers, interacting via AS-IP transport network-UPF.

[0064] Network Exposure Function (NEF) network element: Located between the (5G) core network and external third-party AF network element or AS network element, it is responsible for managing the network data exposed to the outside world. That is, external third-party applications need to go through the NEF network element to access the internal data of the (5G) core network.

[0065] Network slicing: Network slicing is an on-demand networking method that allows operators to separate multiple virtual end-to-end networks on a unified infrastructure. Each network slice is logically isolated from the radio access network to the bearer network and then to the core network to adapt to various types of applications. A network slice can be divided into at least three parts: radio network sub-slices, bearer network sub-slices, and core network sub-slices. The core of network slicing technology is Network Functions Virtualization (NFV). NFV separates the hardware and software components from the traditional network. The hardware is deployed on a unified server, and the software is handled by different Network Functions (NFs), thereby enabling flexible assembly of services. Network slicing is a logical concept and a reorganization of resources. Reorganization involves selecting the necessary virtual machines and physical resources for a specific communication service type according to the Service Level Agreement (SLA). In this application, the terminal device can determine the network slice as the action corresponding to a certain network state based on a specific network state.

[0066] Protocol Data Unit (PDU) Session: A key function of 5G systems, providing a connection to the Data Network (DN) for the User Equipment (UE). To connect to the DN, the UE initiates a PDU session establishment request. The destination of the connection is provided by the UE in the form of a Data Network Name (DNN). This DNN is the identifier of the target Data Network (DN). The DNN can be a custom DNN from an internet service provider, or a well-known DNN, such as one leading to an IMS network. The simplified PDU session establishment process is as follows: The UE initiates a PDU session request to the SMF through the base station and AMF; upon receiving the request, the SMF obtains the UE's subscription data from the UDM; upon receiving the request, the SMF obtains the policy rules for this type of user from the PCF; upon receiving the request, the SMF establishes a session with the UPF to establish a user plane connection; the SMF initiates a radio resource request to the base station; upon receiving the request, the base station sets the corresponding radio resources and replies to the SMF's request; upon receiving the reply, the SMF updates the UPF to establish a tunnel from the UPF to the base station; finally, an end-to-end user plane tunnel (User Equipment-Access Network-User Plane Function, UEANUPF) is generated. The UE reaches the UPF through this tunnel to connect to the target DN. During the PDU session establishment process, the corresponding user plane connection (User Equipment-Data Network, UEDN) will be activated. The function of this user plane connection is to transmit PDUs; the PDU is the protocol type of the terminal user carried on the PDU session; the type of PDU depends on the type of the PDU session (e.g., IP packets or Ethernet frames).

[0067] Reinforcement learning systems typically include the following core components: Agent: This is the learning entity. It selects actions based on the current environmental state and learns based on the results of those actions. In this application, the terminal device can act as the agent to learn from the constantly changing network state and determine the corresponding action based on the current network state. Environment: The external world in which the agent operates and interacts; in this application, it refers to the network / communication environment. The environment receives the agent's actions and returns the next state and the reward for that action. State: Information describing the current state of the environment. The agent selects actions based on this information; in this application, the state specifically refers to the network state. Action: The behavior that the agent can choose given a state. The relationship between state and action can be predicted by a model or indicated by a policy. Reward: Feedback from the environment to the agent's actions, used to evaluate the quality of the actions.

[0068] Based on the aforementioned technologies related to core networks and reinforcement learning, this application provides a network data processing scheme that can ensure the network (communication) quality of the target device when the surrounding environment changes. Figure 1 This application provides a network data processing scenario diagram, such as... Figure 1 As shown, the network data processing scenario provided in this application includes a terminal device 101, an application server 102, communication equipment, and a core network. The network data processing solution provided in this application can be executed by the terminal device 101. The terminal device may include, but is not limited to: smartphones (such as Android phones, iOS phones, etc.), tablet computers, portable personal computers, mobile internet devices (MIDs), smart voice interaction devices, smart home appliances, vehicle terminals, aircraft, wearable devices, etc., and this application embodiment does not limit this; the application server may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, and this application embodiment does not limit this; the communication device may be used to provide 5G wireless network (New Radio, NR) user plane and control plane protocols to the terminal device 101. The communication device is connected to the (5G) core network via the NG (Next Generation) interface. The communication device may be a base station, a roadside unit (RSU), a wireless local area network (Wi-Fi) device, etc.

[0069] It should be noted that, Figure 1 The number of terminal device 101, application server 102, and communication equipment is for illustrative purposes only and does not constitute an actual limitation of this application. Terminal device 101, application server 102, communication equipment, and the core network can be connected via wired or wireless means, and this application does not impose any limitations on this.

[0070] The general flow of the network data processing solution provided in this application is as follows:

[0071] (1) Terminal device 101 acquires status information and network policies; wherein, the status information is used to indicate the current network status. In one embodiment, the status information can be obtained by combining network analysis data and device network data. The network analysis data is obtained by the Network Data Analytics Function (NWDAF) in the core network in response to the analysis request of terminal device 101, which analyzes the current network status. The network analysis data may include, but is not limited to, network status analysis data, traffic pattern analysis data, and device behavior analysis data. The device network data is the data collected by terminal device 101 under the current network status. The device network data may include, but is not limited to, device signal strength and data transmission rate; in addition, it may also include data related to network data such as battery level.

[0072] A network policy is used to indicate actions corresponding to different network states. Specifically, a network policy can be a mapping relationship between different network states and actions; furthermore, it can be carried in a state analysis model. In one implementation, the network policy can be obtained by application server 102 optimizing the network policy to be optimized based on historical data. This historical data includes historical state information (indicating historical network states), historical actions (under those historical network states), and feedback information corresponding to those historical actions (which can be understood as rewards for those historical actions). After obtaining the (optimized) network policy, application server 102 provides it to terminal device 101. In another implementation, the network policy is obtained by adjusting the network policy carried in terminal device 101 based on the feedback information of the actions at the previous moment.

[0073] (2) Terminal device 101 determines the target action corresponding to the current network state based on network policies and state information. For example, terminal device 101 determines that network slice selection needs to be re-performed under the current network state based on network policies and state information. In one embodiment, the network policy is associated with a state analysis model. Terminal device 101 calls the state analysis model and generates the target action corresponding to the current network state based on the state information. This state analysis model can be pre-trained by application server 102 or obtained by terminal device 101 through further optimization of the pre-trained state analysis model. In another embodiment, terminal device 101 determines the mapping relationship of the current network state based on network policies and state information. The mapping relationship includes at least one candidate action and the selection probability of each candidate action under the current network state. After determining the mapping relationship of the current network state, terminal device 101 can determine the candidate action with the highest selection probability as the target action corresponding to the current network state; or randomly select a candidate action as the target action corresponding to the current network state according to the selection probability of each candidate action under the current network state.

[0074] (3) Terminal device 101 executes the target action through the core network to update the current network state. In one embodiment, terminal device 101 sends an execution request for the target action to the core network so that the core network executes the target action and thereby updates the current network state.

[0075] In this embodiment, status information and network policies are acquired. The status information indicates the current network status, and the network policies indicate the actions corresponding to different network statuses. Based on the network policies and status information, the target action corresponding to the current network status is determined, and the target action is executed through the core network to update the current network status. Therefore, determining the action to be performed for the current network status through network policies enables the target device to make timely adjustments when the surrounding environment changes, thereby ensuring the network (communication) quality of the target device.

[0076] Based on the above network data processing scheme, this application proposes a more detailed network data processing method. The network data processing method proposed in this application will be described in detail below with reference to the accompanying drawings.

[0077] Please see Figure 2 , Figure 2 This application provides a flowchart of a network data processing method, which can be executed by a computer device; for example, by... Figure 1 The terminal device 101 shown executes this. (As...) Figure 2 As shown, the network data processing method may include the following steps S201-S203:

[0078] S201. Obtain status information and network policies.

[0079] Status information is used to indicate the current network status. Status information can be obtained based on at least one of network analysis data (data interaction dimension) and device network data (device dimension). Network analysis data is obtained by the network data analysis function in the core network in response to analysis requests from computer devices, analyzing the current network status. Network analysis data may include, but is not limited to, network status analysis data, traffic pattern analysis data, and device behavior analysis data. Device network data is data collected by computer devices themselves under the current network status. Device network data may include, but is not limited to, device signal strength and data transmission rate; in addition, it may include data that may affect the network status, such as battery level. The network data analysis function can be a network element that integrates an Analytical Data Repository Function (ADRF). The ADRF is mainly used for storing network analysis data and collected data. ADRF is introduced to implement the network data analysis function. Furthermore, this network element can integrate at least one of the Analysis Logical Function (AnLF) and the Model Training Logical Function (MTLF). The Analysis Logical Function is responsible for model inference and provides service interfaces such as Nwdaf_AnalyticsSubscription and Nwdaf_AnalyticsInfo. Calling these service interfaces generates analysis results of network data (including static statistical data and dynamic inference results). The Model Training Logical Function is responsible for model training and can provide the trained model to the Analysis Logical Function. AnLF is the only consumer network element providing services to MTLF.

[0080] In one embodiment, the computer device acquires network analysis data through analysis requests and collects its own data regarding the current network state. After obtaining the network analysis data and the device network data, the computer device merges them to obtain state information. In another implementation, the computer device can add the network analysis data and the device network data to the same dataset, or construct a vector or matrix based on the network analysis data and the device network data to obtain the state information.

[0081] It should be noted that if there is conflicting data between network analysis data and device network data; for example, if network analysis data indicates that the current network status is poor, while device network data indicates that the current network is smooth (e.g., low latency, fast transmission speed), the computer device can mark the conflicting data in the network analysis data as abnormal data, take the device network data as the standard, or obtain updated network analysis data through a re-analysis request.

[0082] In another embodiment, the computer device obtains network analysis data through an analysis request and uses the network analysis data as status information.

[0083] In another embodiment, the computer device collects its own data on the current network state and uses the collected device network data as state information.

[0084] Network policies are used to indicate actions corresponding to different network states. Specifically, a network policy can be a mapping relationship between different network states and actions; in addition, a network policy can also be carried in a state analysis model, which then indicates the relationship between network states and actions.

[0085] In one implementation, the network policy is provided by the application server. Figure 3a This is a schematic diagram illustrating a network policy acquisition method provided in an embodiment of this application. Figure 3aAs shown, the target device (User Equipment, or UE), i.e., the computer device, sends an analysis request to the core network. The network data analysis function in the core network responds to the analysis request and returns network analysis data based on the current network state to the UE. Simultaneously, the UE collects its own data under the current network state to obtain device network data. After obtaining the network analysis data and device network data, the UE generates state information based on these data and sends it to the application server. The application server determines the target action based on the network strategy to be optimized and the acquired state information; for example, the application server can call the state analysis model to be trained to analyze the state information and obtain the target action corresponding to the state information. The application server returns the determined target action to the UE, and the UE sends an execution request for the target action to the core network. The core network responds to the execution request, executes the target action, and generates feedback information for the target action (including the target action's reward). The UE obtains the feedback information returned by the core network and sends it to the application server. The application server optimizes the network strategy to be optimized based on this feedback information. Following the steps described above, the application server can continuously optimize the network policy until the optimization termination condition is met (such as reaching a certain number of optimization attempts, or the proportion of positive feedback for a given action exceeding a certain percentage threshold). Furthermore, after optimization is complete, the application server provides the (optimized) network policy to the UE.

[0086] In one implementation, the computer device has acquired a network policy and adjusts the network policy based on feedback information from each action; that is, the network policy acquired by the computer device each time is an adjusted network policy based on feedback information from the previous action. Figure 3b This is a schematic diagram illustrating another method for obtaining network policies provided in an embodiment of this application. (Combined with...) Figure 3a and Figure 3b The difference lies in the fact that the target device has already acquired the network policy, and the adjustment process of the network policy does not require the participation of the application server. After generating state information, the UE determines the target action based on the network policy and state information, and adjusts the network policy after obtaining feedback information on the target action. This method treats the UE as an intelligent agent and further adjusts the network policy through reinforcement learning, enabling the UE to determine the corresponding action based on the adjusted network policy under various network states, thereby ensuring the network (communication) quality of the target device when the surrounding environment changes.

[0087] S202. Based on the network policy and status information, determine the target action corresponding to the current network status.

[0088] Different network states can correspond to different or the same actions; for example, frequency band selection and power allocation. In practical applications, the actions corresponding to each network state can be executed by the core network, or they can be executed collaboratively by computer equipment and the core network; this application does not impose any restrictions on this.

[0089] In one implementation, the network policy is associated with a state analysis model; this can be understood as the network policy being carried within the state analysis model. The computer device invokes the state analysis model to analyze the state information and obtain the target action corresponding to the current network state. Combined with... Figure 3a and Figure 3b It can be seen that the state analysis model can be obtained by pre-training the application server, or by further optimizing the pre-trained state analysis model using computer equipment.

[0090] In one embodiment, the computer device constructs a loss function based on the difference between the expected reward of a historical action and the actual reward of that action, and trains the state analysis model based on this loss function to obtain the trained state analysis model. The process by which the application server pre-trains the model to be trained to obtain the state analysis model is similar to the training process of the state analysis model described above, and will not be repeated here.

[0091] In another implementation, the computer device determines the mapping relationship of the current network state based on network policies and state information. The mapping relationship includes at least one candidate action and the selection probability of each candidate action in the current network state.

[0092] In one implementation, the computer device can calculate the matching degree between the current network state and each network state in the network policy, and determine the mapping relationship of the target network state as the mapping relationship of the current network state, wherein the matching degree between the target network state and the current network state is greater than the matching degree threshold; for example, the matching degree between the target network state and the current network state is the highest among all network states included in the network policy. Figure 3c This is a schematic diagram illustrating a mapping relationship provided for an embodiment of this application. For example... Figure 3c As shown, the network policy includes the mapping relationship between network state A and network state N. Taking network state A as an example, the mapping relationship of network state A includes candidate action 1, candidate action 3, and candidate action 9; among them, the selection probability of candidate action 1 is 72%, the selection probability of candidate action 3 is 15%, and the selection probability of candidate action 9 is 13%. It should be noted that... Figure 3c The mapping relationships shown are for illustrative purposes only. In practical applications, mapping relationships can also be indicated by tables, rules, etc. This application does not impose any restrictions on this.

[0093] In another implementation, the network policy is associated with a relationship prediction model. The computer device can invoke this model to generate a mapping relationship for the current network state based on state information. In other words, the relationship prediction model is used to predict the mapping relationship for a given network state. This model can be trained on historical data, including historical state information, historical actions, and corresponding feedback information. As can be seen from the above, the difference between the relationship prediction model and the state analysis model is that the relationship prediction model predicts the mapping relationship for a given network state, while the state analysis model directly predicts the actions corresponding to that network state.

[0094] After determining the mapping relationship of the current network state, the computer device can either select the candidate action with the highest probability as the target action corresponding to the current network state, or randomly select a candidate action as the target action corresponding to the current network state according to the selection probability (probability distribution) of each candidate action in the current network state. Compared with directly selecting the candidate action with the highest probability as the target action, the random selection method based on probability distribution ensures the diversity of actions and can better adapt to the iterative updates of the environment and strategy.

[0095] In one embodiment, the computer device updates the reward function (such as the Q-function in Q-learning) based on the actual rewards of historical actions. This reward function is derived from the expected rewards of historical actions. The updated reward function is then used to adjust the selection probability of each candidate action in the target network state. The process by which the application server adjusts the mapping relationship is similar to the above and will not be described in detail here.

[0096] S203. Execute the target action through the core network to update the current network status.

[0097] In one implementation, a computer device sends a request to the core network to execute a target action, thereby causing the core network to execute the target action and update the current network state.

[0098] In another implementation, the computer device sends an execution request for a target action to the core network. After receiving the execution request, the core network predicts the reward corresponding to the target action. If the reward meets the execution conditions, the core network executes the corresponding operation according to the target action's instructions to update the current network state. For example, after receiving the execution request, the core network predicts the reward score for the target action. If the reward score is greater than a score threshold, the core network executes the corresponding operation according to the target action's instructions to update the current network state. If the reward score is less than or equal to the score threshold, the core network can return a prompt message to the computer device. This prompt message indicates that the target action is unsuitable for the current network state, or indicates a risk (such as potential deterioration of network quality) and requires secondary confirmation.

[0099] In this embodiment, state information and network policy are acquired. The state information indicates the current network state, and the network policy indicates the actions corresponding to different network states. Based on the network policy and state information, the target action corresponding to the current network state is determined, and the target action is executed through the core network to update the current network state. It is evident that determining the action to be performed for the current network state through the network policy enables the target device to make timely adjustments when the surrounding environment changes, thereby ensuring the network (communication) quality of the target device. Furthermore, the network policy is further adjusted using reinforcement learning, enabling the computer device to determine appropriate actions based on the adjusted network policy under various network states. By having the core network determine whether the target action meets the execution conditions, the probability of executing actions incompatible with the current network state can be further reduced.

[0100] Please see Figure 4 , Figure 4 This is a flowchart illustrating another network data processing method provided in an embodiment of this application. This network data processing method can be executed by a computer device; for example, by... Figure 1 The terminal device 101 (i.e., UE) shown in the figure executes this. Figure 4 As shown, the network data processing method may include the following steps S401-S405:

[0101] S401, Obtain status information and network policies.

[0102] In one implementation, the process of a computer device acquiring status information includes: acquiring network analysis data and device network data; wherein the network analysis data is obtained by the network data analysis function (network element) analyzing the current network status, and the device network data is collected by the computer device under the current network status. After acquiring the network analysis data and device network data, the computer device merges the network analysis data and device network data to obtain the status information.

[0103] In another implementation, the computer device obtains the network policy from the application server. The network policy is obtained by the application server optimizing the network policy based on historical data. The historical data includes historical state information, historical actions, and corresponding feedback information. The process by which the application server optimizes the network policy based on historical data can be found in [reference needed]. Figure 3a This will not be elaborated upon here.

[0104] S402. Based on network policies and status information, determine the target action corresponding to the current network status.

[0105] S403. Execute the target action through the core network to update the current network status.

[0106] For detailed implementation methods of S402 and S403, please refer to... Figure 2 The implementation methods of S202 and S203 will not be described in detail here.

[0107] S404. Obtain feedback information on the target action.

[0108] The feedback information for the target action includes the reward corresponding to the target action, and the feedback information for the target action is generated based on the network state after the target action is executed; for example, the network data analysis function in the core network generates the feedback information for the target action based on the network state after the target action is executed.

[0109] In one embodiment, the network data analysis function can generate feedback information for the target action solely based on the network state after the target action is executed. In another embodiment, the network data analysis function can generate feedback information for the target action based on the difference between the network state after the target action is executed and the network state before the target action is executed.

[0110] S405. Adjust the network strategy based on the feedback information of the target action.

[0111] In one implementation, the network policy is associated with a state analysis model, and the feedback information includes the reward corresponding to the target action (i.e., the actual reward of the target action). The computer device constructs a loss function based on the difference between the expected reward of the target action and the reward corresponding to the target action; wherein the expected reward of the target action indicates the expected reward of taking the target action in the current network state. After constructing the loss function, the computer device trains the state analysis model based on the loss function to obtain the trained state analysis model.

[0112] In another implementation, the network strategy includes mappings to N network states. Each mapping to a network state includes at least one candidate action and the probability of selecting each candidate action in that network state, where N is a positive integer. The mapping relationships can be found in [reference needed]. Figure 3c This will not be elaborated further here. The computer device updates the reward function (such as the Q-function in Q-learning) based on the feedback information of the target action (including the reward of the target action). The reward function is obtained based on the expected reward of the target action. After obtaining the updated reward function, the computer device adjusts the selection probability of each candidate action in the target network state using the updated reward function; wherein, the state matching degree between the target network state and the current network state is higher than the matching degree threshold, and the selection probability of each action is proportional to the reward of that action.

[0113] Through S404 and S405, the computer device can be treated as an intelligent agent, iteratively executing the above S401-S405 to further adjust the network policy. This allows the computer device to determine the corresponding action based on the adjusted network policy under various network conditions, thereby ensuring the network (communication) quality of the target device when the surrounding environment changes. Figure 5a This is a schematic diagram illustrating the principle of network policy adjustment provided in an embodiment of this application. Figure 5a As shown, the computer device responds to action a t-1 The corresponding reward r t Adjust the network policy and based on the adjusted network policy and state s t Determine state s t The corresponding action a t Environment based on action a t State s t Updated to s t+1 And give action a t The corresponding reward r t+1 By repeatedly executing the above process, reinforcement learning of network policies can be achieved. Through reinforcement learning (treating terminal devices as intelligent agents), decision-making can be optimized (enabling computer devices to select actions appropriate to the current network state based on network policies), thereby achieving end-to-end network optimization. This helps improve overall network performance and coverage, and reduces operating costs.

[0114] Figure 5b This is a data interaction flowchart corresponding to a network data processing method provided in an embodiment of this application. For example... Figure 5bAs shown, the UE first initializes (begins interaction with the environment as an intelligent agent), specifically including network access and authentication. The UE can initiate a Protocol Data Unit (PDU) session request through the System Management Function (SMF) to establish a connection with the data network. After establishing the PDU session, the network environment in which the UE is located is defined, including available network slices, bandwidth, latency, and other parameters. Next, the UE collects training data, which includes collecting state information, the corresponding actions, and feedback information. The state information collection process includes: the UE receiving current network state information from the environment and sending an analysis request to the core network through the Network Open Function (NEF) to request the Network Data Analysis Function (NWDAF) in the core network to generate network analysis data based on the current network state. This network analysis data can include network state, traffic patterns, device behavior, etc. Additionally, the UE also collects its own network data, such as signal strength, data transmission rate, and battery level. After obtaining its own network data (i.e., device network data) and the network analysis data returned by the network data analysis function, the UE merges its own network data and the network analysis data returned by the network data analysis function to obtain status information.

[0115] The UE then provides the collected training data to the application server, which pre-trains the network policy using the data. The network policy can be a simple rule, a mapping relationship, or a complex neural network model; this application does not impose any limitations on this. The pre-training process includes: the application server preprocessing the received data, such as cleaning, denoising, and normalizing. The pre-processed data is then used to train the network policy, improving the fit between actions (determined based on the network policy) and the current network state. Higher fit results in higher rewards for the actions. Before providing the pre-trained network policy to the UE, the application server can also evaluate the network policy to ensure it meets requirements. During the pre-training process, the UE can interact with the application server in real time, or it can collect training data in batches before providing it to the application server. The process of real-time interaction between the UE and the application server to train the network policy can be found in [reference needed]. Figure 3a This will not be elaborated further here. After completing the pre-training of the network policy, the application server provides the pre-trained network policy to the UE.

[0116] Furthermore, the UE sends an analysis request to the core network. The network data analysis function in the core network responds to the analysis request and returns network analysis data based on the current network state to the UE. Simultaneously, the UE collects its own data under the current network state to obtain device network data. After obtaining the network analysis data and device network data, the UE generates state information based on the network analysis data and device network data, and determines the target action according to the state information and network policy. Then, the UE sends an execution request for the target action to the core network. The core network executes the target action, obtains a new network state, and generates feedback information for the target action (including a reward for the target action). Further, the UE obtains the feedback information for the target action and evaluates whether the target action is suitable for the network state before the update based on this feedback information, and adjusts the network policy according to the evaluation result. By iterating through the above process, the network policy can be continuously optimized. It is understood that as the number of iterations increases, the action determined by the UE based on the network policy and state information will get closer and closer to the optimal solution. The loop termination condition can be the number of iterations or performance requirements, etc., and this application does not impose any restrictions on this.

[0117] As shown above, the pre-trained network policy can be provided (configured) to multiple different terminal devices, eliminating the need for individual training on each device, thus reducing training costs and improving training efficiency. Reinforcement learning methods can be used to specifically train the network policy, further enhancing the training effect and enabling each terminal device (based on the reinforcement-learned network policy) to determine appropriate actions according to its own network environment.

[0118] In this embodiment, state information and network policies are acquired. The state information indicates the current network state, and the network policies indicate the actions corresponding to different network states. Based on the network policies and state information, the target action corresponding to the current network state is determined, and the target action is executed through the core network to update the current network state. It is evident that determining the action to be executed for the current network state through network policies allows the target device to make timely adjustments when the surrounding environment changes, thereby ensuring the network (communication) quality of the target device. Furthermore, the network policies are further adjusted using reinforcement learning, enabling the computer device to determine appropriate actions based on the adjusted network policies under various network states. Determining whether the target action meets the execution conditions through the core network further reduces the probability of executing actions incompatible with the current network state. Pre-training the network policies through the application server reduces training costs and improves training efficiency. Reinforcement learning (using the terminal device as an agent) optimizes decision-making (enabling the computer device to select actions suitable for the current network state based on the network policies), thereby achieving end-to-end network optimization. This helps improve the overall network performance and coverage, and reduces operating costs.

[0119] The methods of the embodiments of this application have been described in detail above. In order to facilitate better implementation of the above solutions of the embodiments of this application, the apparatus of the embodiments of this application is provided below.

[0120] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a network data processing device provided in an embodiment of this application. Figure 6 The network data processing device shown can be mounted in a computer device, which can specifically be a terminal device or a server. Figure 6 The network data processing device shown can be used to perform the above. Figure 2 and Figure 4 Some or all of the functionality described in the method embodiments. Please refer to [link / reference]. Figure 6 The network data processing device includes:

[0121] The acquisition unit 601 is used to acquire status information and network policy. The status information is used to indicate the current network status, and the network policy is used to indicate the actions corresponding to different network statuses.

[0122] The processing unit 602 is used to determine the target action corresponding to the current network state based on the network policy and state information;

[0123] And used to perform target actions through the core network to update the current network state.

[0124] In one embodiment, the processing unit 602 is further configured to:

[0125] Obtain feedback information for the target action, which is generated based on the network state after the target action is executed;

[0126] The network strategy is adjusted based on feedback information from the target action.

[0127] In one implementation, the network policy is associated with a state analysis model, and the feedback information includes the reward corresponding to the target action; the processing unit 602 is used to adjust the network policy based on the feedback information of the target action, specifically for:

[0128] Construct a loss function based on the difference between the expected reward of the target action and the reward corresponding to the target action;

[0129] The state analysis model is trained based on the loss function to obtain the trained state analysis model.

[0130] In one implementation, the network strategy includes mapping relationships corresponding to N network states. Each mapping relationship corresponding to a network state includes at least one candidate action and the selection probability of each candidate action in that network state, where N is a positive integer. The processing unit 602 is used to adjust the network strategy based on feedback information of the target action, specifically for:

[0131] The reward function is updated based on the feedback information of the target action. The reward function is obtained based on the expected reward of the target action.

[0132] By updating the reward function, the selection probability of each candidate action in the target network state is adjusted, and the state matching degree between the target network state and the current network state is higher than the matching degree threshold.

[0133] In one implementation, the network policy is associated with a state analysis model; the processing unit 602 is used to determine the target action corresponding to the current network state based on the network policy and state information, specifically for:

[0134] The state analysis model is invoked to generate the target action corresponding to the current network state based on the state information.

[0135] In one implementation, the processing unit 602 is configured to determine the target action corresponding to the current network state based on network policies and state information, specifically for:

[0136] Based on network policy and state information, determine the mapping relationship of the current network state. The mapping relationship includes at least one candidate action and the selection probability of each candidate action in the current network state.

[0137] The candidate action with the highest probability is selected as the target action corresponding to the current network state, or a candidate action is randomly selected as the target action corresponding to the current network state according to the selection probability of each candidate action in the current network state.

[0138] In one implementation, the network policy includes mapping relationships corresponding to N network states, where N is a positive integer; the processing unit 602 is used to determine the mapping relationship of the current network state based on the network policy and state information, specifically for:

[0139] Based on the state matching degree, the mapping relationship of the target network state is selected as the mapping relationship of the current network state, and the state matching degree between the target network state and the current network state is higher than the matching degree threshold.

[0140] In one implementation, the network policy is associated with a prediction model; the processing unit 602 is used to determine the mapping relationship of the current network state based on the network policy and state information, specifically for:

[0141] The relationship prediction model is invoked to generate a mapping relationship of the current network state based on the state information.

[0142] In one embodiment, the acquisition unit 601 is used to acquire status information, specifically for:

[0143] Acquire network analysis data and device network data. Network analysis data is obtained by the network data analysis function in the core network analyzing the current network status, while device network data is collected from the target device under the current network status.

[0144] The network analysis data and device network data are merged to obtain status information.

[0145] In one implementation, the processing unit 602 is configured to perform a target action through the core network to update the current network state, specifically for:

[0146] Send an execution request for the target action to the core network so that the core network can predict the reward corresponding to the target action. When the reward corresponding to the target action meets the execution conditions, execute the corresponding operation according to the instructions of the target action to update the current network state.

[0147] In one embodiment, the acquisition unit 601 is used to acquire network policies, specifically for:

[0148] The network policy is obtained from the application server. The network policy is obtained by the application server optimizing the network policy to be optimized based on historical data. The historical data includes historical state information, historical actions, and feedback information corresponding to the historical actions.

[0149] According to one embodiment of this application, Figure 2 and Figure 4 Some steps involved in the network data processing method shown can be derived from... Figure 6 The network data processing apparatus shown is executed by each unit. For example, Figure 2 The S201 shown can be derived from Figure 6 The acquisition unit 601 shown is executed, and S202 and S203 can be performed by... Figure 6 The processing unit 602 shown executes the operation; Figure 4 S401 and S404 shown can be derived from Figure 6 The acquisition unit 601 shown is executed, and S402, S403 and S405 can be performed by... Figure 6 The processing unit 602 shown executes. Figure 6The various units in the network data processing apparatus shown can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of this application. The above-mentioned units are based on logical function division. In practical applications, the function of one unit can also be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the network data processing apparatus may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0150] According to another embodiment of this application, a general-purpose computing device, such as a computer device including processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM), can perform operations such as... Figure 2 and Figure 4 The computer program (including program code) for each step involved in the corresponding method shown, to construct such... Figure 6 The network data processing apparatus shown herein, and the network data processing method for implementing the embodiments of this application, are described. A computer program may be recorded on, for example, a computer-readable recording medium, loaded onto the aforementioned computing device via the computer-readable recording medium, and executed therein.

[0151] Based on the same inventive concept, the principle and beneficial effects of the network data processing device provided in the embodiments of this application in solving the problem are similar to the principle and beneficial effects of the network data processing method in the embodiments of this application in solving the problem. Please refer to the principle and beneficial effects of the method implementation. For the sake of brevity, they will not be repeated here.

[0152] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. The computer device may be a terminal device or a server. Figure 7As shown, the computer device includes at least a processor 701, a communication interface 702, and a memory 703. The processor 701, communication interface 702, and memory 703 can be connected via a bus or other means. The processor 701 (or Central Processing Unit, CPU) is the computing and control core of the computer device. It can parse various instructions within the computer device and process various data. For example, the CPU can parse power-on / off commands issued by objects to the computer device and control the computer device to perform power-on / off operations; it can also transmit various interactive data between internal structures of the computer device, and so on. The communication interface 702 may optionally include standard wired interfaces or wireless interfaces (such as Wi-Fi, mobile communication interfaces, etc.), and can be used to send and receive data under the control of the processor 701; the communication interface 702 can also be used for data transmission and interaction within the computer device. The memory 703 is the storage device in the computer device, used to store programs and data. It can be understood that the memory 703 here can include the computer device's built-in memory, or it can include extended memory supported by the computer device. The memory 703 provides storage space for storing the operating system of the computer device, which may include, but is not limited to, Android, iOS, Windows Phone, etc. This application does not limit this.

[0153] This application embodiment also provides a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the processing system of the computer device. Furthermore, the storage space also stores computer programs suitable for loading and execution by the processor 701. It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device; optionally, it can also be at least one computer-readable storage medium located remotely from the aforementioned processor.

[0154] In one embodiment, processor 701 performs the following operations by running a computer program stored in memory 703:

[0155] Acquire status information and network policies. Status information is used to indicate the current network status, and network policies are used to indicate the actions corresponding to different network statuses.

[0156] Based on network policies and status information, determine the target action corresponding to the current network state;

[0157] The target action is executed through the core network to update the current network state.

[0158] As an optional embodiment, the processor 701, by running a computer program in the memory 703, also performs the following operations:

[0159] Obtain feedback information for the target action, which is generated based on the network state after the target action is executed;

[0160] The network strategy is adjusted based on feedback information from the target action.

[0161] As an optional embodiment, the network policy is associated with a state analysis model, and the feedback information includes the reward corresponding to the target action; the processor 70 adjusts the network policy based on the feedback information of the target action in the following specific embodiment:

[0162] Construct a loss function based on the difference between the expected reward of the target action and the reward corresponding to the target action;

[0163] The state analysis model is trained based on the loss function to obtain the trained state analysis model.

[0164] As an optional embodiment, the network strategy includes mapping relationships corresponding to N network states. Each mapping relationship corresponding to a network state includes at least one candidate action and the selection probability of each candidate action in that network state, where N is a positive integer. A specific embodiment of the processor 701 adjusting the network strategy based on feedback information from the target action is as follows:

[0165] The reward function is updated based on the feedback information of the target action. The reward function is obtained based on the expected reward of the target action.

[0166] By updating the reward function, the selection probability of each candidate action in the target network state is adjusted, and the state matching degree between the target network state and the current network state is higher than the matching degree threshold.

[0167] As an optional embodiment, the network policy is associated with a state analysis model; the processor 701 determines the target action corresponding to the current network state based on the network policy and state information. A specific embodiment of this is as follows:

[0168] The state analysis model is invoked to generate the target action corresponding to the current network state based on the state information.

[0169] As an optional embodiment, the processor 701 determines the target action corresponding to the current network state based on network policies and state information in the following specific embodiment:

[0170] Based on network policy and state information, determine the mapping relationship of the current network state. The mapping relationship includes at least one candidate action and the selection probability of each candidate action in the current network state.

[0171] The candidate action with the highest probability is selected as the target action corresponding to the current network state, or a candidate action is randomly selected as the target action corresponding to the current network state according to the selection probability of each candidate action in the current network state.

[0172] As an optional embodiment, the network policy includes mapping relationships corresponding to N network states, where N is a positive integer; the processor 701 determines the mapping relationship of the current network state based on the network policy and state information in the following specific embodiment:

[0173] Based on the state matching degree, the mapping relationship of the target network state is selected as the mapping relationship of the current network state, and the state matching degree between the target network state and the current network state is higher than the matching degree threshold.

[0174] As an optional embodiment, the network policy is associated with a prediction model; the processor 701 determines the mapping relationship of the current network state based on the network policy and state information in the following specific embodiment:

[0175] The relationship prediction model is invoked to generate a mapping relationship of the current network state based on the state information.

[0176] As an optional embodiment, the specific embodiment in which the processor 701 acquires the status information is as follows:

[0177] Acquire network analysis data and device network data. Network analysis data is obtained by the network data analysis function in the core network analyzing the current network status, while device network data is collected from the target device under the current network status.

[0178] The network analysis data and device network data are merged to obtain status information.

[0179] As an optional embodiment, the processor 701 performs the target action through the core network to update the current network state. A specific embodiment of this is as follows:

[0180] Send an execution request for the target action to the core network so that the core network can predict the reward corresponding to the target action. When the reward corresponding to the target action meets the execution conditions, execute the corresponding operation according to the instructions of the target action to update the current network state.

[0181] As an optional embodiment, the specific embodiment in which the processor 701 obtains the network policy is as follows:

[0182] The network policy is obtained from the application server. The network policy is obtained by the application server optimizing the network policy to be optimized based on historical data. The historical data includes historical state information, historical actions, and feedback information corresponding to the historical actions.

[0183] Based on the same inventive concept, the principle and beneficial effects of the computer device provided in the embodiments of this application in solving the problem are similar to the principle and beneficial effects of the network data processing method in the embodiments of this application in solving the problem. Please refer to the principle and beneficial effects of the implementation of the method. For the sake of brevity, they will not be repeated here.

[0184] This application also provides a computer-readable storage medium storing a computer program adapted to be loaded by a processor and to execute the network data processing method described in the above method embodiments.

[0185] This application also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the network data processing method described above.

[0186] The steps in the method of this application embodiment can be adjusted, combined, or deleted according to actual needs.

[0187] The modules in the device of this application embodiment can be merged, divided, and deleted according to actual needs.

[0188] In the embodiments of this application, the term "module" or "unit" refers to a computer program or part of a computer program with a predetermined function, which works together with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0189] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc.

[0190] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Those skilled in the art will understand that all or part of the processes for implementing the above embodiments and equivalent variations made in accordance with the claims of this application are still within the scope of this application.

Claims

1. A network data processing method, characterized in that, The method includes: Obtain status information and network policies, wherein the status information is used to indicate the current network status and the network policies are used to indicate actions corresponding to different network statuses; Based on the network policy and the state information, determine the target action corresponding to the current network state; The target action is executed through the core network to update the current network state.

2. The method as described in claim 1, characterized in that, The method further includes: Obtain feedback information of the target action, which is generated based on the network state after the target action is executed; The network strategy is adjusted based on the feedback information of the target action.

3. The method as described in claim 2, characterized in that, The network strategy is associated with a state analysis model, and the feedback information includes the reward corresponding to the target action; adjusting the network strategy based on the feedback information of the target action includes: A loss function is constructed based on the difference between the expected reward of the target action and the reward corresponding to the target action; Based on the loss function, the state analysis model is trained to obtain the trained state analysis model.

4. The method as described in claim 2, characterized in that, The network strategy includes mapping relationships corresponding to N network states. The mapping relationship corresponding to any network state includes at least one candidate action and the selection probability of each candidate action in that network state, where N is a positive integer. The adjustment of the network strategy based on the feedback information of the target action includes: Based on the feedback information of the target action, the reward function is updated, and the reward function is obtained based on the expected reward of the target action; The updated reward function adjusts the selection probability of each candidate action in the target network state, wherein the state matching degree between the target network state and the current network state is higher than the matching degree threshold.

5. The method as described in claim 1, characterized in that, The network policy is associated with a state analysis model; determining the target action corresponding to the current network state based on the network policy and the state information includes: The state analysis model is invoked to generate the target action corresponding to the current network state based on the state information.

6. The method as described in claim 1, characterized in that, The step of determining the target action corresponding to the current network state based on the network policy and the state information includes: Based on the network policy and the state information, a mapping relationship for the current network state is determined. The mapping relationship includes at least one candidate action and the selection probability of each candidate action in the current network state. The candidate action with the highest probability is selected as the target action corresponding to the current network state, or a candidate action is randomly selected as the target action corresponding to the current network state according to the selection probability of each candidate action in the current network state.

7. The method as described in claim 6, characterized in that, The network strategy includes mapping relationships corresponding to N network states, where N is a positive integer; determining the mapping relationship of the current network state based on the network strategy and the state information includes: Based on the state matching degree, the mapping relationship of the target network state is selected as the mapping relationship of the current network state, wherein the state matching degree between the target network state and the current network state is higher than the matching degree threshold.

8. The method as described in claim 6, characterized in that, The network strategy is associated with a prediction model; determining the mapping relationship of the current network state based on the network strategy and the state information includes: The relationship prediction model is invoked to generate a mapping relationship for the current network state based on the state information.

9. The method as described in claim 1, characterized in that, Obtain status information, including: The network analysis data and device network data are obtained. The network analysis data is obtained by the network data analysis function in the core network analyzing the current network status, and the device network data is collected by the target device under the current network status. The network analysis data and the device network data are merged to obtain status information.

10. The method as described in claim 1, characterized in that, The step of executing the target action through the core network to update the current network state includes: The core network sends an execution request for the target action to the core network so that the core network can predict the reward corresponding to the target action. When the reward corresponding to the target action meets the execution conditions, the core network performs the corresponding operation according to the instruction of the target action to update the current network state.

11. The method as described in claim 1, characterized in that, Obtain network policies, including: The network policy is obtained from the application server. The network policy is obtained by the application server optimizing the network policy to be optimized based on historical data. The historical data includes historical state information, historical actions, and feedback information corresponding to the historical actions.

12. A network data processing device, characterized in that, The network data processing device includes: An acquisition unit is used to acquire status information and network policies, wherein the status information is used to indicate the current network status and the network policies are used to indicate actions corresponding to different network statuses; The processing unit is configured to determine the target action corresponding to the current network state based on the network policy and the state information. And for performing the target action via the core network to update the current network state.

13. A computer device, characterized in that, include: A memory, wherein a computer program is stored; A processor for loading the computer program to implement the network data processing method as described in any one of claims 1-11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded by a processor and executed as described in any one of claims 1-11.

15. A computer program product, characterized in that, The computer program product includes a computer program adapted to be loaded by a processor and execute the network data processing method as described in any one of claims 1-11.

Images