Mining network flow intelligent scheduling method and device
By using policy output neural networks and policy evaluation algorithms in mining communication networks, the traffic link scheduling strategies are generated and evaluated, and the problem of stability and speed regulation in complex environments is solved, and a more efficient network is achieved Traffic management.
Patent Information
- Application Number
- CN202411676621.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-22
- Publication Date
- 2025-02-14
AI Technical Summary
When facing the complex and changeable network environment in the mine, the mining communication network has poor stability and slow regulation speed, making it difficult to cope with sudden failures and surge in traffic.
By obtaining the historical traffic data and status information of each base station, terminal equipment and network node under the mine, using the policy output neural network for training, generating the traffic link scheduling policy to be updated, and determining the execution of the policy through the policy evaluation algorithm.
It improves the stability and regulation speed of mining communication networks in complex network environments, realizes automatic regulation of network traffic paths, and improves the response speed and security of the network.
Smart Images
Figure CN119450746A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of computer technology, and in particular relates to a method, a device and an electronic device for intelligent scheduling of mining network traffic. Background Art
[0002] In the mine operation environment, the stability and reliability of the wireless communication network are directly related to production safety and efficiency. In the traditional network traffic backup solution, it is achieved through static configuration and manual intervention, which is difficult to cope with the complex and changeable network environment in the mine, such as sudden failures, signal attenuation and electromagnetic interference. These problems often lead to communication interruptions and increased delays, which not only affect the operation efficiency, but also pose a potential threat to the safety of mine workers. In addition, the traditional method lacks intelligence and adaptability in network traffic scheduling and backup strategies, and it is difficult to effectively deal with sudden network failures and traffic surges.
[0003] Therefore, how to enable the mining communication network to cope with the complex and changeable network environment in the mine, improve the working stability of the mining communication network, and automatically control the traffic path in the mining network to improve the control speed are technical problems to be solved by technical personnel in this field. Summary of the invention
[0004] The purpose of the present invention is to solve the technical problems in the prior art that the mining communication network has poor stability and slow control speed when dealing with the complex and changeable network environment in the mine.
[0005] To achieve the above technical objectives, on the one hand, the present invention provides a method for intelligent scheduling of mining network traffic, the method comprising:
[0006] Obtain historical traffic data of each base station, terminal equipment and network node in the mine, and obtain historical status information and historical environment information of each device;
[0007] The strategy output neural network to be trained is trained by using the historical traffic data to obtain a strategy output neural network;
[0008] The current traffic data of each base station, terminal device and network node and the current status information and current environment information of each device are input into the strategy output neural network to obtain the traffic link scheduling strategy to be updated;
[0009] Whether to execute the traffic link scheduling strategy to be updated is determined through a strategy evaluation algorithm.
[0010] Further, the determining whether to execute the traffic link scheduling strategy to be updated through a strategy evaluation algorithm specifically includes:
[0011] Determine whether a network failure occurs in the traffic link scheduling strategy to be updated by using the strategy evaluation algorithm;
[0012] If not, the traffic link scheduling strategy to be updated is updated to the mining network for execution;
[0013] If so, the strategy evaluation algorithm selects the optimal traffic link scheduling strategy from the executed traffic link scheduling strategy library and updates it to the mining network for execution.
[0014] Furthermore, the method further comprises:
[0015] After the mining network executes the updated traffic link scheduling strategy, the current traffic data of each base station, terminal device and network node and the current status information of each device and the current environmental information are obtained in real time to train and update the strategy output neural network, and the latest current traffic data of each base station, terminal device and network node and the current status information of each device and the current environmental information are input into the updated strategy output neural network to obtain the current traffic link scheduling strategy to be updated.
[0016] Furthermore, the strategy output neural network simultaneously executes the traffic conservation equation and the link capacity limitation equation when outputting the traffic link scheduling strategy to be updated.
[0017] Furthermore, the strategy output neural network to be trained is specifically a deep Q network, and the strategy evaluation algorithm is specifically a proximal strategy optimization algorithm.
[0018] Furthermore, the Q value of the deep Q network is updated by the following formula:
[0019]
[0020] In the formula, Q(s t ,a t ) is in state s t Take action a t Q value, α is the learning rate, r t+1 For state s t Take action a t The instant reward, γ is the discount factor, For the next state s t-1 The maximum Q value of all actions a′ in .
[0021] Furthermore, the loss function L(θ) in the deep Q network is specifically shown as follows:
[0022]
[0023] In the formula, Q(s,a;θ) is the predicted Q value of the current network selecting action a in state s, r is the immediate reward, γ is the discount factor, The maximum value of the predicted Q value when selecting all actions a' in the next state s', θ - is the target network parameter, and θ is the network parameter.
[0024] Furthermore, the objective function L in the proximal strategy optimization algorithm is CLIP (θ') is specifically shown in the following formula:
[0025]
[0026] In the formula, E t To find the expectation for all time steps t on the trajectory, r t (θ') is the ratio of the probability of taking action under a given state between the traffic link scheduling strategy to be updated and the traffic link scheduling strategy being executed, is the estimated value of the advantage function, and clip is expressed as r t (θ') sheared to (r t (θ'),1-∈,1+∈), where ∈ is a hyperparameter.
[0027] On the other hand, the present invention also provides a mining flow intelligent dispatching device, the device comprising:
[0028] The acquisition module is used to obtain the historical traffic data of each base station, terminal equipment and network node in the mine, and obtain the historical status information and historical environment information of each device;
[0029] A training module, used for training the strategy output neural network to be trained by using the historical traffic data to obtain a strategy output neural network;
[0030] A strategy module, used to input the current traffic data of each base station, terminal device and network node and the current status information and current environment information of each device into the strategy output neural network to obtain the traffic link scheduling strategy to be updated;
[0031] The determination module is used to determine whether to execute the traffic link scheduling strategy to be updated through a strategy evaluation algorithm.
[0032] The present invention provides a method and device for intelligent scheduling of mining network traffic. Compared with the prior art, the method first obtains historical traffic data of each base station, terminal device and network node in the mine, and simultaneously obtains historical status information and historical environmental information of each device; then the strategy output neural network to be trained is trained by the historical traffic data to obtain a strategy output neural network; then the current traffic data of each base station, terminal device and network node and the current status information and current environmental information of each device are input into the strategy output neural network to obtain a traffic link scheduling strategy to be updated; finally, a strategy evaluation algorithm is used to determine whether to execute the traffic link scheduling strategy to be updated, so that the mining communication network can cope with the complex and changeable network environment in the mine, improve the working stability of the mining communication network, and automatically regulate the traffic path in the mining network to improve the regulation speed. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0034] Figure 1 The figure is a flow chart of the intelligent scheduling method for mining network traffic provided by the embodiment of this specification;
[0035] Figure 2 Shown is a schematic diagram of the structure of the intelligent scheduling device for mining network traffic provided in the embodiment of this specification. DETAILED DESCRIPTION
[0036] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.
[0037] like Figure 1The flowchart of the intelligent scheduling method for mining network traffic provided by the embodiment of this specification is shown. Although this specification provides the method operation steps or device structures shown in the following embodiments or drawings, the method or device may include more or fewer operation steps or module units after partial merger based on routine or no creative labor. In the steps or structures where there is no necessary causal relationship logically, the execution order of these steps or the module structure of the device is not limited to the execution order or module structure shown in the embodiments or drawings of this specification. When the method or module structure is applied in actual devices, servers or terminal products, it can be executed sequentially or in parallel according to the method or module structure shown in the embodiments or drawings (for example, in the environment of parallel processors or multi-threaded processing, or even in the implementation environment of distributed processing and server clusters).
[0038] The intelligent scheduling method for mining network traffic provided in the embodiments of this specification can be applied in various mining environments, such as Figure 1 As shown, the method specifically comprises the following steps:
[0039] Step S101: Obtain historical traffic data of each base station, terminal device and network node in the mine, and simultaneously obtain historical status information and historical environment information of each device.
[0040] Specifically, the acquisition module or the traffic monitoring and analysis module collects network traffic data and link status information in real time, including key indicators such as bandwidth usage, delay, and packet loss rate. After preprocessing and feature extraction, these data are used as input for subsequent algorithm learning. The traffic monitoring and analysis module uses high-performance servers and big data analysis technology to capture and process massive data from base stations, terminal devices, and network nodes in real time. It not only monitors traditional network performance indicators (such as bandwidth utilization, delay, and packet loss rate), but also deeply captures multi-dimensional data such as device status, user behavior patterns, and mine-specific environmental factors (such as temperature, humidity, and gas concentration). These data are encrypted and transmitted to the data center. After strict data cleaning, denoising, and feature extraction, a high-quality data set is formed. The base station is a mine-used wireless communication base station, which is distributed in various key areas of the mine to ensure comprehensive signal coverage.
[0041] Step S102: training the strategy output neural network to be trained by using the historical traffic data to obtain a strategy output neural network.
[0042] In an embodiment of the present application, a training strategy output neural network is first constructed, and the strategy output neural network to be trained is specifically a deep Q network (Deep Q-Networks, DQN), which can predict future benefits (such as reducing delays, improving throughput, etc.) under different scheduling strategies according to the current network state and traffic demand. By constantly interacting with the environment (i.e., the mine wireless communication network), the strategy output neural network to be trained adjusts network parameters according to feedback (such as actual network performance changes) and gradually optimizes the traffic scheduling strategy. This step can achieve efficient distribution and load balancing of network traffic and improve network resource utilization.
[0043] Step S103, the current traffic data of each base station, terminal device and network node and the current status information and current environment information of each device are input into the strategy output neural network to obtain the traffic link scheduling strategy to be updated.
[0044] Specifically, the policy output neural network that is continuously trained and optimized through a large amount of historical data and real-time feedback can predict changes in network performance under different traffic scheduling strategies. The trained policy output neural network can automatically adjust the traffic link scheduling strategy according to the current network status and traffic demand to achieve efficient distribution and load balancing of network traffic. By dynamically adjusting the bandwidth allocation, priority setting and other parameters of each link, network congestion and high load problems in hot spots can be avoided, improving the stability and response speed of the network.
[0045] The strategy output neural network simultaneously executes the traffic conservation equation and the link capacity limitation equation when outputting the traffic link scheduling strategy to be updated.
[0046] The Q value of the deep Q network is updated by the following formula:
[0047]
[0048] In the formula, Q(s t ,a t ) is in state s t Take action a t Q value, α is the learning rate, r t+1 For state s t Take action a t The instant reward, γ is the discount factor, For the next state s t-1 The maximum Q value of all actions a′ in .
[0049] The loss function L(θ) in the deep Q network is specifically shown as follows:
[0050]
[0051] In the formula, Q(s,a;θ) is the predicted Q value of the current network selecting action a in state s, r is the immediate reward, γ is the discount factor, The maximum value of the predicted Q value when selecting all actions a' in the next state s', θ - is the target network parameter, and θ is the network parameter.
[0052] In addition, when the strategy output neural network outputs the traffic link scheduling strategy to be updated, it simultaneously executes the traffic conservation equation and the link capacity limitation equation. The traffic conservation equation is specifically shown in the following formula:
[0053]
[0054] Where E is the edge set f from node u to node v uv The flow rate, Ef vw is the edge set f from node u to node w vw of traffic.
[0055] The link capacity limitation equation is as follows:
[0056] 0≤f uv ≤c uv
[0057] Among them, f uv is the link traffic from node u to node v, and cuv is the link capacity from node u to node v.
[0058] Among them, the strategy output neural network also includes a linear programming model for the maximum flow problem, and its objective function is:
[0059] max{f st}
[0060] The model is to maximize the flow from source point s to sink point t.
[0061] The above linear programming model also includes constraints and capacity restrictions. The constraints are shown in the following formula:
[0062]
[0063] Where, Ef sv is the edge set f from node s to node v sv The flow rate, EF vt is the edge set f from node v to node t vt The flow rate is conserved at the sink.
[0064] The capacity limit is as follows:
[0065]
[0066] Step S104: Determine whether to execute the traffic link scheduling strategy to be updated through a strategy evaluation algorithm.
[0067] In the embodiment of the present application, the policy evaluation algorithm is specifically a proximal policy optimization algorithm (PPO). According to the above-mentioned traffic link scheduling strategy to be updated and the real-time data of the link status, the PPO algorithm can accurately predict the potential network failure risk. Through the predicted network failure risk, it is determined whether to execute the traffic link scheduling strategy to be updated.
[0068] In the embodiment of the present application, determining whether to execute the traffic link scheduling strategy to be updated by a policy evaluation algorithm specifically includes:
[0069] Determine whether a network failure occurs in the traffic link scheduling strategy to be updated by using the strategy evaluation algorithm;
[0070] If not, the traffic link scheduling strategy to be updated is updated to the mining network for execution;
[0071] If so, the strategy evaluation algorithm selects the optimal traffic link scheduling strategy from the executed traffic link scheduling strategy library and updates it to the mining network for execution.
[0072] Specifically, this module builds a fault prediction model by analyzing multi-dimensional information such as historical fault records, current network load conditions, and equipment health conditions, and identifies possible link interruptions or performance degradation events in advance. Through small policy updates, the PPO algorithm gradually optimizes the link backup strategy to ensure that it can quickly switch to the best backup link when a network failure occurs, thereby ensuring the continuity and stability of communication. Once the risk of network failure is predicted, the PPO algorithm will immediately start the link backup mechanism. By quickly evaluating the availability and performance of existing backup links, the PPO algorithm can quickly select the optimal backup link to ensure seamless switching when a network failure occurs, thereby ensuring the continuity and stability of communication. In addition, the PPO algorithm also has self-learning capabilities, and can continuously optimize the backup strategy based on the results of each fault handling, thereby improving the adaptability and reliability of the system.
[0073] In the embodiment of the present application, the reliability function in the proximal strategy optimization algorithm is specifically shown as follows:
[0074]
[0075] In the formula, R system is the system reliability, n is the number of components, R i For component reliability.
[0076] The fault recovery time MTTR in the proximal strategy optimization algorithm = fault detection time + backup link switching time + data retransmission time.
[0077] Link resource consumption function: Cost uv =α·Delay uv +β·LossRate uv , among which Cost uv is the link consumption between node u and node v, α and β are weight coefficients, Delay uv is the link delay between node u and node v, LossRate uv is the service loss rate between node u and node v. Among them, the fault recovery time and link resource consumption function are used as a parameter in the link and returned to the proximal strategy optimization algorithm as a reward for evaluation. The convergence process is the process of continuously reducing link consumption.
[0078] The objective function L in the proximal strategy optimization algorithm CLIP (θ') is specifically shown in the following formula:
[0079]
[0080] In the formula, E t To find the expectation for all time steps t on the trajectory, r t (θ') is the ratio of the probability of taking action under a given state between the traffic link scheduling strategy to be updated and the traffic link scheduling strategy being executed, is the estimated value of the advantage function, and clip is expressed as r t (θ') sheared to (r t (θ'),1-∈,1+∈), where ∈ is a hyperparameter.
[0081] Among them, the advantage function A π (s t ,a t ) is shown in the following formula:
[0082] A π (s t ,a t )=Q π (s t ,a t )-V π (s t )]
[0083] In the formula, Q π (s t ,a t ) is the expected reward of taking action at in state st, V π (st ) is the average expected return in state st.
[0084] The proximal strategy optimization algorithm also includes a gradient update formula, which is as follows:
[0085]
[0086] Where N is the number of trajectories in the batch, Tn is the length of the nth trajectory, is a function of state st and action at under the old strategy, π θ (a t |s t ) is the current policy, where θ is the parameter to be optimized. The advantage function and gradient update formula are automatically updated when the proximal policy optimization algorithm determines the traffic link scheduling policy to be updated.
[0087] In some optional embodiments, the method further includes:
[0088] After the mining network executes the updated traffic link scheduling strategy, the current traffic data of each base station, terminal device and network node and the current status information of each device and the current environmental information are obtained in real time to train and update the strategy output neural network, and the latest current traffic data of each base station, terminal device and network node and the current status information of each device and the current environmental information are input into the updated strategy output neural network to obtain the current traffic link scheduling strategy to be updated.
[0089] Specifically, after the strategy output neural network is trained once, it can be trained according to real-time data. Through a large amount of historical data and real-time feedback, the network can be continuously trained and optimized to predict changes in network performance under different traffic scheduling strategies, thereby achieving precise control of traffic scheduling.
[0090] Based on the above-mentioned intelligent scheduling method for mining network traffic, one or more embodiments of this specification also provide a platform and terminal for intelligent scheduling of mining network traffic. The platform or terminal may include devices, software, modules, plug-ins, servers, clients, etc. using the methods described in the embodiments of this specification and combined with necessary implementation hardware devices. Based on the same innovative concept, the system in one or more embodiments provided in the embodiments of this specification is as described in the following embodiments. Since the implementation scheme and method for solving the problem of the system are similar, the implementation of the specific system in the embodiments of this specification can refer to the implementation of the aforementioned method, and the repetitions will not be repeated. The terms "unit" or "module" used below can implement a combination of software and / or hardware with predetermined functions. Although the system described in the following embodiments is preferably implemented in software, hardware, and the implementation of a combination of software and hardware is also possible and conceived.
[0091] Specifically, Figure 2 This is a schematic diagram of the module structure of an embodiment of the intelligent scheduling device for mining network traffic provided in this specification. Figure 2 As shown, the mining network flow intelligent scheduling device provided in this specification includes:
[0092] The acquisition module 201 is used to acquire the historical flow data of each base station, terminal device and network node in the mine, and acquire the historical status information and historical environment information of each device;
[0093] A training module 202, configured to train the strategy output neural network to be trained using the historical traffic data to obtain a strategy output neural network;
[0094] Strategy module 203, used to input the current traffic data of each base station, terminal device and network node and the current status information and current environment information of each device into the strategy output neural network to obtain the traffic link scheduling strategy to be updated;
[0095] The determination module 204 is used to determine whether to execute the traffic link scheduling policy to be updated through a policy evaluation algorithm.
[0096] It should be noted that the above-mentioned system may also include other implementation modes according to the description of the corresponding method embodiment. The specific implementation modes may refer to the description of the above-mentioned corresponding method embodiment, which will not be described one by one here.
[0097] The present application also provides an electronic device, including:
[0098] processor;
[0099] a memory for storing instructions executable by the processor;
[0100] The processor is configured to execute the method provided in the above embodiment.
[0101] The electronic device provided in the embodiment of the present application stores the executable instructions of the processor in a memory. When the processor executes the executable instructions, it can first obtain the historical traffic data of each base station, terminal device and network node in the mine, and at the same time obtain the historical status information and historical environmental information of each device; then train the strategy output neural network to be trained through the historical traffic data to obtain the strategy output neural network; then input the current traffic data of each base station, terminal device and network node and the current status information and current environmental information of each device into the strategy output neural network to obtain the traffic link scheduling strategy to be updated; finally, determine whether to execute the traffic link scheduling strategy to be updated through a strategy evaluation algorithm, so that the mine communication network can cope with the complex and changeable network environment in the mine, improve the working stability of the mine communication network, and automatically control the traffic path in the mine network to improve the control speed.
[0102] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0103] The methods or devices described in the above embodiments provided in this specification can implement business logic through computer programs and record them on storage media, and the storage media can be read and executed by computers to achieve the effects of the solutions described in the embodiments of this specification, such as:
[0104] Obtain historical traffic data of each base station, terminal equipment and network node in the mine, and obtain historical status information and historical environment information of each device;
[0105] The strategy output neural network to be trained is trained by using the historical traffic data to obtain a strategy output neural network;
[0106] The current traffic data of each base station, terminal device and network node and the current status information and current environment information of each device are input into the strategy output neural network to obtain the traffic link scheduling strategy to be updated;
[0107] Whether to execute the traffic link scheduling strategy to be updated is determined through a strategy evaluation algorithm.
[0108] The storage medium may include a physical device for storing information, which is usually a medium that digitizes information and then stores it in an electrical, magnetic or optical manner. The storage medium may include: a device that stores information in an electrical energy manner, such as various memories, such as RAM, ROM, etc.; a device that stores information in a magnetic energy manner, such as a hard disk, a floppy disk, a magnetic tape, a magnetic core memory, a magnetic bubble memory, a USB flash drive; a device that stores information in an optical manner, such as a CD or a DVD. Of course, there are other readable storage media, such as quantum memory, graphene memory, etc.
[0109] The embodiments of this specification are not limited to complying with industry communication standards, standard computer resource data update and data storage rules, or the situations described in one or more embodiments of this specification. Certain industry standards or slightly modified implementation plans based on the implementation described in the custom method or embodiment can also achieve the same, equivalent or similar, or predictable implementation effects after deformation of the above-mentioned embodiments. The embodiments obtained by using these modified or deformed data acquisition, storage, judgment, processing methods, etc. can still fall within the scope of the optional implementation plans of the embodiments of this specification.
[0110] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a computer-readable medium such as a microprocessor or processor and a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, ATMEL AT91SAM, MICROCHIP PIC18F26K20, and SILICONE LABS C8051F320. The memory controller can also be implemented as part of the control logic of the memory. It is also known to those skilled in the art that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules for implementing the method and structures within the hardware component.
[0111] The device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. There may be other divisions in actual implementation, such as multiple units or plug-ins may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some interface, indirect coupling or communication connection of the device or unit, which may be electrical, mechanical or other forms.
[0112] These computer program instructions may also be loaded onto a computer or other programmable resource data updating device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions executed on the computer or other programmable device for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0113] Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. In the description of this specification, the description of the reference term "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this specification. In this specification, the schematic representation of the above terms does not necessarily target the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction.
[0114] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.
Claims
1. A method for intelligent scheduling of mining network traffic, characterized in that: The method comprises: Obtain historical traffic data of each base station, terminal equipment and network node in the mine, and obtain historical status information and historical environment information of each device; The strategy output neural network to be trained is trained by using the historical traffic data to obtain a strategy output neural network; The current traffic data of each base station, terminal device and network node and the current status information and current environment information of each device are input into the strategy output neural network to obtain the traffic link scheduling strategy to be updated; Whether to execute the traffic link scheduling strategy to be updated is determined through a strategy evaluation algorithm.
2. The method for intelligent scheduling of mining network traffic according to claim 1, characterized in that: The determining whether to execute the traffic link scheduling strategy to be updated by the strategy evaluation algorithm specifically includes: Determine whether a network failure occurs in the traffic link scheduling strategy to be updated by using the strategy evaluation algorithm; If not, the traffic link scheduling strategy to be updated is updated to the mining network for execution; If so, the strategy evaluation algorithm selects the optimal traffic link scheduling strategy from the executed traffic link scheduling strategy library and updates it to the mining network for execution.
3. The method for intelligent scheduling of mining network traffic according to claim 2, characterized in that: The method further comprises: After the mining network executes the updated traffic link scheduling strategy, the current traffic data of each base station, terminal device and network node and the current status information of each device and the current environmental information are obtained in real time to train and update the strategy output neural network, and the latest current traffic data of each base station, terminal device and network node and the current status information of each device and the current environmental information are input into the updated strategy output neural network to obtain the current traffic link scheduling strategy to be updated.
4. The intelligent dispatching method for mining flow according to claim 1, characterized in that: The strategy output neural network simultaneously executes the traffic conservation equation and the link capacity limitation equation when outputting the traffic link scheduling strategy to be updated.
5. The intelligent dispatching method for mining flow according to claim 1, characterized in that: The strategy output neural network to be trained is specifically a deep Q network, and the strategy evaluation algorithm is specifically a proximal strategy optimization algorithm.
6. The intelligent dispatching method for mining flow according to claim 5, characterized in that: The Q value of the deep Q network is updated by the following formula: In the formula, Q(s t ,a t ) is in state s t Take action a t Q value, α is the learning rate, r t+1 For state s t Take action a t The instant reward, γ is the discount factor, For the next state s t-1 The maximum Q value of all actions a′ in .
7. The intelligent dispatching method for mining flow as claimed in claim 5, characterized in that: The loss function L(θ) in the deep Q network is specifically shown as follows: In the formula, Q(s, a; θ) is the predicted Q value of the current network selecting action a in state s, r is the immediate reward, γ is the discount factor, It is the maximum value of the predicted Q value when all actions a′ are selected in the next state s′, θ- is the target network parameter, and θ is the network parameter.
8. The method for intelligent dispatching of mining traffic according to claim 5, characterized in that: The objective function L in the proximal strategy optimization algorithm CLIP (θ′) is specifically shown in the following formula: In the formula, E t To find the expectation for all time steps t on the trajectory, r t (θ′) is the ratio of the probability of taking action under a given state between the traffic link scheduling strategy to be updated and the traffic link scheduling strategy being executed, is the estimated value of the advantage function, and clip is expressed as r t (θ′) sheared to (r t (θ′), 1-∈, 1+∈), ∈ is a hyperparameter.
9. A mining flow intelligent dispatching device, characterized in that: The device comprises: The acquisition module is used to obtain the historical traffic data of each base station, terminal equipment and network node in the mine, and obtain the historical status information and historical environment information of each device; A training module, used for training the strategy output neural network to be trained by using the historical traffic data to obtain a strategy output neural network; A strategy module, used to input the current traffic data of each base station, terminal device and network node and the current status information and current environment information of each device into the strategy output neural network to obtain the traffic link scheduling strategy to be updated; The determination module is used to determine whether to execute the traffic link scheduling strategy to be updated through a strategy evaluation algorithm.