Offline data loading method based on dynamic priority adjustment

Abstract

The invention discloses an off-line data loading method based on dynamic priority adjustment, which belongs to the technical field of electric digital data processing, and comprises the following steps: a user accesses overall data through equipment; defining the comprehensive value of each data block in the overall data; creating a data block loading process as a Markov decision process, and defining a state, an action and an award; establishing a main network and a target network based on a DQN algorithm; executing action generation experience at each time step; training the main network and the target network based on experience playback; and accessing the whole data by the user, and selecting the next loadable data block by using the trained main network. Based on the method, when a user accesses the whole data, the loading priority and strategy can be automatically adjusted according to real-time feedback and monitoring requirements, and the processing speed and the processing flexibility are improved by preferentially loading key data blocks and reducing redundant calculation.

Description

Technical Field

[0001] The invention relates to the technical field of electronic digital data processing, and in particular to an offline data loading method based on dynamic priority adjustment. Background Art

[0002] In recent years, with the rapid development of technologies such as the Internet of Things and artificial intelligence, various devices continue to generate large amounts of data. Especially in offline scenarios with limited resources, unstable networks, or batch processing, how to efficiently and timely obtain and manage this data has become an urgent problem to be solved. In the fields of forestry resource monitoring, geospatial analysis, disaster management, etc., the scale of remote sensing data sets is increasing, bringing huge storage and processing challenges. How to reduce the amount of data while ensuring data quality and improve data processing efficiency has become a key issue in current data management.

[0003] Traditional data loading and management methods often rely on preset static strategies, and it is difficult to flexibly adjust the priority of data loading according to changes in actual use, resulting in waste of resources, low loading efficiency, and even affecting device performance and user experience. Especially in complex offline environments, traditional methods cannot effectively respond to dynamically changing needs, which in turn affects the response speed and overall performance of the system. For example, the invention patent application with application number 202411388666.9 uses a solidified grid model that cannot be updated dynamically, affecting accuracy. Especially when processing high-resolution remote sensing data, the data processing speed is slow.

[0004] DQN (Deep Q-Network) is an algorithm that combines deep learning and reinforcement learning to solve control problems in Markov decision processes (MDPs). It is based on the idea of ​​value iteration and guides the agent to choose the best action in each state by estimating the value function (Q-value function) of each state-action pair. The DQN algorithm uses two key networks: the main network and the target network. Both networks are essentially deep neural networks, but they have different functions: the main network is used to predict the expected reward, that is, the Q value, given a state (s) and action (a). The Q value represents the cumulative reward that the agent expects to obtain after taking an action. The main network learns by interacting with the environment and continuously updates its parameters to better predict the Q value. The target network is an auxiliary network whose parameters (θ') are a copy of the main network parameters (θ). The purpose of the target network is to provide a stable training target for the DQN algorithm. It uses the same structure as the main network, but the weights are updated more slowly. The target network usually directly copies the parameters of the main network after a certain number of training steps. Summary of the invention

[0005] The purpose of the present invention is to provide a solution to the above-mentioned problem, breaking the traditional data loading mechanism based on static rules, and being able to automatically adjust the loading priority according to the real-time needs of the device and the user, and an offline data loading method based on dynamic priority adjustment.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is as follows: an offline data loading method based on dynamic priority adjustment comprises the following steps; S1, users access overall data through devices , where the i-th data block The data space occupied is , the available space for the device to access the overall data D is C, 1≤i≤N; S2, Definition The comprehensive value ; , , , in, is the weight parameter, and , for The expected access probability of for The probability of importance, for Importance rating, for The unvisited state score of , e is a natural constant, , are the mean and standard deviation of N importance scores respectively; S3, creates the data block loading process as a Markov decision process, defining states, actions, and rewards; The status includes a set of loaded data block indexes and remaining available space on the device; The action is to select the next loadable data block, wherein the loadable data block is a data block that has not been loaded and occupies a data space smaller than the remaining available space; The reward is the comprehensive value of the loadable data block selected in the action; S4, based on the DQN algorithm, a main network and a target network are established, and the network structures of the two are the same. The input of the main network is the state, and the output is the Q function value of the action executed in the state; S5, performs actions at each time step to generate experience, where the experience at time step t is , , are the states at time steps t and t+1 respectively, Status The action that maximizes the Q function value, for The comprehensive value of the corresponding loadable data block; state ,action The corresponding Q function value According to the following formula: , Where T is the total number of preset time steps, Indicates in status and actions The expected calculation is performed below. is the discount factor, k is the time step k, t≤k≤T, The comprehensive value corresponding to time step k; S6, preset iteration rounds, target network update time interval , based on experience replay training main network, to minimize the loss function L to update the main network parameters, and when reaching When , the parameters of the target network are updated with the parameters of the main network until the iteration round is reached, and the trained main network and target network are obtained; S7, the user accesses the overall data, uses the trained main network to obtain the current state, calculates the Q function value of each action in this state, and selects the action with the largest Q function value to execute.

[0007] As a preference: , Obtained according to the following method; S21, preset The initial importance score of , Initial unvisited status score , and all data blocks , same; S22, when the user accesses the overall data D multiple times, statistics Number of times visited by users , the number of times not visited , and update according to the following formula Importance Rating , Unvisited Status Score ; , .

[0008] As a preference: in step S4, generating the experience of time step t includes steps S41 to S43; S41, get the state of time step t , ,in is the index set of the loaded data blocks at time step t, is the remaining available space of the device at time step t; S42, in state Execute multiple actions at the same time, and mark the action with the largest Q function value as ,Will The comprehensive value of the loadable data block is marked as ; S43, generate experience at time step t ,in is the state at time step t+1 .

[0009] Preferably, in S6, the loss function L is obtained according to the following formula: , , In the formula, y is the state ,action The target value of In the target network, the state ,action The maximum value of the Q function when .

[0010] Preferably, the overall data is an image set consisting of forestry remote sensing images.

[0011] Compared with the prior art, the advantages of the present invention are: based on the DQN algorithm, the present invention determines the comprehensive value of the data block by real-time monitoring the access frequency of each data block in the device, thereby evaluating the importance of the data block; and determines the order and weight of data block loading according to the real-time comprehensive value of the data block, so that different data blocks have different priorities; finally, by dynamically allocating the priorities of different data blocks, the utilization efficiency of limited resources is maximized, ensuring that the device can quickly obtain the most important data in offline scenarios.

[0012] Therefore, the present invention breaks the traditional data loading mechanism based on static rules and adopts an adaptive optimization strategy, which can automatically adjust the loading priority according to the real-time needs of the device and the user.

[0013] Through this method, data management no longer relies on fixed loading rules, but can flexibly respond to various complex and dynamic application scenarios. Especially for large-scale data sets such as forestry remote sensing data, this method can significantly improve data loading efficiency in an offline environment, reduce system response time, improve the overall performance of the device, and reduce resource waste caused by overloading unimportant data. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0015] The present invention will be further described below in conjunction with embodiments.

[0016] Example 1: See Figure 1 ,An offline data loading method based on dynamic priority adjustment comprises the following steps; S1, users access overall data through devices , where the i-th data block The data space occupied is , the available space for the device to access the overall data D is C, 1≤i≤N; S2, Definition The comprehensive value ; , , , in, is the weight parameter, and , for The expected access probability of for The probability of importance, for Importance rating, for The unvisited state score of , e is a natural constant, , are the mean and standard deviation of N importance scores respectively; S3, creates the data block loading process as a Markov decision process, defining states, actions, and rewards; The status includes a set of loaded data block indexes and remaining available space on the device; The action is to select the next loadable data block, wherein the loadable data block is a data block that has not been loaded and occupies a data space smaller than the remaining available space; The reward is the comprehensive value of the loadable data block selected in the action; S4, based on the DQN algorithm, a main network and a target network are established, and the network structures of the two are the same. The input of the main network is the state, and the output is the Q function value of the action executed in the state; S5, performs actions at each time step to generate experience, where the experience at time step t is , , are the states at time steps t and t+1 respectively, Status The action that maximizes the Q function value, for The comprehensive value of the corresponding loadable data block; state ,action The corresponding Q function value According to the following formula: , Where T is the total number of preset time steps, Indicates in status and actions The expected calculation is performed below. is the discount factor, k is the time step k, t≤k≤T, The comprehensive value corresponding to time step k; S6, preset iteration rounds, target network update time interval , based on experience replay training main network, to minimize the loss function L to update the main network parameters, and when reaching When , the parameters of the target network are updated with the parameters of the main network until the iteration round is reached, and the trained main network and target network are obtained; S7, the user accesses the overall data, uses the trained main network to obtain the current state, calculates the Q function value of each action in the state, and selects the action with the largest Q function value to execute. Since the action in the present invention is to select the next loadable data block, the action with the largest Q function value is executed, that is, the loadable data block with the largest Q function value is loaded, thereby realizing the function of dynamically selecting data blocks based on dynamic priority in the present invention.

[0017] In this embodiment, , Obtained according to the following method; S21, preset The initial importance score of , Initial unvisited status score , and all data blocks , same; S22, when the user accesses the overall data D multiple times, statistics Number of times visited by users , the number of times not visited , and update according to the following formula Importance Rating , Unvisited Status Score ; , .

[0018] In step S4, generating experience at time step t includes steps S41 to S43; S41, get the state of time step t , ,in is the index set of the loaded data blocks at time step t, is the remaining available space of the device at time step t; S42, in state Execute multiple actions at the same time, and mark the action with the largest Q function value as ,Will The comprehensive value of the loadable data block is marked as ; S43, generate experience at time step t ,in is the state at time step t+1 .

[0019] In S6, the loss function L is obtained according to the following formula: , , In the formula, y is the state ,action The target value of In the target network, the state ,action The maximum value of the Q function when .

[0020] The overall data is an image set consisting of forestry remote sensing images.

[0021] Based on the method of the present invention, when users access the overall data, they can automatically adjust the loading priority and strategy according to real-time feedback and monitoring requirements to improve the flexibility of the model. By loading key data blocks first, redundant calculations are reduced and processing speed is improved.

[0022] Embodiment 2: In the DQN algorithm, the deep Q network includes a main network and a target network. This embodiment provides a specific process for training the deep Q network as follows: (1) Initialization: Environment: Select and initialize the environment to be trained, such as OpenAI Gym; Neural network: Initialize two neural network models, the main network and the target network, which have the same structure; Experience playback buffer: used to store experience; Parameters: Setting The initial importance score of , Initial unvisited status score , discount factor , learning rate a, exploration rate , other hyper parameters, etc.; In this embodiment , , =0.9.

[0023] (2) Perform training at each time step. The training at time step t includes the following (2.1) to (2.6); (2.1) At each time step t, an action is selected according to the current state, using - Greedy strategy: based on probability Randomly choose an action with probability Select the action with the largest Q function value as ; (2.2) Execute the action and observe the reward returned by the action and the next state ; (2.3) Experience Store in experience replay buffer; (2.4) Experience replay: randomly extract small batches of experience from the experience replay buffer; (2.5) For each experience, calculate the Q function value; (2.6) Use this small batch experience to update the main network parameters and reach When the number of time steps is large (such as 500 or 1000), copy the main network to the target network.

[0024] (3) Update exploration rate; (4) Check the termination conditions, such as when the preset time step is reached or the remaining available space is insufficient to load any unloaded data blocks, and then terminate the training.

[0025] (5) The data blocks selected at each time step form the final strategy in sequence, and according to the final strategy, the selected data block set is loaded.

[0026] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An offline data loading method based on dynamic priority adjustment, characterized in that: The steps include: S1, users access overall data through devices , where the i-th data block The data space occupied is , the available space for the device to access the overall data D is C, 1≤i≤N; S2, Definition The comprehensive value ; ， ， ， in, is the weight parameter, and , for The expected access probability of for The probability of importance, for Importance rating, for The unvisited state score of , e is a natural constant, , are the mean and standard deviation of N importance scores respectively; S3, creates the data block loading process as a Markov decision process, defining states, actions, and rewards; The status includes a set of loaded data block indexes and remaining available space on the device; The action is to select the next loadable data block, wherein the loadable data block is a data block that has not been loaded and occupies a data space smaller than the remaining available space; The reward is the comprehensive value of the loadable data block selected in the action; S4, based on the DQN algorithm, a main network and a target network are established, and the network structures of the two are the same. The input of the main network is the state, and the output is the Q function value of the action executed in the state; S5, performs actions at each time step to generate experience, where the experience at time step t is , , are the states at time steps t and t+1 respectively, Status The action that maximizes the Q function value, for The comprehensive value of the corresponding loadable data block; state ,action The corresponding Q function value According to the following formula: ， Where T is the total number of preset time steps, Indicates in status and actions The expected calculation is performed below. is the discount factor, k is the time step k, t≤k≤T, The comprehensive value corresponding to time step k; S6, preset iteration rounds, target network update time interval , based on experience replay training main network, to minimize the loss function L to update the main network parameters, and when reaching When , the parameters of the target network are updated with the parameters of the main network until the iteration round is reached, and the trained main network and target network are obtained; S7, the user accesses the overall data, uses the trained main network to obtain the current state, calculates the Q function value of each action in this state, and selects the action with the largest Q function value to execute.

2. The offline data loading method based on dynamic priority adjustment according to claim 1, characterized in that: , Obtained according to the following method; S21, preset The initial importance score of , Initial unvisited status score , and all data blocks , same; S22, when the user accesses the overall data D multiple times, statistics Number of times visited by users , the number of times not visited , and update according to the following formula Importance Rating , Unvisited Status Score ; ， 。 3. The offline data loading method based on dynamic priority adjustment according to claim 1, characterized in that: In step S4, generating experience at time step t includes steps S41 to S43; S41, get the state of time step t , ,in is the index set of the loaded data blocks at time step t, is the remaining available space of the device at time step t; S42, in state Execute multiple actions at the same time, and mark the action with the largest Q function value as ,Will The comprehensive value of the loadable data block is marked as ; S43, generate experience at time step t ,in is the state at time step t+1 .

4. The offline data loading method based on dynamic priority adjustment according to claim 1, characterized in that: In S6, the loss function L is obtained according to the following formula: ， ， In the formula, y is the state ,action The target value of For the target network, the state ,action The maximum value of the Q function when .

5. The offline data loading method based on dynamic priority adjustment according to claim 1, characterized in that: The overall data is an image set consisting of forestry remote sensing images.

Images