An optimization method for discretization of remote sensing data based on deep Q-learning

Abstract

The present invention provides an optimization method for remote sensing data discretization based on deep Q-learning. The method comprises: extracting features from remote sensing images to obtain a feature subset, preprocessing the feature subset to obtain a candidate breakpoint set, and performing binary genetic encoding on the candidate breakpoint set to construct a discretization scheme to be optimized; designing state sets for the crossover phase and the mutation phase through the global and local search capabilities of genetic operators; constructing a fitness function with control variables and an adaptive reward function; and introducing a pair of deep Q-learnings with the same structure to perform real-time calculation of Q values ​​during the transition between the crossover state and the mutation state, thereby determining the crossover segments and mutation points of the discretization scheme to be optimized. By introducing a pair of deep Q-learnings with the same structure to the discretization scheme to be optimized, the present invention improves the search efficiency of the feature discretization method based on the evolutionary model, and can further reduce the number of breakpoints while achieving higher classification accuracy.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing image feature extraction, and in particular to an optimization method for remote sensing data discretization based on deep Q-learning. Background Art

[0002] Remote sensing data, derived from a variety of sensors, is multidimensional, incomplete, random, fuzzy, and subject to strong interference. Currently, much research aims to extract valuable information from this massive and complex collection of remote sensing data and effectively process it. Although neural networks have made significant progress in remote sensing image classification and segmentation, they still face challenges such as high computational complexity, low information utilization, noise interference, insufficient generalization, and a lack of interpretability.

[0003] Feature discretization, one of the most influential data preprocessing techniques, can reduce data complexity by converting continuous features in massive data sets into discrete ones, resulting in shorter, more precise, and easier-to-understand rules. This improves the efficiency of data mining and machine learning algorithms in remote sensing data processing. However, the distribution of target attribute values ​​is often difficult to determine, and features exhibit complex correlations, making it extremely challenging to obtain optimal discretization schemes for remote sensing data. While evolutionary model-based feature discretization methods can achieve promising results, the lack of prior knowledge makes it difficult to formulate appropriate strategies, making the search in multidimensional space inefficient and prone to trapping in local optima. Q-learning, an important reinforcement learning technique, has been widely used to solve optimization problems in Markov decision processes (MDPs). In one study, a Q-learning-based GA was used to optimize discretization schemes for remote sensing data. Each feature subset of a remote sensing image was mapped to a state, and a Q-learning mechanism was introduced during crossover and mutation operations to identify the crossover segments and mutation points of the discretization scheme to be optimized. However, the value range of each feature in remote sensing images is huge, and the Q-tables established for each feature subset cannot carefully characterize the different cross-segments and variation points in each feature, making it difficult to converge the update of Q-values, ultimately leading to suboptimal solutions. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to propose a genetic algorithm based on deep Q-learning to optimize the discretization scheme of remote sensing data.

[0005] To achieve the above-mentioned object of the invention, the present invention provides an optimization method for discretization of remote sensing data based on deep Q-learning, the method comprising:

[0006] S11, extracting features from the remote sensing image to obtain a feature subset, preprocessing the feature subset to obtain a candidate breakpoint set, and performing evolutionary processing on the candidate breakpoint sets respectively;

[0007] S12, inputting the candidate breakpoint set into GA for iterative evolution, saving the global optimal individual in each crossover and each mutation, and then performing binary genetic encoding on the candidate breakpoint set to construct a discretization scheme to be optimized;

[0008] S13, performing state mapping on a feature subset of the remote sensing data. In the crossover phase, a two-dimensional matrix consisting of the feature subset and crossover segments of fixed length on the features contained therein is mapped into a state. In the mutation phase, only the feature subset is mapped into a state, and a mutation point of the current state is obtained based on the absolute dissimilarity between the local optimal individual and the global optimal individual in the form of Gaussian sampling.

[0009] S14. Obtain the fitness value of the individual state by constructing a fitness function with control variables to evaluate the quality of the evolution of the discretization scheme to be optimized, and realize the conversion between the crossover state and the mutation state by constructing an adaptive reward function;

[0010] S15. Introduce a pair of deep Q-learning algorithms with the same structure to calculate the Q value in real time during the transition between the crossover state and the mutation state. At the same time, sample the Q value optimization samples of the discretization scheme to be optimized from the experience replay set to iteratively update the network parameters and determine the crossover segments and mutation points of the discretization scheme to be optimized.

[0011] S16. Perform genetic operations on the crossover segments and mutation points determined in step S15 and the global optimal individual in step S12 to optimize the discretization scheme to be optimized and obtain the optimal discretization scheme.

[0012] Furthermore, in step S11, the method of preprocessing the feature subset to obtain a candidate breakpoint set includes: the feature subset includes the position information of all pixels and the pixel values ​​of all pixels in each band, and all pixels contained in each band are sorted and deduplicated within the band according to the brightness value to obtain a candidate breakpoint set.

[0013] Furthermore, in step S12, performing binary genetic encoding on the candidate breakpoint set specifically includes:

[0014] S21, corresponding each candidate breakpoint in the candidate breakpoint set to each bit in the binary code;

[0015] S22, selecting a candidate breakpoint whose corresponding binary code value is 1, and discarding a candidate breakpoint whose corresponding binary code value is 0;

[0016] S23. Construct a discretization scheme, where the discretization scheme is a set of selected candidate breakpoints.

[0017] Furthermore, in step S13, the processing of the crossover phase specifically includes:

[0018] S31. Assuming that the remote sensing image contains m features, there are 2 m -1 feature subset, each subset of the candidate breakpoint set is a cross segment, and each cross segment corresponds to a state, then there is states, where n k is the number of candidate breakpoints on the k-th band;

[0019] S32, set the candidate breakpoints to be crossed on each feature to a fixed length L c The fixed length expression is:

[0020] L c =min 1≤k≤m {n k} / 3

[0021] The number of states generated is:

[0022]

[0023] S33, the feature subset and the features it contains are of length L c The two-dimensional matrix composed of the cross fragments is mapped into a state, and all states constitute the state set in the cross stage.

[0024] Furthermore, in step S13, the variation point of the current state is obtained, which specifically includes:

[0025] S41. The mutation state is sampled in the form of Gaussian sampling, and the mutation point of the current state is obtained according to the absolute dissimilarity between the local optimal individual and the global optimal individual. The absolute dissimilarity of chromosome x in state s is:

[0026]

[0027] Among them, n i is the number of candidate breakpoints on the i-th band in state s, b(i,j) is the value of the j-th binary bit on the i-th band;

[0028] S42. Calculate the absolute difference between the historical best individual of each chromosome and the global best individual in the population in the current state:

[0029]

[0030] sum AD(l,g,s)=AD(l,s)+AD(g,s)

[0031] dif AD (l,g,s)=AD(l,s)-AD(g,s)

[0032] Among them, AD T () is the theoretical absolute dissimilarity function, AD() is the absolute dissimilarity function, l is the historical optimal individual of a given chromosome, g is the global optimal individual in the population, dif AD () is the absolute dissimilarity function difference function, sum AD () is the absolute dissimilarity function summation function, AD(l,s) is the absolute dissimilarity of the historical optimal individual of a given chromosome in state s, AD(g,s) is the absolute dissimilarity of the global optimal individual in the population in state s, rand() is used to generate a random number uniformly distributed between 0 and 1;

[0033] S43. In state s, the absolute dissimilarity of chromosome x after mutation follows a Gaussian distribution with mean μ = (AD(l,s) + AD(g,s)) / 2 and standard deviation σ = |AD(l,s) - AD(g,s)|. Then the number of mutation points MN(x,s) of chromosome x is:

[0034]

[0035] dif AD (x, l, s) = AD (x, s) - AD (l, s)

[0036] Among them, MN() is the number of mutation points function, round() is the rounding function, and AD(x,s) is the absolute dissimilarity of chromosome x in state s.

[0037] Furthermore, in step S14, a fitness function with control variables is constructed, specifically including:

[0038] S51. The indistinguishability relationship constitutes the division of feature subsets. The fitness function is determined by the number of breakpoints of the discretization scheme and the change of the indistinguishability relationship:

[0039]

[0040] Among them, D is the discretization scheme, C is the conditional attribute set, N C is the number of breakpoints in the candidate breakpoint set, N D is the number of breakpoints obtained after chromosome decoding, R D is the change of the upper and lower approximation sets obtained after discretizing the data using D, R DThe value of is "0" or "1", indicating that the upper and lower approximation sets of D have changed or have not changed after discretization;

[0041] S52. Add control variables to expand the fitness function:

[0042]

[0043] Among them, x is the chromosome, is the number of breakpoints obtained after x is decoded, is the change of the upper and lower approximation sets of x with respect to D after discretization, and p is R D The proportion of potential feasible solutions of “0” in the population,

[0044]

[0045] Among them, z is the control item index, k is the number of categories, d i For category i, C l (d i ) is the d before discretization i The lower approximation set, C u (d i ) is the d before discretization i The upper approximation set of C l (d i )' is the discretized d i The lower approximation set, C u (d i )' is the discretized d i The upper approximate set of , setcmp() is the set comparison function.

[0046] Furthermore, in step S14, the changing pattern of the optimal discretized cases is simulated to construct an adaptive reward function in the reinforcement learning model, specifically including:

[0047] The total number of states in S61, crossover phase and mutation phase is:

[0048]

[0049] S62, current status s t By performing action a t Jump to the next state s t+1 , whose expression is:

[0050] (s t ,a t )→s t+1

[0051] S63. Construct an adaptive reward function by realizing the transition between the crossover state and the mutation state. That is, compare the fitness of the next state with the fitness of the historical best individual to set the reward function after the current state executes the action:

[0052]

[0053] Among them, r() is the reward function, r(s t ,a t ) is state s t Execute action a t The reward value obtained, t is the number of iterations of genetic operation, s t is the state under the tth genetic operation, s t+1 Execute action a for the state under the tth genetic operation t The next state to jump to, l is the historical optimal individual, and fit() is the fitness function.

[0054] Furthermore, in step S15, a pair of deep Q-learning algorithms with the same structure are introduced to update the Q value during the transition between the crossover state and the mutation state, specifically including:

[0055] S71. In the current state, the ε-greedy strategy is used to select the next action based on the Q value corresponding to each action, and the Q value is updated at each step in the learning process. Its expression is:

[0056]

[0057] Among them, a t+1 is the next action, A is the action set, γ is the discount factor, 0≤γ≤1;

[0058] S72, divide the feature subset into 2 m -1 state, m is the number of bands, each state contains several band combinations, that is, the action set A = {G, H, I} covering all states is:

[0059]

[0060] Among them, G is the subset acquisition function of the state, H is the subset acquisition function of the state complement, I is the set acquisition function that intersects with the state and is not a subset of the state, G(s t ) is a random jump to s t One of all subsets of t ) is a random jump to s t One of all subsets of the complement, I(s t ) is a random jump to s t The intersection of is not empty and is not s tOne of all sets of subsets, the w appearing in the three formulas are the required sets that satisfy the above three constraints;

[0061] S73. Evaluate the search for the optimal solution by the Q network by setting a reward function for individual fitness changes, wherein the Q network is deep Q-learning:

[0062]

[0063] Among them, fit(s t+1 ) is the individual fitness of the next state, and fit(l) is the historical best fitness of the discretization scheme to be optimized;

[0064] S74, introduce two Q networks with the same structure to obtain the Q value, that is, the current Q network Q c and target Q network Q T , then s t The target Q value and Q c The loss functions are:

[0065]

[0066] Among them, θ T Q T The network parameters, θ C Q c Network parameters, is_end t The value of "1" means s t+1 Is the terminal state, is_end t The value of "0" means s t+1 is a non-terminal state, h is the number of samples sampled from the experience replay set E;

[0067] S75, by setting a fixed time interval w, Q c θ C Make a copy to update Q T θ T , that is, θ C It is updated by gradient descent, executing θ every w time T =θ C ;

[0068] S76. Perform real-time calculation of the Q value of two Q networks with the same structure during the transition between the crossover state and the mutation state. At the same time, sample the Q value optimization samples of the discretization scheme to be optimized from the experience replay set to iteratively update the network parameters and determine the crossover segments and mutation points of the discretization scheme to be optimized.

[0069] Compared with the prior art, the present invention has the following beneficial effects:

[0070] (1) The two-dimensional matrix consisting of the feature subset and the fixed-length cross-segments of the features it contains is mapped into a cross state, and only the feature subset is mapped into a mutation state. The mutation point of the current state is obtained according to the absolute dissimilarity between the local optimal individual and the global optimal individual in the form of Gaussian sampling, thereby improving the search efficiency of the evolutionary model.

[0071] (2) An action set that covers the entire band is used to realize the transition between the crossover state and the mutation state, and the changing pattern of the optimal discretization scheme is simulated to construct an adaptive reward function, thereby reflecting the evolutionary trend of the genetic population.

[0072] (3) By introducing a pair of deep Q networks with the same structure to calculate the Q values ​​of all states, the cross-segments and mutation points of the discretization scheme to be optimized are determined, thereby solving the storage difficulties of massive Q values ​​and the dimensionality curse problem.

[0073] (4) An adaptive reward function is constructed by constructing a fitness function with control variables and simulating the change pattern of the optimal discretization scheme, thereby maintaining the diversity of the population and following the evolutionary laws of individuals.

[0074] (5) By designing two different genetic evolutionary routes, the optimal evolved individuals of each route are selected for genetic manipulation to evolve high-quality individuals to optimize the discretization of remote sensing data. This significantly improves the search efficiency of the discretization method based on the evolutionary model, and can further reduce the number of breakpoints while achieving higher classification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0076] Figure 1 A flowchart of an optimization method for discretizing remote sensing data based on deep Q-learning is provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0077] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0078] Reference Figure 1The present invention provides an optimization method for discretization of remote sensing data based on deep Q-learning, the method comprising:

[0079] S11. Extract features from the remote sensing image to obtain a feature subset, pre-process the feature subset to obtain a candidate breakpoint set, and perform evolutionary processing on the candidate breakpoint sets respectively.

[0080] As a preferred embodiment, the present invention performs discrete processing on the candidate breakpoint sets respectively. The first route uses conventional genetic evolution on the candidate breakpoint sets, and the second route constructs a discretization scheme to be optimized after processing the candidate breakpoint sets, and performs subsequent genetic evolution processing on it.

[0081] In step S11, the method of preprocessing the feature subset to obtain a candidate breakpoint set includes: the feature subset includes the position information of all pixels and the pixel values ​​of all pixels in each band, all pixels contained in each band are sorted and deduplicated according to the brightness value within the band to obtain a candidate breakpoint set.

[0082] In this embodiment, feature discretization is a process of dividing continuous attribute values ​​into a finite number of small intervals according to a certain specific rule, and associating these small intervals with a set of discrete values. After extracting features from the original remote sensing image, the pixel values ​​in each band are obtained. The normalized pixel values ​​are a set of continuous values ​​ranging from 0 to 1, and each band in the image represents a continuous feature. The goal of remote sensing image feature discretization is to use as few discrete intervals as possible to represent the continuous pixel values ​​in each band while ensuring that the compatibility of the information system is not destroyed. After extracting features from the original remote sensing image, the position information of all pixels and their pixel values ​​in each band are obtained. In the process of preprocessing the extracted remote sensing image information, all pixels contained in each band are sorted and deduplicated according to the brightness value within the band to obtain a set of candidate breakpoints.

[0083] S12. The candidate breakpoint set is fed into a GA for iterative evolution, saving the globally optimal individual from each crossover and mutation. The candidate breakpoint set is then subjected to binary genetic encoding to construct the discretization scheme to be optimized. Searching for the optimal discretization scheme is a constrained optimization problem. GAs, with their inherent implicit parallelism and powerful global search capabilities, can achieve satisfactory results for discretization problems across many datasets. Saving the globally optimal individual from each mutation in the first evolutionary path provides high-quality individuals for the second evolutionary path of the discretization scheme to be optimized, ultimately yielding the optimal discretization scheme during crossover and mutation.

[0084] In step S12, performing binary genetic encoding on the candidate breakpoint set specifically includes:

[0085] S21. Match each candidate breakpoint in the candidate breakpoint set to each bit in the binary code.

[0086] S22: Select the candidate breakpoints whose corresponding binary code values ​​are 1, and discard the candidate breakpoints whose corresponding binary code values ​​are 0.

[0087] S23. Construct a discretization scheme, where the discretization scheme is a set of selected candidate breakpoints.

[0088] As a preferred embodiment, binary encoding is used to encode the candidate breakpoint set. Each bit in the binary code corresponds to a candidate breakpoint, with the values ​​"1" and "0" representing the breakpoint's selection and rejection, respectively. The set of selected candidate breakpoints forms a discretization scheme. According to this discretization scheme, all attribute values ​​are assigned to the discrete intervals formed by their breakpoints.

[0089] S13. State mapping is performed on the feature subset of the remote sensing data. In the crossover phase, the feature subset and a two-dimensional matrix consisting of cross segments of fixed length on the features contained therein are mapped into a state. In the mutation phase, only the feature subset is mapped into a state, and the mutation point of the current state is obtained in the form of Gaussian sampling based on the absolute dissimilarity between the local optimal individual and the global optimal individual.

[0090] As a preferred embodiment, by designing state sets for the crossover and mutation phases, the problem that the Q-tables established from feature subsets cannot accurately represent the different crossover segments in each feature due to the large value range of each feature in remote sensing images, making it difficult to converge the update of Q-values, and the large number of initial breakpoints and the lack of effective heuristic information make it difficult to estimate the number and location of mutation points, is solved, thereby improving the search efficiency of the evolutionary model. In the mutation phase, only the feature subset is mapped into a mutation state, and the mutation point of the current state is obtained based on the absolute dissimilarity between the local optimal individual and the global optimal individual in the form of Gaussian sampling, so as to solve the problem that only each subset of the candidate breakpoint set is regarded as a state, resulting in exponential computational complexity and easily converting the mutation operator into a crossover operator, thereby losing the local search capability, is solved.

[0091] In step S13, the processing of the crossover phase specifically includes:

[0092] S31. Assuming that the remote sensing image contains m features, there are 2 m -1 feature subset, each subset of the candidate breakpoint set is a cross segment, and each cross segment corresponds to a state, then there is states, where n kis the number of candidate breakpoints on the kth band. Each subset of the candidate breakpoint set is regarded as a cross segment and corresponds to a state.

[0093] S32, set the candidate breakpoints to be crossed on each feature to a fixed length L c The fixed length expression is:

[0094] L c =min 1≤k≤m {n k} / 3

[0095] The number of states generated is:

[0096]

[0097] In order to reduce computational complexity and speed up convergence, we set the candidate breakpoints to be crossed on each feature to a fixed length L c , so the number of generated states is reduced to

[0098] S33, the feature subset and the features it contains are of length L c The two-dimensional matrix composed of the cross fragments is mapped into a state, and all states constitute the state set in the cross stage.

[0099] In step S13, the variation point of the current state is obtained, which specifically includes:

[0100] S41. The mutation state is sampled in the form of Gaussian sampling, and the mutation point of the current state is obtained according to the absolute dissimilarity between the local optimal individual and the global optimal individual. The absolute dissimilarity of chromosome x in state s is:

[0101]

[0102] Among them, n i is the number of candidate breakpoints on the i-th band in state s, and b(i,j) is the value of the j-th binary bit on the i-th band. In this way, we can calculate the absolute dissimilarity between the historical optimal individual of each chromosome and the global optimal individual in the population in the current state.

[0103] S42. Calculate the absolute difference between the historical best individual of each chromosome and the global best individual in the population in the current state:

[0104]

[0105] sum AD (l,g,s)=AD(l,s)+AD(g,s)

[0106] difAD (l,g,s)=AD(l,s)-AD(g,s)

[0107] Among them, AD T () is the theoretical absolute dissimilarity function, AD() is the absolute dissimilarity function, l is the historical optimal individual of a given chromosome, g is the global optimal individual in the population, dif AD () is the absolute dissimilarity function difference function, sum AD () is the absolute dissimilarity function summation function, AD(l,s) is the absolute dissimilarity of the historical optimal individual of a given chromosome in state s, AD(g,s) is the absolute dissimilarity of the global optimal individual in the population in state s, and rand() is used to generate random numbers uniformly distributed between 0 and 1.

[0108] S43. In state s, the absolute dissimilarity of chromosome x after mutation follows a Gaussian distribution with mean μ = (AD(l,s) + AD(g,s)) / 2 and standard deviation σ = |AD(l,s) - AD(g,s)|. Then the number of mutation points MN(x,s) of chromosome x is:

[0109]

[0110]

[0111] dif AD (x, l, s) = AD (x, s) - AD (l, s)

[0112] Among them, MN() is the number of mutation points function, round() is the rounding function, and AD(x,s) is the absolute dissimilarity of chromosome x in state s.

[0113] As a preferred embodiment, if AD T If (x,s) is out of bounds, the number of mutation points is determined based on the historical optimal individual of chromosome x, and the locations of the mutation points are then determined. When Temp < 0, this means that the number of loci with a value of "1" on chromosome x is less than the theoretical number of loci that would be "1" on chromosome x after mutation. To favor the theoretical value, MN(x,s) genes are randomly selected from the loci with a value of "0" on chromosome x for mutation. Similarly, when Temp > 0, MN(x,s) genes are randomly selected from the loci with a value of "1" on chromosome x for mutation. If MN(x,s) == 0, mutation points are randomly selected on the features contained in state s. This approach fully utilizes the historical information of the chromosome and the shared information of the population. The mutation operator is directional, thereby improving the search efficiency of the population.

[0114] S14. The fitness value of the individual state is obtained by constructing a fitness function with control variables to evaluate the quality of the evolution of the discretized solution to be optimized, and the conversion between the crossover state and the mutation state is realized by constructing an adaptive reward function. The fitness function can more accurately evaluate the fitness of the individual through the control variables, thereby guiding the genetic algorithm to search the solution space more effectively and quickly approach the optimal solution. The adaptive reward function can dynamically adjust the reward value according to the current search state, which helps the algorithm maintain a balance between exploration and utilization during the evolution process, thereby improving the quality and diversity of evolution, and reducing the risk of the algorithm falling into the local optimum, because the change of the reward function can guide the search process to jump out of the current local optimal area and continue to evolve towards the global optimal solution.

[0115] In step S14, a fitness function with control variables is constructed, specifically including:

[0116] Rough sets understand classification as indistinguishable relationships in feature space, and indistinguishable relationships constitute the division of this space. The principle of selecting the optimal breakpoint set is to generate the least number of breakpoints without destroying the compatibility of the system, thereby constructing the fitness function.

[0117] S51. The indistinguishability relationship constitutes the division of feature subsets. The fitness function is determined by the number of breakpoints of the discretization scheme and the change of the indistinguishability relationship:

[0118]

[0119] Among them, D is the discretization scheme, C is the conditional attribute set, N C is the number of breakpoints in the candidate breakpoint set, N D is the number of breakpoints obtained after chromosome decoding, R D is the change of the upper and lower approximation sets obtained after discretizing the data using D, R D The value of is "0" or "1", indicating that the upper and lower approximation sets of D have changed or have not changed after discretization.

[0120] Since the features of remote sensing images have complex correlations, in the initial population, R D The number of potential feasible solutions with a value of “0” far exceeds that of R D A feasible solution with a value of "1" means that during the population's evolution, there will be a large number of individuals with a fitness value of "0" that cannot be selected. As a result, the majority of potential feasible solutions will be eliminated, thus destroying the population's diversity. To ensure the diversity of individuals in the early stages of the population and accelerate the convergence of individuals to the optimal solution in the later stages of evolution, the fitness function is expanded by adding control variables to dynamically adjust the population's diversity.

[0121] S52. Add control variables to expand the fitness function:

[0122]

[0123] Among them, x is the chromosome, is the number of breakpoints obtained after x is decoded, is the change of the upper and lower approximation sets of x with respect to D after discretization, and p is R D The proportion of potential feasible solutions of “0” in the population,

[0124]

[0125]

[0126] Among them, z is the control item index, k is the number of categories, d i For category i, C l (d i ) is the d before discretization i The lower approximation set, C u (d i ) is the d before discretization i The upper approximation set of C l (d i )' is the discretized d i The lower approximation set, C u (d i )' is the discretized d i The upper approximate set of , setcmp() is the set comparison function.

[0127] In the early stages of evolution, there are no or only a few feasible solutions in the population, that is, p is large; in this way, σ(p) is relatively large, and potential feasible solutions can be selected with a greater probability, guiding the search to gradually enter the feasible solution area. As the evolutionary process proceeds, more and more feasible solutions appear in the population, that is, p becomes smaller and smaller; accordingly, σ(p) also becomes smaller and smaller, which accelerates the convergence of feasible solutions to the optimal solution.

[0128] In step S14, the changing pattern of the optimal discretized cases is simulated to construct an adaptive reward function in the reinforcement learning model, specifically including:

[0129] The state set designed in the crossover phase meticulously characterizes the different crossover segments in each feature. To achieve the transition between the crossover state and the mutation state, the mutation state is considered as a special representation of the crossover state containing the same feature subset, where all loci in the crossover segment take the value of "0", thereby constructing an adaptive reward function.

[0130] The total number of states in S61, crossover phase and mutation phase is:

[0131]

[0132] S62, current status s t By performing action a t Jump to the next state s t+1 , whose expression is:

[0133] (s t ,a t )→s t+1

[0134] S63. Construct an adaptive reward function by realizing the transition between the crossover state and the mutation state. That is, compare the fitness of the next state with the fitness of the historical best individual to set the reward function after the current state executes the action:

[0135]

[0136] Among them, r() is the reward function, r(s t ,a t ) is state s t Execute action a t The reward value obtained, t is the number of iterations of genetic operation, s t is the state under the tth genetic operation, s t+1 Execute action a for the state under the tth genetic operation t The next state to jump to, l is the historical optimal individual, and fit() is the fitness function.

[0137] As a preferred embodiment, this adaptive reward function can reflect the changing trend of the reward value as the population evolves. That is, as the population continues to evolve, updating the historical optimal individual will become increasingly difficult, and accordingly, the reward value should increase. As the objective function value gradually approaches the optimal solution, the increase in the reward value becomes smaller and smaller to accelerate the convergence of the global search. log2(t) is an increasing function whose slope decreases with the number of iterations, which can well describe the above-mentioned population evolution trend.

[0138] S15. A pair of deep Q-learning algorithms with the same structure are introduced to perform real-time calculations on the Q value during the transition between the crossover state and the mutation state. At the same time, the Q value optimization samples of the discretization scheme to be optimized are sampled from the experience replay set to iteratively update the network parameters and determine the crossover segments and mutation points of the discretization scheme to be optimized. Q-learning has broad application prospects in solving complex control and decision-making problems. It can be used to determine the crossover segments and mutation points of the discretization scheme to be optimized. However, when the number of Q-learning states is large, it becomes difficult to use a Q-Table to store the Q value of each "state-action" set. In order to select the optimal crossover segment and mutation point to perform genetic iteration operations with the optimal global individual of the first evolutionary path, a pair of deep Q-learning algorithms with the same structure are introduced to perform real-time calculations on the Q value during the transition between the crossover state and the mutation state.

[0139] In step S15, a pair of deep Q-learning algorithms with the same structure are introduced to update the Q value during the transition between the crossover state and the mutation state, specifically including:

[0140] S71. In the current state, the ε-greedy strategy is used to select the next action based on the Q value corresponding to each action, and the Q value is updated at each step in the learning process. Its expression is:

[0141]

[0142] Among them, a t+1 is the next action, A is the action set, γ is the discount factor, 0≤γ≤1.

[0143] S72, divide the feature subset into 2 m -1 state, m is the number of bands, each state contains several band combinations, that is, the action set A = {G, H, I} covering all states is:

[0144]

[0145] Among them, G is the subset acquisition function of the state, H is the subset acquisition function of the state complement, I is the set acquisition function that intersects with the state and is not a subset of the state, G(s t ) is a random jump to s t One of all subsets of t ) is a random jump to s t One of all subsets of the complement, I(s t ) is a random jump to s t The intersection of is not empty and is not s tOne of all sets of subsets, the w appearing in the three formulas are the required sets that satisfy the above three constraints. The setting of this action set can cover the action set of the entire band to realize the transition between the crossover state and the mutation state.

[0146] S73. Evaluate the search for the optimal solution by the Q network by setting a reward function for individual fitness changes, wherein the Q network is deep Q-learning:

[0147]

[0148] Among them, fit(s t+1 ) is the individual fitness of the next state, and fit(l) is the historical best fitness of the discretization scheme to be optimized. Based on the designed reward value, the values ​​in the two Q-tables of the discretization scheme to be optimized can be updated in the crossover operation and mutation operation, thereby providing reliable decision support for the determination of crossover segments and mutation points.

[0149] S74, introduce two Q networks with the same structure to obtain the Q value, that is, the current Q network Q c and target Q network Q T , then s t The target Q value and Q c The loss functions are:

[0150]

[0151] Among them, θ T Q T The network parameters, θ C Q c Network parameters, is_end t The value of "1" means s t+1 Is the terminal state, is_end t The value of "0" means s t+1 is a non-terminal state, h is the number of samples sampled from the experience replay set E. In two Q networks with the same structure, Q c Used to select actions, Q T Used to calculate the target Q value. The Q network consists of 3 hidden layers with 20 nodes, m input nodes and 1 output node. Each input node receives a feature breakpoint vector, and the output node generates the Q value vector of the input state under all actions. t Enter Q c Then the Q values ​​corresponding to all actions are obtained. According to the current Q value, the ε-greedy strategy is adopted to select a t , s t Execute a t Jump to s t+1 , and obtain rt . Quintuple (s t ,a t ,r t ,s t+1 ,is_end t ) is stored in E. Randomly select h samples from E and pass them through Q T Calculate the corresponding target Q value.

[0152] S75, by setting a fixed time interval w, Q c θ C Make a copy to update Q T θ T , that is, θ C It is updated by gradient descent, executing θ every w time T =θ C That is, θ T The network parameters are delayed updates and do not need to be updated iteratively, but are updated from Q at regular intervals. c Copy it, so that the correlation between the target Q value and the current Q value can be weakened.

[0153] S76. Perform real-time calculation of the Q value of two Q networks with the same structure during the transition between the crossover state and the mutation state. At the same time, sample the Q value optimization samples of the discretization scheme to be optimized from the experience replay set to iteratively update the network parameters and determine the crossover segments and mutation points of the discretization scheme to be optimized.

[0154] S16. Perform genetic operations on the crossover segments and mutation points determined in step S15 and the global optimal individual in step S12 to optimize the discretization scheme to be optimized and obtain the optimal discretization scheme.

[0155] As a preferred embodiment, the present invention proposes an optimization method based on deep Q-learning remote sensing data discretization algorithm flow including:

[0156] The exploration rate ε is gradually reduced from 1 to 0.05 using a linear decay method. This helps the model converge to a high-quality solution stably in the later stages of training and prevents the model from completely stopping exploration. To reduce the correlation between the target Q-value and the current Q-value and ensure a reasonable convergence speed, the update frequency w is set to 5. The sampling size h is selected to be 8 to balance memory requirements and model performance.

[0157] First, candidate breakpoints in remote sensing data are binary genetically encoded, generating a set of states and actions for the discretization scheme to be optimized. Next, a greedy algorithm is used to select an action to jump to the next state within the current state of the discretization scheme to be optimized. If the current state is a crossover state, the next state is a mutation state. Based on the absolute dissimilarity between the local optimal individual and the global optimal individual, the mutation point of the discretization scheme to be optimized is determined on the feature subset corresponding to the mutation state. If the current state is a mutation state, the next state is a crossover state, and a crossover operation is performed with the global optimal individual on the intersection segment of the feature subset corresponding to the crossover state.

[0158] The reward value after executing an action in the current state is obtained by comparing the fitness of the next state with the fitness of the historically optimal individual. Two Q networks with identical structures are introduced to update the Q value of each "state-action" pair, determining the crossover segments and mutation points of the discretization scheme to be optimized. Conventional genetic operations are performed on the candidate breakpoint set input into the GA, and the globally optimal individual is saved in each crossover and mutation. After the current population undergoes a crossover and mutation cycle, a new population is formed by combining the top 90% of individuals in terms of fitness with 10% of randomly generated individuals to enter the next genetic operation. The identified crossover segments and mutation points are genetically manipulated with the globally optimal individual to optimize the discretization scheme to obtain the optimal discretization scheme. The program finally outputs the optimal discretization scheme for both global and local variables.

[0159] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An optimization method for discretization of remote sensing data based on deep Q-learning, characterized in that: The method comprises: S11, extracting features from the remote sensing image to obtain a feature subset, preprocessing the feature subset to obtain a candidate breakpoint set, and performing evolutionary processing on the candidate breakpoint sets respectively; S12, inputting the candidate breakpoint set into GA for iterative evolution, saving the global optimal individual in each crossover and each mutation, and then performing binary genetic encoding on the candidate breakpoint set to construct a discretization scheme to be optimized; S13, performing state mapping on a feature subset of the remote sensing data. In the crossover phase, a two-dimensional matrix consisting of the feature subset and crossover segments of fixed length on the features contained therein is mapped into a state. In the mutation phase, only the feature subset is mapped into a state, and a mutation point of the current state is obtained based on the absolute dissimilarity between the local optimal individual and the global optimal individual in the form of Gaussian sampling. S14. Obtain the fitness value of the individual state by constructing a fitness function with control variables to evaluate the quality of the evolution of the discretization scheme to be optimized, and realize the conversion between the crossover state and the mutation state by constructing an adaptive reward function; S15. Introduce a pair of deep Q-learning algorithms with the same structure to calculate the Q value in real time during the transition between the crossover state and the mutation state. At the same time, sample the Q value optimization samples of the discretization scheme to be optimized from the experience replay set to iteratively update the network parameters and determine the crossover segments and mutation points of the discretization scheme to be optimized. S16. Perform genetic operations on the crossover segments and mutation points determined in step S15 and the global optimal individual in step S12 to optimize the discretization scheme to be optimized and obtain the optimal discretization scheme.

2. The optimization method for discretization of remote sensing data based on deep Q-learning according to claim 1, characterized in that: In step S11, the method of preprocessing the feature subset to obtain a candidate breakpoint set includes: the feature subset includes the position information of all pixels and the pixel values ​​of all pixels in each band, all pixels contained in each band are sorted and deduplicated according to the brightness value within the band to obtain a candidate breakpoint set.

3. The optimization method for discretization of remote sensing data based on deep Q-learning according to claim 1, characterized in that: In step S12, performing binary genetic coding on the candidate breakpoint set specifically includes: S21, corresponding each candidate breakpoint in the candidate breakpoint set to each bit in the binary code; S22, selecting a candidate breakpoint whose corresponding binary code value is 1, and discarding a candidate breakpoint whose corresponding binary code value is 0; S23. Construct a discretization scheme, where the discretization scheme is a set of selected candidate breakpoints.

4. The optimization method for discretization of remote sensing data based on deep Q-learning according to claim 3, characterized in that: In step S13, the processing of the crossover phase specifically includes: S31. Assuming that the remote sensing image contains m features, then feature subsets, each subset of the candidate breakpoint set is a cross segment, and each cross segment corresponds to a state, then states, of which is the number of candidate breakpoints on the kth band; S32, set the candidate breakpoints to be crossed on each feature to a fixed length The fixed length expression is: The number of states generated is: S33, the length of the feature subset and the features it contains is The two-dimensional matrix composed of the cross fragments is mapped into a state, and all states constitute the state set in the cross stage.

5. The optimization method for discretization of remote sensing data based on deep Q-learning according to claim 4, characterized in that: In step S13, the variation point of the current state is obtained, which specifically includes: S41. The mutation state is sampled in the form of Gaussian sampling, and the mutation point of the current state is obtained according to the absolute dissimilarity between the local optimal individual and the global optimal individual. The absolute dissimilarity of chromosome x in state s is: in, Status Middle The number of candidate breakpoints on the band, For the The first band The value of the binary bit; S42. Calculate the absolute difference between the historical best individual of each chromosome and the global best individual in the population in the current state: in, () is the theoretical absolute dissimilarity function, () is the absolute dissimilarity function, is the historical best individual of a given chromosome, is the global optimal individual in the population, () is the absolute dissimilarity function difference function, () is the absolute dissimilarity function sum function, is the absolute dissimilarity of the historical best individual of a given chromosome in state s, is the absolute dissimilarity of the global optimal individual in the population under state s, Used to generate random numbers uniformly distributed between 0 and 1; S43. In state s, the absolute dissimilarity of chromosome x after mutation operation obeys the mean , standard deviation Gaussian distribution, then the number of mutation points on chromosome x for: in, () is the function of the number of mutation points, d() is the rounding function, is the absolute dissimilarity of chromosome x in state s.

6. The optimization method for discretization of remote sensing data based on deep Q-learning according to claim 5, characterized in that: In step S14, a fitness function with control variables is constructed, specifically including: S51. The indistinguishability relationship constitutes the division of feature subsets. The fitness function is determined by the number of breakpoints of the discretization scheme and the change of the indistinguishability relationship: in, Discretization scheme, is the conditional attribute set, is the number of breakpoints in the candidate breakpoint set, is the number of breakpoints obtained after chromosome decoding, To adopt The changes in the upper and lower approximation sets obtained after discretizing the data, The value of is "0" or "1", which means The upper and lower approximation sets of have changed or not changed after discretization; S52. Add control variables to expand the fitness function: in, For chromosomes, for The number of breakpoints obtained after decoding, for about The changes of the upper and lower approximation sets after discretization, for The proportion of potential feasible solutions of "0" in the population, in, is the control item index, is the number of categories, For the category, Before discretization The lower approximation set of Before discretization The upper approximation set of After discretization The lower approximation set of After discretization The upper approximation set of A set comparison function.

7. The optimization method for discretization of remote sensing data based on deep Q-learning according to claim 6, characterized in that: In step S14, the adaptive reward function is constructed in the reinforcement learning model by simulating the change pattern of the optimal discretized case. Specifically, the following steps are performed: The total number of states in S61, crossover phase and mutation phase is: S62, Current Status By performing actions Jump to the next state , whose expression is: S63. Construct an adaptive reward function by realizing the transition between the crossover state and the mutation state. That is, compare the fitness of the next state with the fitness of the historical best individual to set the reward function after the current state executes the action: in, () is the reward function, Status Execute an action The reward value obtained, is the number of iterations of genetic operation, For the The state under the genetic operation, For the State execution action under genetic operation The next state to jump to, is the best individual in history, is the fitness function.

8. The optimization method for discretization of remote sensing data based on deep Q-learning according to claim 7, characterized in that: In step S15, a pair of deep Q-learning algorithms with the same structure are introduced to update the Q value during the transition between the crossover state and the mutation state, specifically including: S71. In the current state, according to the Q value corresponding to each action, The strategy selects the next action and updates the Q value at each step in the learning process. Its expression is: in, is the reward function, is the reward value, For the next action, For action collection, is the discount factor, ; S72, divide the feature subset into states, is the number of bands, each state contains several band combinations, that is, the action set covering all states for: in, Get functions for a subset of states, Get functions for subsets of the complement of the state, Get functions for sets that intersect the state and are not subsets of the state, To randomly jump to One of all subsets of To randomly jump to One of all subsets of the complement, To randomly jump to The intersection of is not empty and is not One of all sets of subsets that appear in the three formulas are the desired sets that satisfy the above three constraints respectively; S73. Evaluate the search for the optimal solution by the Q network by setting a reward function for individual fitness changes, wherein the Q network is deep Q-learning: in, The individual fitness of the next state, is the historical best fitness of the discretization scheme to be optimized; S74, introduce two Q networks with the same structure to obtain the Q value, that is, the current Q network and target Q network ,but The target Q value and The loss functions are: in, for The network parameters, for The network parameters, The value of "1" means is the terminal state, The value of "0" means is a non-terminal state, h is the number of samples sampled from the experience replay set E; S75, by setting a fixed time interval w, of Copy to update of ,Right now Updated by gradient descent, executed every w time = ; S76. Perform real-time calculation of the Q value of two Q networks with the same structure during the transition between the crossover state and the mutation state. At the same time, sample the Q value optimization samples of the discretization scheme to be optimized from the experience replay set to iteratively update the network parameters and determine the crossover segments and mutation points of the discretization scheme to be optimized.

Images