Hyper-parameter tuning method and device for spatial hierarchical machine learning model

Abstract

The invention provides a hyper-parameter tuning method and device for a spatial hierarchical machine learning model, and belongs to the technical field of machine learning. The method comprises the following steps: acquiring a to-be-optimized hyper-parameter of a local model of each sub-region; constructing a multi-task constraint single-target optimization problem; wherein the multi-task constraint single-target optimization problem takes the hyper-parameter optimization of each local model as an optimization task, and takes the hyper-parameter optimization of the multi-local model as a multi-task constraint single-target optimization problem; solving the multi-task constraint single-target optimization problem by adopting a single-population multi-task optimization algorithm embedded with geographic association so as to obtain a hyper-parameter optimization result of each local model; according to the single-population multi-task optimization algorithm embedded with geographical association, a geographical adjacent differential evolution Geo-DE operator is adopted to carry out hybridization variation, and the Geo-DE operator generates filial generations based on the difference between optimal hyper-parameters of local models of adjacent sub-regions. According to the method, the hyper-parameter tuning precision and efficiency can be improved.

Description

Technical Field

[0001] This application relates to the technical field of machine learning, and particularly to a hyperparameter tuning method and device for a spatially hierarchical machine learning model. Background Art

[0002] With the wide application of spatial data in various fields, the importance of data-driven machine learning methods in spatial analysis has become increasingly prominent. However, the performance of the model highly depends on hyperparameter tuning (HPT), and this process is particularly complex when dealing with data with spatial stratification heterogeneity (SSH).

[0003] Spatial stratification heterogeneity (SSH feature) results in significantly different statistical characteristics of data in different sub-regions within the research area. Therefore, it is necessary to construct local models separately for each sub-region. Currently, the hyperparameter tuning methods of machine learning models are mainly divided into four categories: empirical setting method, grid search method, random search method, and metaheuristic optimization algorithm, etc. However, the above hyperparameter tuning methods usually set a unified hyperparameter configuration for all local models, or optimize hyperparameters independently in each sub-region. The former ignores the differences in hyperparameters between sub-regions, and the latter causes the algorithm to easily fall into local optima. Therefore, it is necessary to provide an improved technical solution for the deficiencies of the above existing technologies. Summary of the Invention

[0004] The purpose of this application is to provide a hyperparameter tuning method and device for a spatially hierarchical machine learning model that can improve the accuracy and efficiency of hyperparameter tuning, so as to solve or alleviate the problems existing in the above existing technologies.

[0005] To achieve the above purpose, this application provides the following technical solutions: This application provides a hyperparameter tuning method for a spatially hierarchical machine learning model, including: Obtain the hyperparameters to be optimized of the local model for each sub-region; Construct a multi-task constrained single-objective optimization problem; the multi-task constrained single-objective optimization problem takes the hyperparameter optimization of each local model as an optimization task, regards the hyperparameter optimization of multiple local models as a multi-task constrained single-objective optimization problem, and is defined by an objective function and a boundary constraint; Use a single-population multi-task optimization algorithm embedded with geographical association to solve the multi-task constrained single-objective optimization problem to obtain the hyperparameter optimization results of each local model; The single-population multi-task optimization algorithm embedded with geographical association is based on an evolutionary algorithm and uses the geographical proximity differential evolution (Geo-DE) operator for hybridization and mutation. The Geo-DE operator generates offspring based on the differences between the optimal hyperparameters of the local models in neighboring sub-regions to utilize the spatial association between sub-regions to guide the search direction.

[0006] In a possible implementation, the single-population multi-task optimization algorithm embedded with geographical association is used to solve the multi-task constrained single-objective optimization problem to obtain the hyperparameter optimization results of each local model, including: Based on the training sets of each pre-acquired sub-region, use the initial population to train each local model, and evaluate the accuracy of each individual in the initial population on each local model based on the validation sets of each pre-acquired sub-region to obtain the evaluation results; Use the initial population as the parent population, and use the Geo-DE operator to generate offspring, or randomly select an evolutionary operator from the Geo-DE operator and the classical differential evolution operator to generate offspring; Conduct offspring evaluation and update the population; Use the updated population as the new parent population, re-train each local model based on the new parent population and evaluate the accuracy until the optimization goal of maximizing the accuracy of each local model is achieved, and output the hyperparameter optimization results of each local model.

[0007] In a possible implementation, when choosing the Geo-DE operator to generate offspring, the following steps are executed: From all individuals in the population, select the individual with the best performance on the local model with the best performance of the current individual, denoted as the parent optimal hyperparameter; Randomly select two models from all neighboring models of the local model with the best performance of the current individual, denoted as the first neighboring model and the second neighboring model; Search for the optimal hyperparameters corresponding to the first neighboring model and the second neighboring model respectively in the parent population; use the parent optimal hyperparameter to indicate the optimization direction, and correct the optimization direction and step size based on the difference between the two optimal hyperparameters to generate offspring.

[0008] In a possible implementation, after generating offspring, it further includes: using a polynomial mutation operator to mutate each component in the offspring to achieve local micro-search in the solution space.

[0009] In a possible implementation, the multi-task constrained single-objective optimization problem further includes constraint conditions, and the constraint conditions are the upper and lower limits of the values of each hyperparameter; After completing hybridization and mutation, it further includes: Based on the above constraints, repair the components in the offspring that violate the constraints to make them meet the constraints again.

[0010] In a possible implementation, offspring evaluation is performed and the population is updated, including: randomly selecting one strategy from a global update strategy and a neighborhood update strategy to update the population, where the neighborhood update strategy uses a geographical neighborhood selection operator to update the population, including: traversing the offspring population, and adding the local model with the best performance of the current offspring individual and all its neighbor models to a set ; for each local model in the set find the parental individual with the best performance on this local model from the parental population, and update the parental individual with the offspring individual.

[0011] In a possible implementation, the coefficient of determination of each sub-region on the validation set is used to characterize the accuracy of the local model.

[0012] In a second aspect, the present embodiment provides a hyperparameter tuning device for a spatially hierarchical machine learning model, including: A parameter acquisition unit configured to acquire the hyperparameters to be optimized of the local models of each sub-region; A model construction unit configured to construct a multi-task constrained single-objective optimization problem; the multi-task constrained single-objective optimization problem takes the hyperparameter optimization of each local model as an optimization task, and regards the hyperparameter optimization of multiple local models as a multi-task constrained single-objective optimization problem, which is defined by an objective function and a boundary constraint; A solving unit configured to solve the multi-task constrained single-objective optimization problem by using a single-population multi-task optimization algorithm embedded with geographical association to obtain the hyperparameter optimization results of each local model; The single-population multi-task optimization algorithm embedded with geographical association is based on an evolutionary algorithm, and uses a geographical neighborhood differential evolution Geo-DE operator for hybridization and mutation. The Geo-DE operator generates offspring based on the differences between the optimal hyperparameters of the local models of neighboring sub-regions to guide the search direction by utilizing the spatial association between sub-regions.

[0013] In a third aspect, the present embodiment provides an electronic device, including a memory, a processor, and a computer program stored on the memory. The processor executes the computer program to implement the steps of the hyperparameter tuning method for a spatially hierarchical machine learning model according to any one of the above embodiments.

[0014] In a fourth aspect, a computer-readable storage medium stores a computer program / instructions, characterized in that when the computer program / instructions are executed by a processor, the steps of the hyperparameter tuning method for a spatially hierarchical machine learning model according to any one of the above embodiments are implemented.

[0015] The technical solution of the embodiment of the present application has the following beneficial effects: In the technical solution of this embodiment, with maximizing the accuracy of each local model as the optimization goal, a multi-task constrained single-objective optimization problem for hyperparameter tuning of a machine learning model facing spatial hierarchical heterogeneity is constructed; a single-population multi-task optimization algorithm embedded with geographical association is used to solve the multi-task constrained single-objective optimization problem to obtain the hyperparameter optimization results of each local model; among them, the single-population multi-task optimization algorithm embedded with geographical association performs hybrid mutation by combining the geographical proximity differential evolution Geo-DE operator and the classical differential evolution operator. The Geo-DE operator generates offspring based on the differences between the optimal hyperparameters of the local models in adjacent sub-regions, and uses the spatial association between sub-regions to guide the search direction. In this way, the hyperparameter tuning of multiple local models is regarded as a multi-task optimization problem, making full use of the spatial dependence between adjacent sub-regions. The Geo-DE operator is used to guide the hyperparameter optimization process of another local model with the solution generated in the hyperparameter optimization process of one local model, realizing the collaborative optimization between local models in different sub-regions during the search process, and the optimization is limited to the adjacent sub-region range, improving the search accuracy and efficiency of the algorithm. Description of the Drawings

[0016] Figure 1 It is a schematic diagram of a method for hyperparameter tuning of a machine learning model facing spatial stratification according to some embodiments of the present application.

[0017] Figure 2 It is a schematic diagram of the optimal hyperparameters of local models in different sub-regions according to some embodiments of the present application.

[0018] Figure 3 It is a schematic diagram of the execution process of a single-population multi-task optimization algorithm embedded with geographical association Figure 1 .

[0019] Figure 4 It is a schematic diagram of the execution process of a single-population multi-task optimization algorithm embedded with geographical association Figure 2 .

[0020] Figure 5 It is a schematic diagram of the execution process of a single-population multi-task optimization algorithm embedded with geographical association Figure 3 .

[0021] Figure 6 It is a schematic diagram of the spatial position relationship of the study area and its sub-regions (sub-regions 1 to 4).

[0022] Figure 7 It is the result of population initialization; among them, (a) is the parental population, and (b) is the scores of the parental individuals on each local model.

[0023] Figure 8 For the offspring population , where (a) is the matrix form of the offspring population , and (b) is the score of each offspring on each local model.

[0024] Figure 9 For the comparison of the scores of the offspring and the parent individuals, where (a) is the offspring population and (b) is the score of the offspring population.

[0025] Figure 10 For the schematic diagram of population update, where (a) is the offspring population, (b) is the score of the offspring population, (c) is the updated parent population, and (d) is the score of the updated parent population.

[0026] Figure 11 For the schematic diagram of the structure of the electronic device provided by some embodiments. Detailed implementation manners

[0027] The following explains the related concepts involved in this embodiment.

[0028] Spatial data refers to data containing geographical location information, accounting for approximately 80% of the global data resources and being of great value for decision support and scientific discovery.

[0029] With the massive growth of spatial data in various fields, data-driven machine learning models (Machine Learning, ML) are widely used in spatial data analysis, such as urban traffic flow prediction, air pollution assessment, and microbial spatial distribution mapping.

[0030] However, the performance of machine learning models depends to a large extent on a series of important parameters, which can be divided into two categories: model parameters (Parameters) and hyperparameters (Hyperparameters). Model parameters are parameters optimized through data during the model learning process, such as the weights of neural networks. Hyperparameters are preset before model training, have an important impact on the model learning process, but will not be updated during training, such as the depth or learning rate of decision trees. The configuration of hyperparameters directly affects whether the model can accurately fit the data features. Therefore, hyperparameter tuning (Hyperparameters Tuning, HPT) is one of the key steps to improve the performance of machine learning models.

[0031] In the prior art, the hyperparameter tuning methods of machine learning models are mainly divided into four categories: empirical setting method, grid search method, random search method, and metaheuristic optimization algorithm.

[0032] The empirical setting method directly sets hyperparameters for the model based on historical research experience. This method is simple and easy to use. However, due to the sensitivity of machine learning models to datasets and hyperparameters, hyperparameters that perform well on other datasets are often difficult to transfer to the target dataset.

[0033] The Grid Search (GS) method discretizes the search space of hyperparameters and then traverses all possible combinations one by one to determine the optimal hyperparameters. Although this method is simple, its efficiency and accuracy highly depend on the number of grids, and it has limited processing ability for real-valued hyperparameters.

[0034] The Random Search (RS) method randomly samples in the hyperparameter space and iterates multiple times to select the best hyperparameter configuration. Compared with the grid search method, the random search method avoids the high computational cost of comprehensive search. However, its search accuracy is limited by the number of sampling times, and it is difficult to quickly converge to the global optimal solution when the hyperparameter space is large.

[0035] Meta-heuristic optimization algorithms: Such algorithms randomly generate an initial population containing multiple hyperparameter candidate solutions (i.e., individuals) and evaluate each individual in the population. During the iteration process, the algorithm uses excellent individuals to guide the generation of new solutions and update the population, and efficiently searches for the optimal solution in the hyperparameter space in a heuristic way. Compared with the grid search method and the random search method, it has higher search efficiency.

[0036] Although these methods have been widely applied in traditional machine learning models and demonstrated certain optimization capabilities, when dealing with data with the characteristics of spatial hierarchical heterogeneity, their applicability and effectiveness still face significant challenges.

[0037] Spatial hierarchical heterogeneity (SSH) is a typical feature of spatial data, which means that the study area can be divided into several sub-regions with homogeneous statistical distributions, and there are significant differences in the means, variances, or variable relationships of geographical variables between different sub-regions. The SSH feature makes it difficult for traditional machine learning models with globally unified parameters to fully adapt to the differences between regions, limiting the ability to depict complex spatial patterns.

[0038] To address the impact brought by SSH, researchers usually divide the study area into several sub-regions and build local models for each sub-region to better adapt to the internal characteristics of the region. However, this local zoning modeling poses higher requirements for hyperparameter tuning.

[0039] Ideally, the local models of each sub-region should have differentiated hyperparameter configurations to reflect the data characteristics of the sub-region. However, directly optimizing the hyperparameters of all local models separately will lead to a rapid expansion of the search space and extremely high computational costs.

[0040] To simplify the optimization process, a common practice is to set unified hyperparameters for all local models. However, this approach ignores the differences in data characteristics between sub-regions and limits the accuracy of the model.

[0041] Some studies have attempted to use the grid search method to independently optimize hyperparameters in each sub-region. However, as the number of hyperparameters and the scale of local models increase, the search efficiency and feasibility are severely limited. Although meta-heuristic algorithms have high search efficiency and can handle hyperparameter optimization problems in high-dimensional search spaces, most existing studies have used them for hyperparameter tuning (HPT) of a single global model or independently optimized hyperparameters in each sub-region. The former ignores the differences in hyperparameters between sub-regions, and the latter causes the algorithm to easily fall into local optima.

[0042] Based on this, this embodiment provides a hyperparameter tuning method for a spatially hierarchical machine learning model. This hyperparameter tuning method can take into account the differences between regions and computational efficiency to improve the usability and accuracy of machine learning models in the context of spatial hierarchical heterogeneity.

[0043] The embodiments of the present application will be described below with reference to the accompanying drawings.

[0044] Embodiment 1:

[0045] This embodiment provides a hyperparameter tuning method for a spatially hierarchical machine learning model. As Figure 1 shown, this method includes steps S101 to S103: Step S101, obtain the hyperparameters to be optimized for the local models of each sub-region; Step S102, construct a multi-task constrained single-objective optimization problem; wherein, the multi-task constrained single-objective optimization problem takes the hyperparameter optimization of each local model as an optimization task, and regards the hyperparameter optimization of multiple local models as a multi-task constrained single-objective optimization problem, which is defined by an objective function and a boundary constraint; Step S103, use a single-population multi-task optimization algorithm embedded with geographical association to solve the multi-task constrained single-objective optimization problem to obtain the hyperparameter optimization results of each local model; Among them, the single-population multi-task optimization algorithm embedded with geographical association uses the geographical proximity differential evolution (Geo-DE) operator for hybridization and mutation. The Geo-DE operator generates offspring based on the differences between the optimal hyperparameters of local models in neighboring sub-regions to guide the search direction using the spatial association between sub-regions.

[0046] In this embodiment, the hyperparameter tuning method for the spatial hierarchical machine learning model, also known as the automated hyperparameter tuning framework (GeoEvo) for the spatial hierarchical heterogeneous machine learning model, includes two parts: the optimization problem definition and the optimization algorithm design, that is, constructing a multi-task constrained single-objective optimization problem for the hyperparameter tuning of the spatial hierarchical heterogeneous machine learning model, and solving the multi-task constrained single-objective optimization problem (referred to as the optimization problem) through a single-population multi-task optimization algorithm embedded with geographical association.

[0047] Specifically, first, by defining the decision variables, objective function, and constraint conditions of the optimization problem, a multi-task constrained single-objective optimization problem for the hyperparameter tuning of the spatial multi-local model is constructed; then, a single-population multi-task optimization algorithm embedded with spatial association is proposed, and through the design of customized evolutionary operators (called Geo-DE operators), the spatial collaborative optimization of the hyperparameters of multiple local models is realized.

[0048] The following refers to Figure 2 to illustrate the technical principle of this embodiment.

[0049] In addition to hierarchical heterogeneity, spatial data also has significant spatial dependence, that is, things in the geographical space are usually related to each other, and this correlation increases with the proximity of geographical locations. Therefore, although the data distributions in different sub-regions are different, there may still be spatial associations between sub-regions, which makes the optimal hyperparameters between local models of sub-regions may be correlated. Specifically, when optimizing the hyperparameters of these local models, geographically adjacent sub-regions may share similar hyperparameter configurations, and this association may affect each other during the optimization process.

[0050] Figure 2 Shows a schematic diagram of the optimal hyperparameters of local models in different sub-regions. As Figure 2 shown, the left figure is the geographical space, the abscissa is the longitude, and the ordinate is the dimension. Each cross in the left figure represents a sub-region, corresponding to a local machine learning model (local model). The right figure is the hyperparameter space, and each point represents a multi-dimensional hyperparameter combination. The goal of hyperparameter tuning is to find the optimal hyperparameter combination from the hyperparameter space.

[0051] In Figure 2 , since the orange area and the blue area are closer in the geographical space, the optimal hyperparameters of the two corresponding local models may be more similar. In this case, the solution generated during the hyperparameter optimization process of one local model may help to guide the hyperparameter optimization process of another local model. Therefore, regarding the hyperparameter tuning of multiple local models as a multi-task optimization problem and considering the collaborative optimization between local models of different sub-regions during the search process is expected to improve the search efficiency and accuracy of the algorithm.

[0052] The problem faced by this embodiment belongs to the multi-task hyperparameter tuning problem.

[0053] Considering that the multi-task evolutionary algorithm based on multiple populations will increase the memory and computational resource overhead, the proposed GeoEvo framework aims to use a single evolutionary population to simultaneously optimize the hyperparameters of multiple local machine learning models, and implicitly achieve spatial collaborative optimization capabilities by improving the hybridization and mutation processes of traditional evolutionary algorithms. Therefore, the goal of this embodiment is to construct an optimization problem that can cover all the hyperparameter tuning tasks of local machine learning models simultaneously.

[0054] Before defining the optimization problem, the following rules for symbol usage are agreed upon: (1) Bold lowercase letters represent vectors (e.g., ), and by default, vectors are column vectors unless otherwise specified; (2) Bold uppercase letters represent matrices or sets (e.g., ); (3) Lowercase letters represent scalars (e.g., ).

[0055] In this embodiment, the hyperparameter combination that needs to be optimized for the machine learning model can be encoded as a -dimensional decision vector , represents the number of model hyperparameters, and each component in represents a hyperparameter. In the evolutionary algorithm, a decision vector represents an individual, and a candidate hyperparameter set composed of individuals is called a population.

[0056] The hyperparameter optimization of each local model is usually regarded as an optimization task. The HPT of multiple local models can be regarded as a single-objective optimization problem with multi-task constraints, which is defined by an objective function and a boundary constraint. The optimization objective of this problem is to maximize the optimal accuracy of each local model, and the objective function is defined as: (1) In the formula, is the accuracy of the hyperparameter combination on the th local model , , is the total number of sub-regions and local models, represents finding an individual with the best performance on the th local model among all individuals in the population , that is, , represents the th individual.

[0057] To avoid overfitting of the local model on the training set, the coefficient of determination of each sub-region on the validation set is used to characterize the accuracy of the local model. That is, the accuracy of the local model is defined as the coefficient of determination on the validation set of the corresponding sub-region: (2) where is the size of the validation set of the th sub-region. and are the th prediction result and label of the th local machine learning model, respectively. is the average value of the labels of the validation set of the

[0058] Understandably, ranges from to The larger the value, the higher the prediction accuracy of the model on the validation set.

[0059] Preferably, the multi-task constrained single-objective optimization problem further includes constraint conditions, and the constraint conditions are the upper and lower limits of the values of each hyperparameter.

[0060] That is to say, the constraint of the optimization problem is that the decision vector must satisfy the boundary conditions to ensure that the algorithm finally outputs valid optimal hyperparameters. The boundary constraint is defined as: (3) where is the dimensional decision space, , represents the real number space (converted to the integer space when the hyperparameter is an integer), and are the minimum and maximum values of the th hyperparameter, respectively, and define the lower and upper bounds of the decision space.

[0061] Based on the above objective function and boundary constraints, the hyperparameter optimization of the spatially stratified heterogeneity machine learning model can be defined as the following multi-task constrained single-objective optimization problem:

[0062] (4) In step S103, an embedded geographical association single-population multi-task optimization algorithm is used to solve the multi-task constrained single-objective optimization problem.

[0063] It should be noted that the single-population multi-task optimization algorithm embedded with geographical association is based on the evolutionary algorithm and follows the iterative process of population initialization, evaluation, hybridization, mutation, and update of the evolutionary algorithm.

[0064] In the evolutionary algorithm, the evolutionary operator is used to implement hybridization and mutation to generate new individuals and update the population, and it is a fundamental component that determines the performance of the algorithm. Figure 3 、 4 、5 shows a schematic diagram of the execution process of the single-population multi-task optimization algorithm embedded with geographical association. As Figure 3 、 4 shown, to enhance the search ability of the algorithm, in this embodiment, during the search process of the single-population multi-task optimization algorithm embedded with geographical association, a geographical proximity differential evolution operator Geo-DE is set to explore the solution space to generate new individuals, and the search direction of the algorithm is guided by the optimal hyperparameters of the local model in the neighboring sub-region, accelerating the search process and improving the search accuracy at the same time.

[0065] In some embodiments, in step S103, the single-population multi-task optimization algorithm embedded with geographical association is used to solve the multi-task constrained single-objective optimization problem to obtain the hyperparameter optimization results of each local model, including: Step S113: Based on the training sets of each sub-region obtained in advance, use the initial population to train each local model, and evaluate the accuracy of each individual in the initial population on each local model based on the validation sets of each sub-region obtained in advance to obtain the evaluation results; Step S123: Use the initial population as the parent population, and generate offspring using the Geo-DE operator, or randomly select an evolutionary operator from the Geo-DE operator and the classical differential evolution operator to generate offspring; Step S133: Conduct offspring evaluation and update the population; Step S143: Use the updated population as the new parent population, re-train each local model based on the new parent population and evaluate the accuracy until the optimization goal of maximizing the accuracy of each local model is achieved, and output the hyperparameter optimization results of each local model.

[0066] In this embodiment, the purpose of step S113 is to obtain and evaluate the initial population. Among them, the initial population refers to a group of candidate solutions (individuals) generated for the first time before the algorithm starts iterative optimization in the evolutionary algorithm (such as genetic algorithm, differential evolution, evolutionary strategy, etc.).

[0067] In this embodiment, the initial population is obtained through population initialization. For example, the initial population can be generated by random uniform sampling in the defined hyperparameter search space, or heuristic initialization can be performed using prior knowledge, such as sampling partially from historical tuning results. This embodiment does not limit the method of population initialization.

[0068] Exemplarily, a set of candidate solutions (hyperparameters) can be randomly generated .

[0069] Evaluate the initial population, that is, based on the pre-acquired validation sets of each sub-region, use the coefficient of determination (formula (2)) to evaluate the accuracy of each individual in the initial population on each local model, and obtain the evaluation results.

[0070] The purpose of step S123 is to generate offspring for each individual in the population to form a new population . In this embodiment, the Geo-DE operator can be directly used to generate offspring, or an evolutionary operator can be randomly selected from the Geo-DE operator and the classical differential evolution operator to generate offspring. The algorithm for generating offspring will be described in detail below.

[0071] Considering that the hyperparameter optimization processes of different local models can cooperate with each other, especially the optimal hyperparameters of spatially adjacent local machine learning models may be similar, this embodiment proposes the geographical proximity differential evolution operator Geo-DE, which uses the information of the hyperparameter optimization processes of adjacent local machine learning models to guide the search direction and achieve collaborative optimization of spatially adjacent tasks; uses the classical differential evolution operator (DE / current-to-best / 1) to guide the population to search for the optimal solution and implicitly perform global spatial collaborative optimization.

[0072] In practice, the Geo-DE operator can be directly used to generate offspring, or two differential evolution operators, the Geo-DE operator and the classical differential evolution (DE / current-to-best / 1) operator, can be randomly selected to generate offspring. The steps are as follows: First, determine the local model with the best performance for the current individual. For each individual in the population , the local model with the best performance is denoted as . Then, select the individual with the best performance on from all individuals in the population as the parent individual , denoted as the optimal hyperparameters of the parent. Then, randomly select one operator from DE / current-to-best / 1 and Geo-DE to generate offspring.

[0073] For the population each individual in , when the DE / current-to-best / 1 operator is selected, the offspring are generated according to the following steps: First, randomly select two individuals without replacement from all individuals in the population except and as the parents. Then, generate the offspring according to the following formula and : : (5) In the formula, is the scaling factor used to control the mutation intensity. The larger is, the greater the mutation intensity of the parents; , , , are the sequence numbers of the candidate solutions (individuals) of the hyperparameters. Among them, the th individual is the current individual, and the th individual is the optimal individual, that is, the optimal hyperparameters of the parents. and are two different individuals randomly selected from the population and are not equal to .

[0074] Since the local models with the best performance of each individual may be located in different sub-regions, the individual hybridization in DE / current-to-best / 1 essentially establishes a mechanism for different local models to guide each other in the hyperparameter search process. Therefore, DE / current-to-best / 1 can implicitly integrate the information generated in the hyperparameter optimization process of different local models globally and achieve global collaborative optimization in space.

[0075] Specifically, when and differ greatly, it indicates that the optimal hyperparameters of different local models globally differ greatly, and the value of will be larger, thus increasing the mutation intensity of DE / current-to-best / 1 and using the global distribution characteristics of hyperparameters to guide the population to explore the solution space more; when and differ little, it indicates that the optimal hyperparameters of different local models globally tend to be similar. At this time, the mutation intensity of DE / current-to-best / 1 is small, which helps to promote the convergence of the algorithm.

[0076] When the Geo-DE operator is selected, the following steps are executed to generate the offspring: Step S1231: Select, from all individuals in the population, the individual that performs optimally on the local model with the best performance among the current individuals, and denote it as the optimal hyperparameters of the parent generation; Step S1232: Randomly select two models from all neighboring models of the local model with the best performance among the current individuals, and denote them as the first neighboring model and the second neighboring model; Step S1233: Search for the optimal hyperparameters corresponding to the first neighboring model and the second neighboring model respectively in the parent population; Step S1234: Use the optimal hyperparameters of the parent generation to indicate the optimization direction, and correct the optimization direction and step size based on the difference between the two optimal hyperparameters to generate offspring.

[0077] Among them, Step S1231 determines the optimal hyperparameters of the parent generation, which is the same as the execution method of the aforementioned optimal individual and will not be elaborated here.

[0078] In Step S1232, the local model with the best performance among the current individuals is , and two models are randomly selected from all neighbor models of the local model , and denoted as the first neighboring model and the second neighboring model . Then, in Step S1233, using the individuals in the population that perform best on and (i.e., the optimal hyperparameters of and respectively) as the parents, generate offspring according to the following formula and : : (6) In the formula, and represent the current optimal hyperparameters of and respectively.

[0079] It should be noted that although both formulas (5) and (6) are differential evolution operators, they have different physical meanings. Specifically: characterizes the distribution characteristics of the hyperparameters of spatially neighboring local models. When the difference between and is small, it means that the optimal hyperparameters of and are similar. At this time, Geo-DE will reduce the degree of variation of , and promote the convergence of the algorithm. When the difference between and is large, it means that and The optimal hyperparameters have significant differences. At this time, it is less likely that the optimal hyperparameters of are similar to those of neighboring local models. Geo-DE will increase the variation degree of

[0080] When obtaining the offspring of using DE / current-to-best / 1 or Geo-DE, to further enhance the search ability of the algorithm and improve the diversity of the population, it also includes: using the polynomial mutation operator to mutate each component in the offspring to achieve local micro-search in the solution space. The relevant expressions are as follows: (7) (8) In the formula, and are control parameters used to control the intensity of local mutation. is usually set to 20. is a random number uniformly distributed within and are respectively the minimum and maximum values of the th hyperparameter, and represents the number of model hyperparameters.

[0081] When obtaining the offspring of using DE / current-to-best / 1 or Geo-DE, combined with the boundary constraints of the multi-task constrained single-objective optimization problem, determine whether the values of each hyperparameter exceed the upper and lower limits of the decision space. When any component in violates the boundary constraints, repair it back to the decision space according to the following formula: (9) Step S133, perform offspring evaluation, that is, evaluate the performance of in each local model to determine whether operations such as mutation and crossover have generated better candidate solutions. The specific evaluation method is executed according to the steps of the foregoing embodiments and will not be elaborated here.

[0082] Parent population After all individuals in the parent population have completed hybridization and mutation, return all newly generated individuals to form the offspring population for updating the parent population .

[0083] Furthermore, the pseudocode and its annotations for generating offspring using the Geo-DE operator are as follows: Table 1 Pseudocode for generating offspring using the Geo-DE operator

[0084] The above is the process of the geographical proximity differential evolution operator Geo-DE for exploring the solution space to generate new individuals. After that, steps S133 and S143 can be executed, that is, offspring evaluation is performed, the population is updated, and then the new population is evaluated until the optimization goal of maximizing the accuracy of each local model is achieved, and the hyperparameter optimization results of each local model are output.

[0085] It should be noted that in traditional single-objective evolutionary algorithms, the offspring generated by the algorithm are usually only used to update their parents . However, in the hyperparameter optimization of the spatially stratified heterogeneity machine learning model, there may be spatial correlations between the optimal hyperparameters of local models in different sub-regions. In particular, the optimal hyperparameters of local models in neighboring sub-regions may be similar. Implementing hyperparameter sharing of local models in neighboring sub-regions can make more efficient use of computing resources and promote algorithm convergence.

[0086] Therefore, in this embodiment, in order to further improve the utilization efficiency of computing resources, as a further improvement, a Geo-SL selection operator is proposed for population update, that is, offspring evaluation is performed and the population is updated, including: randomly selecting one strategy from the global update strategy and the neighboring update strategy to update the population, where the neighboring update strategy uses the geographical proximity selection (Geo-SL) operator to update the population, including: traversing the offspring population, adding the local model with the best performance of the current offspring individual and all its neighbor models to the set ; for each local model in the set , find the parent individual with the best performance on this local model from the parent population , and use the offspring individual to update the parent individual .

[0087] Update the population through the Geo-SL selection operator so that the optimal hyperparameters can be shared among local models. At the same time, by proposing a multi-task constrained single-objective optimization problem and combining the geographical proximity difference operator Geo-DE and the geographical proximity selection operator Geo-SL, the three promote each other and jointly enhance the spatial collaborative optimization of the hyperparameters of local models in different sub-regions, improving the optimization efficiency.

[0088] Specifically, the Geo-SL selection operator uses each offspring in the offspring population to update the parent population , including the following steps: First, initialize an empty set to store the local models with optimal hyperparameters to be updated; then, randomly select one of the following two strategies: the global update strategy and the proximity update strategy, to determine the local models with optimal hyperparameters to be updated. Among them, when the global update strategy (abbreviated as strategy (1)) is selected, the local models of all sub-regions are added to the set ; when the proximity update strategy (abbreviated as strategy (2)) is selected, then traverse the offspring population , and add the local model with the best performance and all its neighbor models to the set .

[0089] Next, for each local model in , find the individual with the best performance in from the parent population , and update it with the offspring . Specifically, according to formulas (1) and (2), compare and in terms of their performance on (that is, compare the accuracy difference between the two on ). If is better than , then replace in the parent population with , otherwise keep in the parent population . Finally, after updating the optimal hyperparameters of all local models in , return the updated population as the new parent population to participate in the next round of iteration.

[0090] As Figure 5 shown, use a random number to select the update strategy. When When it is, select the optimal individuals of all local models, that is, update the population using strategy (1); otherwise, select the optimal individuals of each local model and its neighboring models, that is, update the population using strategy (2).

[0091] In this embodiment, the proposed Geo-SL selection operator (i.e., strategy (2)) can enable the sharing of the optimal hyperparameters of local models in each sub-region, further improving the utilization efficiency of computing resources. Further, strategy (1) can update the optimal individuals corresponding to each local model in the parent population, enabling the algorithm to converge rapidly; strategy (2) is more conservative and only updates the optimal individuals corresponding to the local models in neighboring sub-regions, realizing the sharing of the optimal hyperparameters between the local models in neighboring sub-regions. The combination of the two forms a balance between the convergence and exploration of the algorithm.

[0092] As an example, the pseudo-code and annotations of the Geo-SL selection operator are as follows: Table 2 Pseudo-code and annotations of the Geo-SL selection operator

[0093] As a further improvement, it also includes the optimization process of GeoEvo. Specifically, it includes the following: First, complete the initialization of the algorithm. Randomly sample and generate candidate solutions from the decision space as individuals to form a population . Use each candidate solution to set the hyperparameters for the local machine learning models in each sub-region , and obtain the local machine learning models under different hyperparameter settings in this sub-region . Assume there are sub-regions, then local models are generated. After the hyperparameter setting is completed, use the training set of each sub-region to independently complete feature selection and train all corresponding local models in each sub-region. Use the validation set to evaluate each candidate solution on each trained local machine learning model to obtain the objective value of each initial candidate solution.

[0094] Secondly, iteratively search for the optimal hyperparameters. Generate the offspring of each individual to produce an offspring population, and use the offspring population to update the parent population.

[0095] Finally, when the algorithm termination condition is reached, output the optimal hyperparameters of each local machine learning model, otherwise continue to iterate, search for new individuals and update the population.

[0096] Specifically, the optimization pseudo-code of GeoEvo is as follows: Table 3 Optimization process of GeoEvo

[0097] In summary, in this embodiment, by defining a multi-task constrained single-objective optimization problem for hyperparameter tuning (HPT) of a machine learning model for spatial hierarchical heterogeneity, a single-population multi-task optimization algorithm embedded with geographical association is used to solve this multi-task constrained single-objective optimization problem. A geographical proximity differential evolution Geo-DE operator is proposed, and the optimal hyperparameters of local models in neighboring sub-regions are used to guide the search, that is, the spatial dependence between neighboring sub-regions is utilized to achieve collaborative optimization between local models in different sub-regions during the search process, improving the search efficiency and accuracy of the algorithm.

[0098] A further technical contribution lies in defining a multi-task constrained single-objective optimization problem for HPT of a machine learning model for spatial hierarchical heterogeneity. Through the design of the objective function, the algorithm can take into account the hyperparameter optimization tasks of multiple local models under the condition of a single evolutionary population, and endow the traditional evolutionary algorithm with the ability to implicitly perform global spatial collaborative optimization during the search process, maintaining the diversity of the population and improving the search accuracy of the algorithm.

[0099] A further technical contribution lies in proposing a geographical proximity differential evolution operator Geo-DE, which uses the optimal hyperparameters of local models in neighboring sub-regions to guide the search, enhancing the search ability of the algorithm.

[0100] A further technical contribution lies in designing a geographical proximity selection operator Geo-SL, enabling local models to selectively share optimal hyperparameters and improving the utilization rate of computing resources.

[0101] To verify the effectiveness and robustness of the proposed GeoEvo framework in multiple machine learning models, it also includes: implementing the GeoEvo framework on multiple different datasets and verifying it, where the different datasets include German soil organic carbon and Chinese PM 2.5 dataset. Below, taking the Chinese PM 2.5 dataset as an example, the verification process is described.

[0102] Suppose a study area is divided into four sub-regions (as Figure 6 shown), and the adjacency between sub-regions is determined by edge adjacency. For example, partition 1 is adjacent to partition 2 and partition 3, and partition 2 is adjacent to partition 1 and partition 4.

[0103] The following settings are made for the single-population multi-task optimization algorithm embedded with geographical association (hereinafter referred to as: the algorithm): the population size is set to 5; the number of iterations is set to 1; the individual coding is set as: a two-dimensional vector represents an individual, and each individual represents a hyperparameter configuration. Hyperparameters and their boundaries: the first hyperparameter ∈ [1, 15], and the second hyperparameter ∈ [1, 10].

[0104] The algorithm can be executed according to the following process: 1. Population initialization (1). Generate the initial population: Randomly generate a population , where each row represents an individual and each column corresponds to a hyperparameter (as Figure 7 shown). The two components of Figure 7 represent the first and second hyperparameters of the model respectively. In (a) is the parental population matrix, where rows represent individuals and columns represent hyperparameters. In (b) are the scores of individuals on each local model. The row number represents the individual index and the column number represents the local model index. The red markings indicate the row maxima, representing the local models on which each individual performs best. The background shading indicates the column maxima, representing the individuals on which each local model performs best. For example, individuals perform best on local models respectively, while the individuals that perform best on local model are respectively.

[0105] (2). Evaluate the initial population: Calculate the scores of each individual on each local model to obtain a score matrix, where each row represents an individual and each column represents a local model. As shown in Figure A2, individual performs best on local model with a score of 0.2, while the individual that performs best on local model is with a score of 0.3.

[0106] 2. Population evolution The search in the solution space is guided by crossover and mutation operators. This process is divided into two stages: (1) hybridization and mutation of parental individuals; (2) evaluation of offspring individuals.

[0107] Stage 1: Hybridization and mutation of parental individuals For each individual, offspring are generated using a randomly selected operator (Geo-DE or DE / current-to-best / 1), and the polynomial mutation operator further guides the generation of offspring. Taking individuals and as an example, the steps are as follows: (I). Evolve using the Geo-DE operator (1). Determine the parents: Individual performs best on local model , so select the individual that performs best on as the parent .

[0108] (2) Neighboring Parent Selection: Randomly select the individual that performs best on the adjacent models of and to obtain and .

[0109] (3) Offspring Generation: Use the Geo-DE formula (Formula (6)) to calculate the offspring : , (4) Polynomial Mutation: Randomly mutate the hyperparameter components (Formulas (7) and (8)). Assume the result is .

[0110] (5) Boundary Repair: Since the second component exceeds the upper bound, repair it to (Formula (9)).

[0111] (II) Evolve using the DE / current-to-best / 1 operator (1) Parent Determination: The individual performs best on the local model , so select the individual that performs best on as the parent .

[0112] (2) Randomly Select Parents: Randomly select two additional parents from the remaining individuals in the parent population excluding and to obtain and .

[0113] (3) Offspring Generation: Use the DE / current-to-best / 1 formula (Formula (5)) to calculate the offspring : , (4) Polynomial Mutation: Randomly mutate the hyperparameter components (Formulas (7) and (8)). Assume the result is .

[0114] (5) Boundary Repair: Since the second component exceeds the upper bound, repair it to (Formula (9)).

[0115] (III) Evolution Completed Following the above steps, all individuals in the parent population complete evolution to obtain the offspring population .

[0116] Phase 2: Offspring Evaluation Evaluate the offspring population Performance on each local model and update the corresponding score matrix, as Figure 8 shown. Among them, (a) is the offspring population matrix. (b) The scores of each offspring on each local model.

[0117] 3. Population Update Use the Geo-SL operator to update the parent population. Take the offspring individuals and as an example: (I). Update using the global update strategy

[0118] (1). Determine the set of local models to be updated : Add all local models to the set , to obtain .

[0119] (2). Score comparison: Compare the scores of the offspring individual on each local model in with the optimal score of the corresponding parent population. If the offspring performs better than the parent, update the optimal individual of this local model. As Figure 9 shown, since 0.5 < 0.9, 0.5 = 0.5, 0.1 < 0.3, and 0.7 = 0.7, in the performance of any local model is not better than that of the parent individual. Therefore, is not used to update the optimal hyperparameters of any local model in .

[0120] (II). Update using the proximity update strategy

[0121] (1). Determine the set of local models to be updated : Add the best-performing local model and its adjacent local models ( and ) to , to obtain .

[0122] (2). Score comparison: In , the offspring individual is better than the parent on the local model (0.4 > 0.3), so The optimal individual is replaced. As Figure 10 shown, the parent individual and its score are replaced in the parent population and the score matrix with Figure 10 and its score, obtaining the updated parent population and score matrix, specifically referring to

[0123] 4. Algorithm termination Repeat the steps of "2. Population evolution" and "3. Population update" until the termination condition is met. As Figure 10 shown in (c) and (d) in : , : , : and : .

[0124] The above execution process is only an exemplary description and does not constitute a limitation to this application.

[0125] Embodiment 2: This embodiment provides a hyperparameter tuning device for a spatially hierarchical machine learning model, and the device includes: A parameter acquisition unit configured to acquire the hyperparameters to be optimized of the local models in each sub-region; Encode the combinations of the hyperparameters to be optimized to form a multi-dimensional decision vector; Construct a multi-task constrained single-objective optimization problem for hyperparameter tuning of a spatially hierarchical heterogeneous machine learning model with the objective of maximizing the accuracy of each local model; Use each decision vector as an individual, and multiple individuals form a population, and use a single-population multi-task optimization algorithm with embedded geographical association to solve the multi-task constrained single-objective optimization problem to obtain the hyperparameter optimization results of each local model; The single-population multi-task optimization algorithm with embedded geographical association performs hybrid mutation by combining the geographical proximity differential evolution Geo-DE operator and the classical differential evolution operator. The Geo-DE operator generates offspring based on the differences between the optimal hyperparameters of the local models in neighboring sub-regions to guide the search direction using the spatial association between sub-regions.

[0126] The hyperparameter tuning device for a spatially hierarchical machine learning model provided in this embodiment can implement the steps and processes of the hyperparameter tuning method for a spatially hierarchical machine learning model provided in any of the above embodiments and achieve the same technical effects, which will not be elaborated here one by one.

[0127] Example 3: The embodiments of the present application can be applied to Figure 11 the electronic device shown in the figure. The electronic device can be, but is not limited to, mobile terminals such as mobile phones, tablet computers, handheld computers, personal digital assistants (PDAs), etc., smart home devices such as smart TVs, smart cameras, etc., wearable devices such as smart bracelets, smart watches, smart glasses, or other computer devices such as desktop computers, laptop computers, notebook computers, ultra-mobile personal computers (UMPCs), netbooks, and smart screens.

[0128] As Figure 11 shown in the figure, the electronic device 200 may include one or more of the following components: a processor 201, a memory 203, a communication interface 202, and a communication bus 204. Among them, the memory 203 can be connected to the processor 201 through the bus 204. The bus can transmit data between the processor 201 and the memory 203. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0129] The processor 201 may include one or more processing cores. The processor 201 can connect various parts within the entire electronic device 200 using various interfaces and circuits. By running or executing instructions, programs, code sets, or instruction sets stored in the memory 203, and by invoking the data stored in the memory 203, it performs various functions of the electronic device 200 and processes data. Exemplarily, the processor 201 may include an application processor (AP), a modem processor, a CPU, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a field-programmable gate array (FPGA), a programmable logic array (PLA), and / or a neural-network processing unit (NPU), etc. Among them, the CPU mainly processes the operating system, user interface, and application programs, etc.; the GPU is responsible for rendering and drawing the content to be displayed; the NPU is used to implement artificial intelligence (AI) functions; the modem is used to process wireless communication. Different processing units can be independent devices or integrated in one or more processors. For example, the multiple processing units shown above are all integrated in one SoC, or the AP is a separate semiconductor chip, and the other processing units are integrated in one SoC. This application does not make any limitations in this regard.

[0130] The memory 203 (computer-readable storage medium) may include a random access memory (RAM), may also include a read-only memory (ROM), and may further include a non-transitory computer-readable storage medium. The memory 203 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 203 may include a program storage area and a data storage area. Among them, the program storage area can store instructions for implementing the operating system, instructions for at least one function, such as the method for hyperparameter tuning of the spatial hierarchical machine learning model, etc.; the data storage area can store data created according to the use of the electronic device 200, such as training data in the research area, intermediate model output results, etc.

[0131] In addition, those skilled in the art can understand that the structure of the electronic device 200 shown in the above drawings does not limit the electronic device 200. The electronic device may include more or fewer components than shown in the drawings, or combine certain components, or have different component arrangements. For example, the electronic device 200 also includes components such as a microphone, a speaker, a radio frequency circuit, a sensor, an audio circuit, a power supply, and a Bluetooth module, which will not be elaborated here.

[0132] The embodiment of the present application also provides a computer program product including computer-executable instructions. In one embodiment, the computer-executable instructions are used to cause a computer to execute the functions in the above method embodiment.

[0133] The computer-executable instructions can be stored in a computer-readable storage medium. The embodiment of the present application also provides a computer-readable storage medium, in which executable instructions are stored. In one embodiment, the computer-executable instructions are used to cause a computer to execute the functions in the above method embodiment.

[0134] The foregoing is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A hyperparameter tuning method for a spatial hierarchical machine learning model, characterized in that: include: Obtain the hyperparameters to be optimized of the local model of each sub-region; Constructing a multi-task constrained single-objective optimization problem; the multi-task constrained single-objective optimization problem takes the hyperparameter optimization of each local model as an optimization task, and regards the hyperparameter optimization of multiple local models as a multi-task constrained single-objective optimization problem, which is defined by an objective function and a boundary constraint; A single-species multi-task optimization algorithm embedded with geographic association is used to solve the multi-task constrained single-objective optimization problem to obtain hyperparameter optimization results of each local model; The single-population multi-task optimization algorithm embedded with geographic association is based on an evolutionary algorithm and adopts a geographic proximity differential evolution Geo-DE operator for hybridization mutation. The Geo-DE operator generates offspring based on the differences between the optimal hyperparameters of the local models of neighboring sub-regions to guide the search direction by utilizing the spatial associations between sub-regions.

2. The method according to claim 1, characterized in that The multi-task constrained single-objective optimization problem is solved by using a single-species multi-task optimization algorithm embedded with geographic association to obtain the hyperparameter optimization results of each local model, including: Based on the pre-acquired training sets of each sub-region, each local model is trained using the initial population, and the accuracy of each individual in the initial population on each local model is evaluated based on the pre-acquired validation sets of each sub-region to obtain an evaluation result; The initial population is used as the parent population, and the Geo-DE operator is used to generate offspring, or an evolutionary operator is randomly selected from the Geo-DE operator and the classical differential evolution operator to generate offspring; Conduct offspring evaluation and update the population; The updated population is used as the new parent population. Based on the new parent population, each local model is retrained and its accuracy is evaluated until the optimization goal of maximizing the accuracy of each local model is achieved. The hyperparameter optimization results of each local model are output.

3. The method according to claim 1, characterized in that When the Geo-DE operator is used to generate offspring, the following steps are performed: From all individuals in the population, select the individual that performs best on the local model with the best performance of the current individual, and record it as the optimal hyperparameter of the parent generation; Randomly select two models from all neighboring models of the local model with the best individual performance at the current time, and record them as the first neighboring model and the second neighboring model; The optimal hyperparameters corresponding to the first neighbor model and the second neighbor model are searched in the parent population; the optimization direction is indicated by the optimal hyperparameters of the parent, and the optimization direction and step size are corrected based on the difference between the two optimal hyperparameters to generate offspring.

4. The method according to claim 1, characterized in that After generating the offspring, it also includes: A polynomial mutation operator is used to mutate each component in the offspring to achieve local micro-search in the solution space.

5. The method according to claim 1, characterized in that The boundary constraints are the upper and lower limits of each hyperparameter value; After hybridization and mutation, it also includes: Based on the boundary constraints, the components in the offspring that violate the constraint conditions are repaired so that the components satisfy the constraint conditions again.

6. The method according to claim 2, characterized in that Updating the population, including: randomly selecting a strategy from the global update strategy and the neighboring update strategy to update the population, Among them, the neighbor update strategy uses the geographic neighbor selection operator to update the population, including: traversing the offspring population, adding the local model with the best performance of the current offspring individual and all its neighbor models to the set ; for the set For each local model in , find the parent individual with the best performance on the local model from the parent population and update the parent individual with the child individual.

7. The method according to claim 1, characterized in that The determination coefficient of each sub-region on the validation set is used to characterize the accuracy of the local model.

8. A hyperparameter tuning device for a spatial hierarchical machine learning model, characterized in that: include: A parameter acquisition unit, configured to acquire a hyperparameter to be optimized of a local model of each sub-region; A model building unit is configured to build a multi-task constrained single-objective optimization problem; the multi-task constrained single-objective optimization problem takes the hyperparameter optimization of each local model as an optimization task, regards the hyperparameter optimization of multiple local models as a multi-task constrained single-objective optimization problem, and is defined by an objective function and a boundary constraint; A solving unit is configured to solve the multi-task constrained single-objective optimization problem by using a single-species multi-task optimization algorithm embedded with geographic association to obtain hyperparameter optimization results of each local model; The single-population multi-task optimization algorithm embedded with geographic association is based on an evolutionary algorithm and adopts a geographic proximity differential evolution Geo-DE operator for hybridization mutation. The Geo-DE operator generates offspring based on the differences between the optimal hyperparameters of the local models of neighboring sub-regions to guide the search direction by utilizing the spatial associations between sub-regions.

9. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the hyperparameter tuning method for a spatial hierarchical machine learning model as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instruction is executed by a processor, the steps of the hyperparameter tuning method for a spatial hierarchical machine learning model as described in any one of claims 1 to 7 are implemented.

Images