Multi-objective optimization method, system and related products based on CMA-ES

Abstract

The application provides a multi-objective optimization method and system based on CMA-ES and related products, and relates to the technical field of image processing. The dimensionless loss change rate makes the subsequent preference weight only used for adjusting the optimization priority of different objectives, without balancing the order of magnitude difference of different optimization objectives, thereby significantly reducing the complexity of weight parameter adjustment. The loss scalar is converted into a loss ranking value, which can obtain a certain loss value range without changing the original optimization objective. The loss ranking value and the global optimal anchor value are combined to generate a final loss scalar value, which is conducive to more accurately determining the improvement degree of the candidate sample relative to the historical observation, and improving the reliability of the convergence criterion. The final loss scalar value is fed back to a single optimizer, which can avoid the calculation and storage overhead caused by maintaining multiple optimizers, effectively solve the multi-objective optimization problem without changing the single-objective update mechanism, and significantly improve the optimization efficiency and resource utilization.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a multi-objective optimization method, system and related products based on CMA-ES. Background Technology

[0002] Covariance Matrix Adaptation Evolution Strategy (CMA-ES) is an evolutionary algorithm for solving continuous black-box optimization problems. It has advantages such as low requirements for the continuity of the objective function and strong adaptability to noise and non-convex problems, and is widely used in scenarios such as engineering parameter optimization, model calibration, and hyperparameter search for complex systems. The standard CMA-ES algorithm typically uses a single fitness scalar as the optimization criterion. In each iteration, the optimizer samples multiple candidate samples from the current search distribution and associates each candidate sample with a fitness scalar. Then, based on the ranking of the sample fitness, it updates the center position and covariance structure of the search distribution, thereby gradually approaching the optimal solution.

[0003] However, in practical applications, optimization tasks often require trade-offs among multiple objectives, forming multi-objective optimization problems. Since the standard CMA-ES is essentially a single-objective optimization algorithm, its update mechanism relies on a one-to-one correspondence between candidate samples and fitness scalars. Therefore, when dealing with multi-objective problems, the fitness vector composed of multiple objectives needs to be converted into a single scalar. Traditional schemes typically use a weighted sum method to scalarize the multi-objective fitness vector, that is, assigning weights to each objective and performing a linear combination. This method is simple to implement, but the selection of weights and scale normalization are heavily dependent on experience, and the calculation logic of the weights, to some extent, determines the actual optimization direction. This can easily lead to the optimization process deviating from the originally expected multi-objective trade-off relationship, making it difficult to stably reflect the degree of optimization improvement across multiple objectives. Summary of the Invention

[0004] The purpose of this application is to provide a multi-objective optimization method, system and related products based on CMA-ES, so as to overcome the defects of traditional technology that uses weighted sum to scalarize the multi-objective fitness vector, which causes the optimization process to deviate from the original expected multi-objective trade-off relationship and makes it difficult to stably reflect the degree of optimization improvement of multiple objectives.

[0005] Firstly, this application proposes a multi-objective optimization method based on CMA-ES, including: The CMA-ES optimizer is used to sample multiple candidate samples for the current generation, where each candidate sample corresponds to multiple targets; For each candidate sample, the loss value of each target in the candidate sample is calculated, and the loss change rate of each target is determined based on the loss statistics of the previous generation candidate samples on the corresponding targets; the loss change rates of each target are summed based on the preset preference weights to obtain the loss scalar of the candidate sample. The candidate samples in the current generation are sorted according to the loss scalar, and the sorting result and the global optimal anchor value are combined to generate the final loss scalar value of each candidate sample; wherein, the global optimal anchor value is determined based on the sum of the multi-objective losses of each candidate sample in the iteration process; The final loss scalar value is provided to the CMA-ES optimizer so that the CMA-ES optimizer updates its search distribution parameters based on the final loss scalar value and generates candidate samples for the next iteration.

[0006] In one embodiment, the step of calculating the loss value of each target in each candidate sample and determining the loss change rate of each target based on the loss statistics of previous candidate samples on the corresponding targets includes: For each objective, determine the loss statistic for that objective; Calculate the magnitude of the loss change of each candidate sample in the current generation on the target relative to the loss statistic; Based on the loss values ​​of each candidate sample in the current generation on the target, the scale parameter of the target is determined, and a variance stability coefficient is superimposed on the scale parameter to obtain a standardized scale. The ratio of the loss change magnitude to the standardized scale is calculated to obtain the loss change rate of the target.

[0007] In one embodiment, the loss statistic is the mean loss of each candidate sample in the previous generation on the target, or the global minimum loss value recorded by the target in the historical iteration process, and the scale parameter is the standard deviation of the loss of each candidate sample in the current generation on the target.

[0008] In one embodiment, the loss statistic is the median loss of each candidate sample in the previous generation on the target, or the global minimum loss value recorded by the target in the historical iteration process, and the scale parameter is the interquartile range of each candidate sample in the current generation on the target.

[0009] In one embodiment, the method for determining the globally optimal anchor value based on the sum of multi-objective losses of each candidate sample during the iteration process includes: For each candidate sample in the current generation, the loss values ​​of all targets in the candidate sample are summed to obtain the total target loss; the minimum target loss of the current generation is determined from the total target loss. The minimum target loss sum of the current generation is compared with the minimum target loss sum recorded in the historical iterations, and the smaller value is taken as the global optimal anchor point value.

[0010] In one embodiment, the step of sorting the candidate samples of the current generation according to the loss scalar, and combining the sorting result with the global optimal anchor value to generate the final loss scalar value for each candidate sample, includes: Convert the loss scalar into loss sorted values ​​in ascending order; Multiply the loss ranking value by a scaling factor, and add the scaled loss ranking value to the global optimal anchor value to obtain the final loss scalar value for each candidate sample.

[0011] In one embodiment, the method for presetting preference weights includes: During the iterative optimization process, the numerical sequence of the optimization variables used to generate candidate samples within a preset time window is obtained; Calculate the numerical stability index of the optimization variable based on the numerical sequence; The preference weights corresponding to each target are adaptively updated based on the numerical stability index, so as to increase the preference weights corresponding to targets whose numerical stability meets the preset conditions, and decrease the preference weights corresponding to targets whose numerical stability does not meet the preset conditions.

[0012] Secondly, this application proposes a multi-objective optimization system based on CMA-ES, the system comprising: The sampling module is used to sample multiple candidate samples of the current generation using the CMA-ES optimizer, where each candidate sample corresponds to multiple targets; The processing module is used to calculate the loss value of each target in each candidate sample, and determine the loss change rate of each target based on the loss statistics of the previous generation candidate samples on the corresponding targets; sum the loss change rates based on preset preference weights to obtain the loss scalar of the candidate sample; sort the candidate samples of the current generation according to the loss scalar, and combine the sorting result with the global optimal anchor value to generate the final loss scalar value of each candidate sample; wherein, the global optimal anchor value is determined based on the sum of the multi-target losses of each candidate sample in the iteration process; An update module is used to provide the final loss scalar value to the CMA-ES optimizer so that the CMA-ES optimizer updates its search distribution parameters based on the final loss scalar value to generate candidate samples for the next iteration.

[0013] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method steps in the first aspect.

[0014] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the method steps of the first aspect.

[0015] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the method steps in the first aspect.

[0016] The aforementioned CMA-ES-based multi-objective optimization methods, systems, and related products have at least the following advantages: This application employs the CMA-ES optimizer to sample multiple candidate samples for the current generation and calculates the rate of change of loss for each objective. Based on preset preference weights, the rate of change of loss is summed to obtain the loss scalar for each candidate sample. The candidate samples are then ranked according to their loss scalars, and the ranking result is combined with the globally optimal anchor value to generate the final loss scalar value for each candidate sample. The final loss scalar value is provided to the CMA-ES optimizer, enabling it to update its search distribution parameters and generate candidate samples for the next iteration. Using this approach, this application calculates the rate of change of loss for each objective based on the previous generation's loss statistics and the current loss value. This dimensionless rate of change of loss ensures that subsequent applied preference weights are only used to adjust the optimization priority of different objectives, without needing to balance the order-of-magnitude differences between different optimization objectives. This significantly reduces the complexity of weight tuning and avoids single-objective dominance caused by differences in objective dimensions, thus improving the stability and controllability of multi-objective optimization. Furthermore, the loss scalar is converted into a loss ranking value. The range of the loss ranking value is uniquely determined by the number of samples in the current generation, and the CMA-ES optimizer is invariant to monotonic changes in loss. The above conversion operation obtains a definite range of loss values ​​without changing the original optimization objective. Further, the loss ranking value is combined with the global optimal anchor value to generate the final loss scalar value used for updating the CMA-ES optimizer. This allows samples of the same generation to share the same global optimal anchor value, thus the quality of samples in the same generation is entirely determined by the loss ranking value. This helps to more accurately determine the degree of improvement of candidate samples relative to historical observations and improves the reliability of the convergence criterion. Finally, by feeding the final loss scalar value back to a single CMA-ES optimizer to update the search distribution parameters, the computational and storage overhead of maintaining multiple optimizers can be avoided. Therefore, without changing the single-objective update mechanism of CMA-ES, an effective solution to multi-objective optimization problems can be achieved, significantly improving optimization efficiency and resource utilization. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a multi-objective optimization method based on CMA-ES in one embodiment; Figure 2 This is a flowchart illustrating the steps for determining the rate of change of loss for each objective in one embodiment. Figure 3 This is a flowchart illustrating a multi-objective optimization method based on CMA-ES in another embodiment; Figure 4 Here is a block diagram of a CMA-ES-based multi-objective optimization system in one embodiment; Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0018] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0019] Some exemplary embodiments of this application have been described for illustrative purposes. It should be understood that this application may be implemented in other ways not specifically shown in the accompanying drawings.

[0020] Please see Figure 1 In one exemplary embodiment, this application provides a multi-objective optimization method based on CMA-ES, specifically including the following steps: Step 102: The CMA-ES optimizer is used to sample multiple candidate samples for the current generation, where each candidate sample corresponds to multiple targets.

[0021] Specifically, a candidate sample refers to a vector of values ​​for a set of optimization variables. In each iteration, the CMA-ES optimizer randomly samples λ candidate samples for the current generation based on the current search distribution. For example, the steps include: determining the distribution parameters of the current search, which include the mean vector, covariance matrix, and step size; generating a random vector Zi for the i-th candidate sample, which typically follows a standard normal distribution; mapping the random vector to a candidate solution, i.e., the optimization variables, based on the above distribution parameters; repeating the above steps λ times to finally obtain the λ candidate samples for the current generation.

[0022] Generally, for a set of candidate samples, the same set of objective functions are used for evaluation, meaning that each candidate sample corresponds to N objectives. Here, a set of candidate samples refers to the collection of candidate samples obtained by the CMA-ES optimizer in a single iteration based on the same search distribution.

[0023] Step 104: For each candidate sample, calculate the loss value of each target in the candidate sample, and determine the loss change rate of each target based on the loss statistics of the previous generation candidate samples on the corresponding targets; sum the loss change rates based on the preset preference weights to obtain the loss scalar of the candidate sample. Specifically, the loss value is used to measure the difference between a sample solution and the ideal optimal solution under the same optimization objective. For an optimization objective, a lower loss for a sample indicates a better result for that sample on that objective. For example, one way to calculate the loss value includes: comparing the model output corresponding to the candidate sample with the expected target value to obtain an error value, and inputting the error value into a preset loss function to calculate the loss value.

[0024] Loss statistics are calculated based on the set of loss values ​​of each candidate sample in the previous and / or current generation on the same objective.

[0025] The loss change rate refers to the magnitude of the change in loss value of a candidate sample for a specific objective relative to the loss statistic of the corresponding objective in the previous generation. This loss change is standardized using the scaling statistics of the corresponding objective in the previous or current generation, resulting in a dimensionless ratio. It characterizes the degree of optimization improvement of the candidate sample on that objective and enables comparable measurement between different objectives. In this embodiment, since the loss change rate has undergone scaling, it is a dimensionless value. Therefore, in the subsequent weighted summation step, the applied preference weights no longer need to balance the order-of-magnitude differences between different optimization objectives; they are only used to adjust the optimization improvement priority of different objectives, thereby converting multi-objective information into a single scalar input suitable for the CMA-ES optimizer.

[0026] Step 106: Sort the candidate samples of the current generation according to the loss scalar, and combine the sorting result with the global optimal anchor value to generate the final loss scalar value of each candidate sample; wherein, the global optimal anchor value is determined based on the sum of the multi-objective losses of each candidate sample in the iteration process.

[0027] Specifically, the distribution parameters of the CMA-ES optimizer are updated based on the quality of candidate samples. By ranking the candidate samples, the relative merits of their multi-objective comprehensive performance can be provided to the CMA-ES optimizer. Furthermore, multi-objective loss evaluation often involves measurement noise. Converting the loss scalar to a ranking value ensures that the differences between candidate samples are mainly reflected in their rankings, significantly reducing the interference of measurement noise on the update process and improving robustness. Moreover, the range of ranking values ​​is uniquely determined by the number of samples in the current generation, and the CMA-ES optimizer is invariant to monotonic changes in loss. This conversion operation, while obtaining a defined range of loss values, does not alter the original optimization objective.

[0028] The global optimal anchor value refers to the sum of the best multi-objective losses across historical iterations. The ranking results and the global optimal anchor value are combined to form the final loss metric for each candidate sample participating in CMA-ES optimization. Since candidate samples of the same generation share the same global optimal anchor value, the quality of candidate samples within the same generation is entirely determined by the loss ranking value. Simultaneously, the global optimal anchor value serves as an optimization benchmark across generations and can be used as a criterion for determining whether the iteration has converged.

[0029] Step 108: Provide the final loss scalar value to the CMA-ES optimizer so that the CMA-ES optimizer updates its search distribution parameters based on the final loss scalar value to generate candidate samples for the next iteration.

[0030] Specifically, after receiving the final loss scalar values ​​corresponding to each candidate sample in the current generation, the CMA-ES optimizer selects a preset number of elite samples as parent samples in ascending order. It then re-weights and reorganizes these elite samples according to a weighting rule related to their ranking position to update the mean vector of the search distribution, shifting the center of the search distribution towards the low-loss region. Simultaneously, based on the displacement direction of the elite samples relative to the old mean and their correlation statistics, the CMA-ES optimizer adaptively updates the covariance matrix to adjust the expansion scale and principal direction of the search distribution in different variable directions, thereby enhancing its search capability in directions more likely to produce low-loss solutions. Furthermore, the CMA-ES optimizer adjusts the step size according to changes in the cross-generational evolution path to adaptively control the overall search intensity. Finally, the CMA-ES optimizer uses the updated mean vector, covariance matrix, and step size to generate the search distribution for the next iteration and samples the next generation of candidate samples from this search distribution.

[0031] The aforementioned CMA-ES-based multi-objective optimization method uses a CMA-ES optimizer to sample multiple candidate samples for the current generation and calculates the rate of change of loss for each objective. Based on preset preference weights, the loss rates of change are summed to obtain the loss scalar for each candidate sample. The candidate samples are then ranked according to their loss scalars, and the ranking result is combined with the global optimal anchor value to generate the final loss scalar value for each candidate sample. This final loss scalar value is provided to the CMA-ES optimizer, allowing it to update its search distribution parameters and generate candidate samples for the next iteration. Using this approach, this application calculates the rate of change of loss for each objective based on the previous generation's loss statistics and the current loss value. This dimensionless rate of change of loss ensures that subsequent applied preference weights are only used to adjust the optimization priority of different objectives, without needing to balance the order-of-magnitude differences between different optimization objectives. This significantly reduces the complexity of weight tuning and avoids single-objective dominance caused by differences in objective dimensions, thus improving the stability and controllability of multi-objective optimization. Furthermore, the loss scalar is converted into a loss ranking value. The range of the loss ranking value is uniquely determined by the number of samples in the current generation, and the CMA-ES optimizer is invariant to monotonic changes in loss. The above conversion operation obtains a definite range of loss values ​​without changing the original optimization objective. Further, the loss ranking value is combined with the global optimal anchor value to generate the final loss scalar value used for updating the CMA-ES optimizer. This allows samples of the same generation to share the same global optimal anchor value, thus the quality of samples in the same generation is entirely determined by the loss ranking value. This helps to more accurately determine the degree of improvement of candidate samples relative to historical observations and improves the reliability of the convergence criterion. Finally, by feeding the final loss scalar value back to a single CMA-ES optimizer to update the search distribution parameters, the computational and storage overhead of maintaining multiple optimizers can be avoided. Therefore, without changing the single-objective update mechanism of CMA-ES, an effective solution to multi-objective optimization problems can be achieved, significantly improving optimization efficiency and resource utilization.

[0032] Please see Figure 2 Optionally, for each candidate sample, the loss value of each target in the candidate sample is calculated, and the loss change rate of each target is determined based on the loss statistics of previous candidate samples on the corresponding targets, including: Step 202: For each target, determine the loss statistic for that target.

[0033] Step 204: Calculate the magnitude of the change in the loss value of each candidate sample in the current generation on the target relative to the loss statistic.

[0034] Step 206: Based on the loss values ​​of each candidate sample in the current generation on the target, determine the scale parameter of the target, and superimpose the variance stability coefficient on the scale parameter to obtain the standardized scale.

[0035] Step 208: Calculate the ratio of the magnitude of the loss change to the standardized scale to obtain the loss change rate of the target.

[0036] Specifically, for each objective, the set of loss values ​​for each candidate sample from the previous generation on that objective is obtained, and a loss statistic is derived based on this set of loss values. In the first iteration, since there is no previous generation data, the loss statistic is calculated using the set of loss values ​​for each candidate sample from the current generation on that objective. This set of loss values ​​serves as the basis for characterizing the comprehensive performance of the candidate sample on the optimization objective. Its value can be a set of the candidate sample's loss values ​​on each objective, or it can be the total loss value obtained by aggregating the losses of each objective. Standardization can be based on Z-score standardization or Robust standardization. Z-score standardization measures the deviation of a sample from the mean of its sample set, expressing this deviation in terms of standard deviation. Robust standardization uses the median and interquartile range instead of the mean and standard deviation; this method is less sensitive to outliers and is more robust.

[0037] Optionally, the loss statistic is the mean loss of each candidate sample in the previous generation on the target, or the global minimum loss value recorded by the target in the historical iteration process, and the scale parameter is the standard deviation of the loss of each candidate sample in the current generation on the target.

[0038] For example, taking the g-th generation as the current generation, CMA-ES samples λ candidate samples Xi (i∈{0,1,2,…, λ-1}) in the g-th generation, and each candidate sample corresponds to the loss Ln (n∈{0,1,2,…,N-1}) of N targets.

[0039] When the loss statistic is the average loss of all candidate samples in the previous generation on that target, the average loss of the Nth target used in the current generation comes from the previous generation, i.e., in the (g-1)th generation. The expression for the mean loss preMean of candidate samples on the Nth target is: .in, Let λ represent the mean loss of the nth objective used in the g-th iteration; λ represents the number of candidate samples obtained in each generation. This represents the mean loss of the nth objective used in the (g-1)th iteration.

[0040] Since the first generation does not have the previous generation's loss statistics, the average loss of the λ candidate samples in the first generation on the nth target is used as the initial average loss. The preMean of other generations uses the average loss of the λ candidate samples in the previous generation on N targets respectively.

[0041] The standard deviation σ of the loss of the current generation of λ candidate samples on the nth target is calculated as follows:

[0042] in, Let be the average loss of the current generation of λ candidate samples on the nth target.

[0043] The loss of the nth target for the i-th candidate sample is standardized using the corresponding preMean and σ to obtain the rate of change of loss. Its expression is:

[0044] in, This is the variance stability coefficient, used to avoid numerical instability caused by excessively small scale parameters.

[0045] Furthermore, when the loss statistic is the global minimum loss value recorded by the objective during historical iterations, the rate of change of loss... The expression is:

[0046] in, This represents the global minimum loss value for the objective, and the historical iteration process includes both previous and current generations.

[0047] Optionally, the loss statistic is the median loss of each candidate sample in the previous generation on the target, or the global minimum loss value recorded by the target in the historical iteration process, and the scale parameter is the interquartile range of each candidate sample in the current generation on the target.

[0048] Specifically, when the loss statistic is the median loss of each candidate sample in the previous generation on this target, the rate of change of loss is... The expression is:

[0049] in, This represents the median loss of each candidate sample in the previous generation on the nth objective. It represents the interquartile range of each candidate sample in the current generation on the nth target.

[0050] Correspondingly, when the loss statistic is the global minimum loss value recorded by the objective during historical iterations, the rate of change of loss is... The expression is:

[0051] By adopting the above scheme, this application converts the loss of each objective into a dimensionless and comparable rate of change of loss based on the previous generation of loss statistics. This significantly weakens the impact of differences in the dimensions and scales of different objectives on multi-objective optimization, so that the subsequent preference weights are only used to express the optimization priority without balancing the differences in the order of magnitude of different optimization objectives. This reduces the difficulty of weight parameter tuning and improves the stability of multi-objective trade-offs. At the same time, by superimposing the variance stability coefficient on the scale parameter and performing standardization, numerical instability caused by the scale parameter being too small can be avoided.

[0052] Optionally, methods for determining the globally optimal anchor value based on the sum of multiple objectives from previous candidate samples include: For each candidate sample in the current generation, the loss values ​​of all targets in the candidate sample are summed to obtain the total loss of multiple targets; the minimum target loss of the current generation is determined from the total loss of each target; the minimum target loss of the current generation is compared with the minimum target loss recorded in the historical iteration process, and the smaller value is taken as the global optimal anchor value.

[0053] For example, for each candidate sample Xi in the current generation, the sum of its N target losses Ln is calculated, resulting in multiple target loss sums. The minimum value within a generation is selected from these sums, and then compared with the minimum value in the historical records. The smaller of the two values ​​is taken as the globally optimal anchor value, expressed as:

[0054] In each iteration, the global optimal anchor value is updated using the above method before subsequent calculations are performed.

[0055] Optionally, the methods for presetting preference weights include: During the iterative optimization process, the numerical sequence of the optimization variables used to generate candidate samples within a preset time window is obtained; the numerical stability index of the optimization variables is calculated based on the numerical sequence; and the preference weights corresponding to each objective are adaptively updated according to the numerical stability index, so as to improve the preference weights corresponding to the objectives whose numerical stability meets the preset conditions, and reduce the preference weights corresponding to the objectives whose numerical stability does not meet the preset conditions.

[0056] Specifically, after each iteration, a representative optimization vector is selected from the candidate samples of the current generation as an elite sample. Here, "representative" means the variable vector corresponding to the candidate sample with the best loss scalar in the current generation.

[0057] Furthermore, during the iterative optimization process, the stability index of the variable vector corresponding to the elite samples within a preset time window is calculated. For example, the stability index is the fluctuation amplitude of each variable vector, which can be obtained by calculating the change between two adjacent generations or the degree of dispersion within the time window. The fluctuation amplitude of each variable vector is summarized to obtain the stability index. The smaller the stability index, the more stable the optimization variable is within the preset time window.

[0058] The preset condition refers to the fact that the optimization variable has converged or the change value is less than the preset threshold. If the stability index of an optimization variable meets the preset condition, the preference weight of the corresponding target will be increased accordingly to strengthen the priority of the optimization of the target. If the stability index of an optimization variable does not meet the preset condition, the preference weight of the corresponding target will be decreased accordingly to suppress the excessive pull of the target on the overall scalarization.

[0059] Furthermore, in addition to updating the preference weights based on the stability index mentioned above, in practical applications, the preference weights can also be updated based on other statistics within a preset time window. For example, the weights of targets with high noise can be reduced to avoid noise-driven updates based on the fluctuation of the loss rate of each target; or the circle related to the target can be increased to enhance the exploration tendency if the distribution of the current generation of candidate samples is too concentrated on some targets; or the preference weights can be set to a fixed value based on the results of the previous iteration.

[0060] For example, preference weights are applied to each objective. The loss scalar is obtained by summing the N target loss change rates for each candidate sample. Its expression is:

[0061] Optionally, the candidate samples of the current generation are sorted according to the loss scalar, and the sorting result is combined with the globally optimal anchor value to generate the final loss scalar value for each candidate sample, including: Convert the loss scalars into loss ranking values ​​in ascending order; multiply the loss ranking values ​​by a scaling factor, and add the scaled loss ranking values ​​to the global optimal anchor value to obtain the final loss scalar value for each candidate sample.

[0062] Specifically, the loss scalar of λ candidate samples Converting to the loss sort value R(s_Li) in ascending order, its expression is:

[0063] in, This is an indicator function; its value is 1 when the condition is true and 0 when the condition is false. During the process of converting sorted values, if equal scalar loss values ​​appear, the sorted values ​​are incremented according to their original order of appearance.

[0064] The loss ranking value of each candidate sample and the global optimal anchor value of the current generation are combined to form the final loss scalar value Fi for each candidate sample participating in CMA-ES optimization, and its expression is:

[0065] in, This is the scaling factor, which, for example, takes a value of 0.001.

[0066] By employing the above scheme, the loss ranking value can be used to fix the numerical range while providing information on the relative magnitude of the loss among current generation candidate samples. Since the loss ranking value cannot provide the absolute loss value of the original objective, to avoid misjudging the convergence status of the optimization, the loss ranking value is multiplied by a scaling factor, and then added to the global optimal anchor value to obtain the final loss for each candidate sample. Because candidate samples of the same generation have the same global optimal anchor value, the quality of candidate samples within the same generation is entirely determined by the loss ranking value, rather than the absolute numerical difference, reducing noise sensitivity and improving the stability of the multi-objective optimization process. On the other hand, the scaled loss ranking value is much smaller than the global optimal anchor value, ensuring that the CMA-ES optimizer still uses the global optimal anchor value as the primary factor in determining whether the optimization has converged. This makes the convergence basis based on historical bests rather than the ranking of the same generation, avoiding the phenomenon of ranking changes without significant improvement under multi-objective variations, and improving the reliability of optimization progress monitoring.

[0067] Please see Figure 3 The following is combined Figure 4 The working steps of the CMA-ES-based multi-objective optimization method of this application are described in detail below: This application replaces the fitness metric used in traditional techniques with a loss metric. Specifically, for λ candidate samples sampled by a CMA-ES optimizer in the current generation, the loss values ​​of N objectives for each candidate sample are calculated. If the current generation is the first generation initially sampled, the loss statistic for each objective in the current generation is calculated, and the global optimal anchor value for each candidate sample is calculated. If the current generation is not the first generation, the global optimal anchor value is updated. The method for calculating the global optimal anchor value includes: for each candidate sample in the current generation, summing the loss values ​​of all objectives in that candidate sample to obtain a sum of multiple objective losses; determining the minimum objective loss sum of the current generation from the sums of objective losses; comparing the minimum objective loss sum of the current generation with the minimum objective loss sum recorded in the historical iterations, and taking the smaller value as the global optimal anchor value.

[0068] Furthermore, the loss change rate for each target is standardized sample by sample. Specifically, the magnitude of the loss change of each target relative to the loss statistic is calculated. Based on the loss values ​​of each candidate sample in the current generation on that target, the scale parameter of that target is determined, and a variance stability coefficient is superimposed on the scale parameter to obtain the standardized scale. The ratio of the magnitude of the loss change to the standardized scale is calculated to obtain the loss change rate of that target. Here, the loss statistic is the mean loss of each candidate sample in the previous generation on that target, or the global minimum loss value recorded for that target during historical iterations, and the scale parameter is the standard deviation of the loss of each candidate sample in the current generation on that target; or, the loss statistic is the median loss of each candidate sample in the previous generation on that target, or the global minimum loss value recorded for that target during historical iterations, and the scale parameter is the interquartile range of each candidate sample in the current generation on that target.

[0069] Furthermore, if the current generation is not the first generation sample, the loss statistic for each target is updated and used to calculate the loss change rate for each target in the next generation.

[0070] Furthermore, after obtaining the loss change rates of N targets in a candidate sample, the loss change rates of the N targets are summed to obtain the loss scalar of the candidate sample. In the summation process, a preference weight can be applied to the loss change rate of each target.

[0071] Furthermore, after obtaining the loss scalars of λ candidate samples, the loss scalars of λ candidate samples are converted into loss ranking values ​​in ascending order. Then, the loss ranking value of each candidate sample is multiplied by a scaling factor, and the global optimal anchor value of the target loss sum is added to obtain the final loss scalar value of that candidate sample.

[0072] Furthermore, the aforementioned final loss scalar value is provided to the CMA-ES optimizer so that the CMA-ES optimizer can complete the update and then enter the next generation of optimization loop.

[0073] Please refer to Table 1. Table 1 shows the generational loss changes of two objectives A and B with different loss scales, respectively, when the CMA-ES optimizer uses the traditional weighted and scalar methods and the multi-objective optimization method based on CMA-ES of this application. The loss values ​​in Table 1 are all from the best candidate samples in the corresponding generation. Wa represents the preference weight applied to objective A, and Wb represents the preference weight applied to objective B.

[0074] As shown in Table 1, the order of magnitude of the loss value for target A is 10, and the order of magnitude of the loss value for target B is 1, with the upper and lower bounds of both losses unknown. If the loss cannot be normalized, summing the two losses proportionally will cause the loss of target A to dominate the optimization process. Furthermore, if preference weights are to be applied based on traditional methods, these preference weights need to achieve two objectives: balancing the loss dimensions and guiding the optimization preference. Using the CMA-ES-based multi-objective optimization method provided in this application, when the preference weights of different objectives are equal, the standard CMA-ES optimizer can optimize different objectives in a balanced manner, while the applied preference weights only affect the optimization preference.

[0075] Table 1. Impact of different scalarization methods and optimization preference weights on the optimization process.

[0076] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0077] Based on the same inventive concept, this application also provides a CMA-ES-based multi-objective optimization system. This system is applicable to the above-mentioned CMA-ES-based multi-objective optimization method. The solution provided by this system is similar to the solution described in the above-mentioned method. Therefore, the specific limitations of one or more system embodiments provided below can be found in the limitations of the method above, and will not be repeated here.

[0078] Please see Figure 4 In one embodiment, the CMA-ES-based multi-objective optimization system includes a sampling module, a processing module, and an update module.

[0079] The sampling module is used to sample multiple candidate samples for the current generation using the CMA-ES optimizer, where each candidate sample corresponds to multiple targets.

[0080] The processing module is used to calculate the loss value of each target in each candidate sample, and determine the loss change rate of each target based on the loss statistics of the previous generation candidate samples on the corresponding targets; sum the loss change rates based on preset preference weights to obtain the loss scalar of the candidate sample; sort the candidate samples of the current generation according to the loss scalar, and combine the sorting result with the global optimal anchor value to generate the final loss scalar value of each candidate sample; wherein, the global optimal anchor value is determined based on the sum of the multi-target losses of each candidate sample in the iteration process.

[0081] The update module provides the final loss scalar value to the CMA-ES optimizer, enabling the CMA-ES optimizer to update its search distribution parameters based on the final loss scalar value to generate candidate samples for the next iteration.

[0082] Optionally, for each candidate sample, the processing module calculates the loss value of each target in the candidate sample and determines the loss change rate of each target based on the loss statistics of the previous generation candidate samples on the corresponding targets, including: for each target, determining the loss statistics of the target; calculating the loss change magnitude of the current generation candidate samples on the target relative to the loss statistics; determining the scale parameter of the target based on the loss values ​​of the current generation candidate samples on the target, and superimposing the variance stability coefficient on the scale parameter to obtain the standardized scale; calculating the ratio of the loss change magnitude to the standardized scale to obtain the loss change rate of the target.

[0083] Optionally, the loss statistic is the mean loss of each candidate sample in the previous generation on the target, or the global minimum loss value recorded by the target in the historical iteration process, and the scale parameter is the standard deviation of the loss of each candidate sample in the current generation on the target.

[0084] Optionally, the loss statistic is the median loss of each candidate sample in the previous generation on the target, or the global minimum loss value recorded by the target in the historical iteration process, and the scale parameter is the interquartile range of each candidate sample in the current generation on the target.

[0085] Optionally, the processing module is also used to determine the globally optimal anchor value based on the sum of the multi-target losses of each candidate sample during the iteration process. The method includes: for each candidate sample in the current generation, summing the loss values ​​of all targets in the candidate sample to obtain a sum of multiple target losses; determining the minimum target loss sum of the current generation from the sums of target losses; comparing the minimum target loss sum of the current generation with the minimum target loss sum recorded in the historical iteration process, and taking the smaller value as the globally optimal anchor value.

[0086] Optionally, the processing module is also used to preset preference weights, which includes: during the iterative optimization process, obtaining the numerical sequence of the optimization variables used to generate candidate samples within a preset time window; calculating the numerical stability index of the optimization variables based on the numerical sequence; and adaptively updating the preference weights corresponding to each objective according to the numerical stability index, so as to improve the preference weights corresponding to objectives whose numerical stability meets the preset conditions, and reduce the preference weights corresponding to objectives whose numerical stability does not meet the preset conditions.

[0087] Optionally, the processing module sorts the candidate samples of the current generation according to the loss scalar, and combines the sorting result with the global optimal anchor value to generate the final loss scalar value of each candidate sample, including: converting the loss scalar into a loss sorting value in ascending order; multiplying the loss sorting value by a scaling factor, and adding the scaled loss sorting value with the global optimal anchor value to obtain the final loss scalar value of each candidate sample.

[0088] The aforementioned CMA-ES-based multi-objective optimization system uses a CMA-ES optimizer to sample multiple candidate samples for the current generation and calculates the rate of change of loss for each objective. Based on preset preference weights, the loss rates of change are summed to obtain the loss scalar for each candidate sample. The candidate samples are then ranked according to their loss scalars, and the ranking result is combined with the global optimal anchor value to generate the final loss scalar value for each candidate sample. This final loss scalar value is provided to the CMA-ES optimizer, allowing it to update its search distribution parameters and generate candidate samples for the next iteration. Using this approach, the application calculates the rate of change of loss for each objective based on the previous generation's loss statistics and the current loss value. This dimensionless rate of change of loss ensures that subsequent applied preference weights are only used to adjust the optimization priority of different objectives, without needing to balance the order-of-magnitude differences between different optimization objectives. This significantly reduces the complexity of weight tuning and avoids single-objective dominance caused by differences in objective dimensions, thus improving the stability and controllability of multi-objective optimization. Furthermore, the loss scalar is converted into a loss ranking value. The range of the loss ranking value is uniquely determined by the number of samples in the current generation, and the CMA-ES optimizer is invariant to monotonic changes in loss. The above conversion operation obtains a definite range of loss values ​​without changing the original optimization objective. Further, the loss ranking value is combined with the global optimal anchor value to generate the final loss scalar value used for updating the CMA-ES optimizer. This allows samples of the same generation to share the same global optimal anchor value, thus the quality of samples in the same generation is entirely determined by the loss ranking value. This helps to more accurately determine the degree of improvement of candidate samples relative to historical observations and improves the reliability of the convergence criterion. Finally, by feeding the final loss scalar value back to a single CMA-ES optimizer to update the search distribution parameters, the computational and storage overhead of maintaining multiple optimizers can be avoided. Therefore, without changing the single-objective update mechanism of CMA-ES, an effective solution to multi-objective optimization problems can be achieved, significantly improving optimization efficiency and resource utilization. The modules in the aforementioned CMA-ES-based multi-objective optimization system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0089] In one feasible embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 5As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements the aforementioned CMA-ES-based multi-objective optimization method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0090] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0091] In one feasible embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method steps in the above-described CMA-ES-based multi-objective optimization method.

[0092] In one feasible embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the method steps in the above-described CMA-ES-based multi-objective optimization method.

[0093] In one feasible embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the method steps in the CMA-ES-based multi-objective optimization method described above.

[0094] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0095] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0096] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A multi-objective optimization method based on CMA-ES, characterized in that, The method includes: The CMA-ES optimizer is used to sample multiple candidate samples for the current generation, where each candidate sample corresponds to multiple targets; For each candidate sample, the loss value of each target in the candidate sample is calculated, and the loss change rate of each target is determined based on the loss statistics of the previous generation candidate samples on the corresponding targets; the loss change rates of each target are summed based on the preset preference weights to obtain the loss scalar of the candidate sample. The candidate samples in the current generation are sorted according to the loss scalar, and the sorting result and the global optimal anchor value are combined to generate the final loss scalar value of each candidate sample; wherein, the global optimal anchor value is determined based on the sum of the multi-objective losses of each candidate sample in the iteration process; The final loss scalar value is provided to the CMA-ES optimizer so that the CMA-ES optimizer updates its search distribution parameters based on the final loss scalar value and generates candidate samples for the next iteration.

2. The method according to claim 1, characterized in that, For each candidate sample, the loss value of each target in that candidate sample is calculated, and the loss change rate of each target is determined based on the loss statistics of previous candidate samples on the corresponding targets, including: For each objective, determine the loss statistic for that objective; Calculate the magnitude of the loss change of each candidate sample in the current generation on the target relative to the loss statistic; Based on the loss values ​​of each candidate sample in the current generation on the target, the scale parameter of the target is determined, and a variance stability coefficient is superimposed on the scale parameter to obtain a standardized scale. The ratio of the loss change magnitude to the standardized scale is calculated to obtain the loss change rate of the target.

3. The method according to claim 2, characterized in that: The loss statistic is the average loss of each candidate sample in the previous generation on the target, or the global minimum loss value recorded by the target in the historical iteration process, and the scale parameter is the standard deviation of the loss of each candidate sample in the current generation on the target.

4. The method according to claim 2, characterized in that: The loss statistic is the median loss of each candidate sample in the previous generation on the target, or the global minimum loss value recorded by the target in the historical iteration process, and the scale parameter is the interquartile range of each candidate sample in the current generation on the target.

5. The method according to claim 1, characterized in that, Methods for determining the globally optimal anchor value based on the sum of multi-objective losses of each candidate sample during the iteration process include: For each candidate sample in the current generation, the loss values ​​of all targets in the candidate sample are summed to obtain the total target loss; the minimum target loss of the current generation is determined from the total target loss. The minimum target loss sum of the current generation is compared with the minimum target loss sum recorded in the historical iterations, and the smaller value is taken as the global optimal anchor point value.

6. The method according to claim 5, characterized in that, The step of sorting the candidate samples of the current generation according to the loss scalar, and combining the sorting result with the global optimal anchor value to generate the final loss scalar value for each candidate sample, includes: Convert the loss scalar into loss sorted values ​​in ascending order; Multiply the loss ranking value by a scaling factor, and add the scaled loss ranking value to the global optimal anchor value to obtain the final loss scalar value for each candidate sample.

7. The method according to claim 1, characterized in that, Methods for presetting preference weights include: During the iterative optimization process, the numerical sequence of the optimization variables used to generate candidate samples within a preset time window is obtained; Calculate the numerical stability index of the optimization variable based on the numerical sequence; The preference weights corresponding to each target are adaptively updated based on the numerical stability index, so as to increase the preference weights corresponding to targets whose numerical stability meets the preset conditions, and decrease the preference weights corresponding to targets whose numerical stability does not meet the preset conditions.

8. A multi-objective optimization system based on CMA-ES, characterized in that, The system includes: The sampling module is used to sample multiple candidate samples of the current generation using the CMA-ES optimizer, where each candidate sample corresponds to multiple targets; The processing module is used to calculate the loss value of each target in each candidate sample, and determine the loss change rate of each target based on the loss statistics of the previous generation candidate samples on the corresponding targets; sum the loss change rates based on preset preference weights to obtain the loss scalar of the candidate sample; sort the candidate samples of the current generation according to the loss scalar, and combine the sorting result with the global optimal anchor value to generate the final loss scalar value of each candidate sample; wherein, the global optimal anchor value is determined based on the sum of the multi-target losses of each candidate sample in the iteration process; An update module is used to provide the final loss scalar value to the CMA-ES optimizer so that the CMA-ES optimizer updates its search distribution parameters based on the final loss scalar value to generate candidate samples for the next iteration.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-7.

Images