Combination loss function search optimization method and system based on genetic algorithm

Abstract

The invention discloses a combined loss function search optimization method and system based on a genetic algorithm, and belongs to the technical field of computer vision. The search optimization method comprises the following steps: constructing a combined loss function L, and forming a combined loss parameter vector; non-negative constraint, normalization constraint and interval cutting constraint are applied, so that the combined loss parameter is mapped into a finite discrete feasible search space; performing outer-layer search by adopting a genetic algorithm to ensure that the optimal fitness in each generation of population is not inferior to that of the previous generation of population; a gradient optimization method is adopted in the inner layer to carry out training of a preset step number on a preset model, and a performance index is calculated to serve as fitness; repeating the above steps until a preset termination condition is met, and obtaining an optimal combined loss parameter vector; and carrying out complete training on the predetermined model. The system comprises a parameter coding module, a constraint mapping module, an outer layer search module, an inner layer training evaluation module and an output module. The method can improve the training stability and generalization performance of the model, and reduces the manual parameter adjustment cost.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and more particularly to a combined loss function search optimization method and system based on a genetic algorithm. BACKGROUND

[0002] With the wide application of deep learning in medical image segmentation, industrial defect detection, remote sensing target extraction and other scenarios, the loss function design in the model training process has become one of the key factors affecting the performance. In small sample, positive and negative sample imbalance, and difficult and easy sample imbalance, the existing model is highly sensitive to the loss function, and has problems such as difficult manual parameter tuning, unstable training process, and weak generalization ability.

[0003] To solve the above problems, the existing technology usually uses a combined loss function to combine basic losses such as cross-entropy, Dice loss, Focal loss, and Tversky loss according to certain weights. For example, the Chinese patent application with publication number CN115965780A discloses a retinal blood vessel thinning segmentation method based on a multi-loss function, which includes: obtaining a retinal blood vessel training sample, the retinal blood vessel training sample set containing multiple retinal blood vessel images with labeled blood vessel regions; constructing a retinal blood vessel segmentation model based on an image feature extraction encoder and a multi-interaction step-by-step thinning encoder; training the retinal blood vessel segmentation model using the retinal blood vessel training sample until the model converges to segment the blood vessel region from the actual retinal blood vessel image. Among them, the multi-loss function of the retinal blood vessel segmentation model is defined as L, and the expression of L is as follows: ；α k The weight index of each layer loss function is represented, and Σ represents the summation operation. The weight parameters of the combined loss function are usually set by human experience and are fixed, and the weight parameters of the loss function are not optimized during the training process.

[0004] The Chinese patent with publication number CN119203037A discloses a method for automatically searching for an optimal modal fusion strategy, which constructs a multi-task loss function and introduces a double-layer framework of inner-layer optimization and outer-layer optimization, optimizes the model parameters in the inner layer, and optimizes the fusion strategy parameters in the outer layer. It is applied to the robustness improvement of multi-modal Transformer in the modal missing scene. Although the patent uses an "inner-outer" optimization structure, the optimization object is the modal fusion strategy parameter, and the outer-layer optimization occurs in a continuous differentiable space, which relies on gradient or differentiable approximation methods, and is not suitable for non-differentiable or discrete loss configuration search. SUMMARY

[0005] This invention addresses the problems of existing combined loss function training under conditions of small sample size, imbalanced positive and negative samples, and imbalanced easy and difficult samples. It proposes a combined loss function search optimization method and system based on genetic algorithm.

[0006] The present invention adopts the following technical solution: The genetic algorithm-based combined loss function search optimization method includes the following steps: Step 1: Construct a combined loss function consisting of several basic loss functions. L The weight parameters and internal parameters of each basic loss function are used as optimization objects to form a combined loss parameter vector λ.

[0007] Step 2: Apply nonnegativity and normalization constraints to the weight parameters in the combined loss parameter vector λ, and apply interval pruning constraints to the internal parameters in the combined loss parameter vector λ, so that the combined loss parameters are mapped to a finite discrete feasible search space.

[0008] Step 3: Encode the combined loss parameter vector λ into individuals of the genetic algorithm, perform outer-layer search using the genetic algorithm in the feasible search space, and ensure that the optimal fitness in each generation is not inferior to that of the previous generation through the elite retention strategy.

[0009] Step 4: For each candidate combination loss parameter vector λ obtained from the outer layer search, the predetermined model is trained for a predetermined number of steps using a gradient optimization method with a fixed random seed and uniform initialization in the inner layer, and the performance index is calculated on the validation set as the fitness of the candidate combination loss parameter vector.

[0010] Step 5: Update the genetic population based on fitness, and repeat steps 3 and 4 until the preset termination condition is met to obtain the optimal combined loss parameter vector.

[0011] Step 6: Use the optimal combined loss parameter vector as the configuration of the combined loss function to train the predetermined model completely.

[0012] In a preferred embodiment, the aforementioned basic loss function includes at least two of the following: binary cross-entropy loss, Dice loss, Focal loss, and Tversky loss.

[0013] In a preferred implementation, the set of basic loss functions is set as follows: ,in, L i For the first i A basic loss function, θ These are the parameters of the model to be trained; φ i For the first iinternal parameters of the loss function; define the combined loss function as: wherein, λ is a combined loss parameter vector: ; w i is a weight parameter of the base loss function, .

[0014] In a preferred embodiment, the interval clipping constraint of step two above specifically includes: for each weight parameter of the base loss function and internal parameter of the loss function, respectively, setting the upper and lower bounds as: w i ∈[0, w max ], φ i ∈[ φ min , φ max ] and performing boundary clipping operation on the out-of-bound parameters after genetic algorithm crossover or mutation.

[0015] In a preferred embodiment, the outer search process of step three above includes: randomly generating a number of initial population individuals and performing constraint projection thereon; calling the inner training evaluation process to obtain the fitness value of each individual; selecting parent individuals according to fitness; performing crossover and recombination on the parent individual parameters to generate offspring; performing interval clipping and normalized projection after random disturbance on part of the parameters; directly retaining one or more individuals with the optimal fitness of the current generation to the next generation.

[0016] In a preferred embodiment, step four above specifically includes: when the given combined loss parameter vector λ is generated by the outer search of the inner ring, uses Adam to update K by steps to obtain : ; then calculates on the validation set and selects and breeds according to ; wherein, is a validation data set, is the number of samples in the validation data set, (x,y) represents the input sample x and its labeled label y, represents the average loss obtained by taking the arithmetic average of the combined loss function value on the validation data set, is the combined loss function value calculated on the sample (x,y).

[0017] The above combined loss function search optimization method based on genetic algorithm is applied to small sample unbalanced image semantic segmentation, including retinal blood vessel segmentation or industrial surface defect segmentation.

[0018] The application also provides a combined loss optimization system for implementing the search optimization method, comprising: a parameter encoding module, configured to encode weight parameters and internal parameters of a basic loss function into a combined loss parameter vector; a constraint mapping module, configured to impose non-negative constraints, normalization constraints and interval clipping constraints on the weight parameters and the internal parameters of the combined loss parameter vector, and map the combined loss parameter vector into a finite-scale feasible search space; an outer search module, configured to perform genetic algorithm outer search in the feasible search space, and perform selection, crossover, mutation and elite reservation; an inner training and evaluation module, configured to perform model training on each candidate combined loss parameter vector and calculate a verification fitness; and an output module, configured to update a population according to the fitness and output an optimal combined loss parameter vector.

[0019] As can be seen from the above description of the application, compared with the prior art, the application has the following advantages: 1. The application imposes structural constraints on the weight parameters and the internal parameters of the combined loss parameter vector, constructs a finite and controllable search space, and adopts a double-layer framework combining outer genetic search and inner fixed budget gradient optimization, thereby improving the training stability and generalization performance of the model without changing the network structure, and reducing the cost of manual parameter adjustment.

[0020] 2. The outer genetic algorithm can be used for non-differentiable and discretized loss configuration search, and does not depend on gradient and Gumbel-Softmax differentiable approximation; and considering the loss sensitivity and training stability under small sample data, the inner layer improves the reproducibility through fixed budget and unified initialization training mechanisms. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 The flowchart of the first embodiment of the application.

[0022] Figure 2 The system composition schematic diagram of the second embodiment of the application. DETAILED DESCRIPTION

[0023] The specific embodiments of the application will be described below with reference to the accompanying drawings. In order to fully understand the application, many details are described below, but the application can also be implemented without these details for those skilled in the art. For well-known components, methods and processes, the following will not be described in detail. EMBODIMENT

[0024] The present embodiment provides a combined loss function search optimization method based on genetic algorithm, referring to Figure 1 , comprising the following steps: S1, constructing a combined loss function composed of a plurality of basic loss functions Land the weight parameters and internal parameters of each base loss function are taken as optimization objects to form a combined loss parameter vector λ.

[0025] wherein the base loss functions include at least two of binary cross-entropy loss, Dice loss, Focal loss and Tversky loss. The set of base loss functions is set as , L i is the i-th base loss function, i is the j-th internal parameter of the i-th base loss function; and the combined loss function is defined as: θ φ i is the i-th base loss function, i is the j-th internal parameter of the i-th base loss function; and the combined loss function is defined as: , λ is the combined loss parameter vector: ; w i is the weight parameter of the base loss function, .

[0026] S2, non-negative constraints and normalization constraints are imposed on the weight parameters in the combined loss parameter vector λ, and interval clipping constraints are imposed on the internal parameters in the combined loss parameter vector λ, so that the combined loss parameters are mapped to a finite discrete feasible search space.

[0027] The non-negative constraint is specifically: w i ≥ 0, and the normalization constraint is: ∑ w i = 1. The interval clipping constraint is specifically: for the weight parameters and internal parameters of each base loss function, upper and lower bounds are set as: w i ∈ [0, w max ], φ i ∈ [ φ min , φ max ], and after crossover or mutation of the genetic algorithm, a boundary truncation operation is performed on the out-of-bound parameters.

[0028] S3, the combined loss parameter vector λ is encoded as an individual of the genetic algorithm, an outer search is performed in the feasible search space using the genetic algorithm, and an elite reservation strategy is used to ensure that the optimal fitness in each generation is not worse than that of the previous generation; the optimization objective of the outer layer is: . Wherein, is the fitness function calculated on the validation dataset, used to measure the pros and cons of the candidate combined loss parameter vector; is the fitness function calculated on the validation dataset, used to measure the pros and cons of the candidate combined loss parameter vector; ​optimal or near-optimal combination loss parameter vector corresponding to the minimum value; representing the value of the independent variable corresponding to the minimum value of the objective function; The value range of is limited in the feasible search space consisting of non-negative constraint, normalization constraint and interval clipping constraint.

[0029] The outer search process includes: randomly generating a number of initial population individuals and performing constraint projection on them; calling the inner training evaluation process to obtain the fitness value of each individual; selecting parent individuals according to fitness; performing cross-recombination on the parent individual parameters to generate offspring; performing interval clipping and normalization projection after random disturbance of part of the parameters; directly retaining one or more individuals with the optimal fitness of the current generation to the next generation.

[0030] S4, for each candidate combination loss parameter vector obtained by the outer search, the inner layer adopts a gradient optimization method with fixed random seed and unified initialization to train the predetermined model for a predetermined number of steps, and calculates the performance index on the validation set as the fitness of the candidate combination loss parameter vector. Specifically as follows: The inner ring starts from the given combination loss parameter vector generated by the outer ring search λ , uses Adam to update with the unified initialization K to obtain : ; then calculates the performance index on the validation set and selects and breeds according to it: ; wherein, is the validation data set, is the number of samples in the validation data set, (x, y) represents the input sample x and its labeled label y, represents the average loss obtained by taking the arithmetic average of the combination loss function value on the validation data set, is the combination loss function value calculated on the sample (x, y).

[0031] S5, update the genetic population according to the fitness, repeat steps S3 and S4 until the predetermined termination condition is met, and obtain the optimal combination loss parameter vector.

[0032] S6, use the optimal or near-optimal combination loss parameter vector as the combination loss function configuration to train the predetermined model completely.

[0033] The following is the application of the combination loss function search optimization method based on genetic algorithm in the task of retinal vessel segmentation.

[0034] 1. Base loss function selection: In retinal vessel segmentation, the binary cross-entropy loss (BCE), Dice loss, Focal loss and Tversky loss are selected as the base loss functions, i.e., m = 4.

[0035] 2. Parameter vector construction: The weight vector w [ w 1, w 2, w 3, w 4] is constructed; the internal parameters include the positive class weight parameter η (for class imbalance weighting) contained in the binary cross-entropy loss, the smoothing factor of the Dice loss, the focusing parameter γ of the Focal loss, the class balance parameter , and the and of the Tversky loss. The internal parameters are uniformly denoted as φ , and the weight vector φ is denoted as w . The weight vector φ and the internal parameters λ are concatenated into the overall vector .

[0036] 3. Constraint setting: It is required that w i ≥ 0, and w i = 1; it is agreed that w i ∈[0,1] and η∈[ , ] (for class imbalance weighting), preferably η∈[0.5,20]; γ ∈[0,5] and the smoothing factor , where , preferably ∈[, ], ∈[0,1], ∈[0,1] and ∈[0,1]; after genetic operation, the out-of-bound parameters are boundary-truncated and normalized-projected.

[0037] 4. Outer genetic search configuration: The population size is set to N (preferably 20-50); the iteration number G (preferably 20-50 generations); tournament selection, single-point or uniform crossover and Gaussian perturbation mutation are adopted, and a certain number of individuals with the highest fitness (such as 1-2) are retained in each generation.

[0038] 5、Inner training evaluation: using a fixed structure of U-Net or custom vascular segmentation network; using a fixed random seed to initialize network parameters and data loading order; using Adam optimizer to train under each candidate configuration K Step (as 5~10) as evaluation budget; calculate the value on the validation set to verify the average loss of the validation set As fitness.

[0039] 6、Results: after the genetic search is completed, the combination loss configuration with the highest fitness is selected, and complete training is performed on the same training data. Embodiments

[0040] A combination loss optimization system for implementing the above search optimization method is provided, referring to Figure 2 , comprising: A parameter encoding module 10 for encoding the weight parameters and internal parameters of the basic loss function into a combination loss parameter vector.

[0041] A constraint mapping module 20 for applying non-negative constraints, normalization constraints and interval clipping constraints to the weight parameters and internal parameters of the combination loss parameter vector, and mapping to a finite scale feasible search space.

[0042] An outer search module 30 for performing genetic algorithm outer search in the feasible search space, and performing selection, crossover, mutation and elite reservation.

[0043] An inner training evaluation module 40 for model training and validation fitness calculation for each candidate combination loss parameter vector λ.

[0044] An output module 50 for updating the population according to the fitness and outputting the optimal combination loss parameter vector.

[0045] The above is only a specific embodiment of the present application, but the design concept of the present application is not limited to this, and any non-essential modification of the present application using this concept should be considered as an infringement of the protection scope of the present application.

Claims

1. A combined loss function search and optimization method based on genetic algorithm, characterized in that, Includes the following steps: Step 1: Construct a combined loss function consisting of several basic loss functions. L The weight parameters and internal parameters of each basic loss function are used as optimization objects to form a combined loss parameter vector λ. Step 2: Apply nonnegativity and normalization constraints to the weight parameters in the combined loss parameter vector λ, and apply interval pruning constraints to the internal parameters in the combined loss parameter vector λ, so that the combined loss parameters are mapped to a finite discrete feasible search space. Step 3: Encode the combined loss parameter vector λ into individuals of the genetic algorithm, perform outer search using the genetic algorithm in the feasible search space, and ensure that the optimal fitness in each generation is not inferior to that of the previous generation through the elite retention strategy. Step 4: For each candidate combination loss parameter vector λ obtained from the outer layer search, the predetermined model is trained for a predetermined number of steps using a gradient optimization method with a fixed random seed and uniform initialization in the inner layer, and the performance index is calculated on the validation set as the fitness of the candidate combination loss parameter vector. Step 5: Update the genetic population based on fitness, and repeat steps 3 and 4 until the preset termination condition is met to obtain the optimal combined loss parameter vector; Step 6: Use the optimal combined loss parameter vector as the configuration of the combined loss function to train the predetermined model completely.

2. The combined loss function search and optimization method based on genetic algorithm as described in claim 1, characterized in that: The basic loss function includes at least two of the following: binary cross-entropy loss, Dice loss, Focal loss, and Tversky loss.

3. The combined loss function search and optimization method based on genetic algorithm as described in claim 1, characterized in that: Define the set of basic loss functions as follows ,in, L i For the first i A basic loss function, θ These are the parameters of the model to be trained; φ i For the first i The internal parameters of each loss function are defined; the combined loss function is defined as: ,in, λ The combined loss parameter vector: ; w i The weight parameters of the basic loss function, .

4. The combined loss function search and optimization method based on genetic algorithm as described in claim 3, characterized in that, The interval pruning constraint in step two specifically includes: setting upper and lower bounds for the weight parameters and internal parameters of each basic loss function. w i ∈[0, w max ], φ i ∈[ φ min , φ max Furthermore, after crossover or mutation in the genetic algorithm, a boundary truncation operation is performed on parameters that exceed the bounds.

5. The combined loss function search and optimization method based on genetic algorithm as described in claim 3, characterized in that, The outer search process in step three includes: randomly generating a number of initial population individuals and performing constraint projection on them; calling the inner training and evaluation process to obtain the fitness value of each individual; selecting parent individuals based on fitness; performing cross-recombination on the parameters of the parent individuals to generate offspring; performing interval pruning and normalized projection after randomly perturbing some parameters; and directly retaining one or more individuals with the best fitness in the current generation to the next generation.

6. The combined loss function search and optimization method based on genetic algorithm as described in claim 3, characterized in that, Step four specifically includes: the given combined loss parameter vector generated by the inner loop searching the outer loop. λ At that time, from unified initialization Let's go, update with Adam. K Step to get : Then compute on the validation set. And based on this, they select and reproduce: ,in, To validate the dataset, Let (x, y) represent the number of samples in the validation dataset, and (x, y) represent the input sample x and its labeled label y. This represents the average loss obtained by taking the arithmetic mean of the combined loss function values ​​on the validation dataset. This is the combined loss function value calculated on the sample (x,y).

7. The combined loss function search and optimization method based on genetic algorithm as described in claim 1, characterized in that: It is applied to image semantic segmentation with small sample imbalance, including retinal vessel segmentation or industrial surface defect segmentation.

8. A system for implementing a combination loss optimization method as described in any one of claims 1-7, characterized in that, include: The parameter encoding module is used to encode the weight parameters and internal parameters of the basic loss function into a combined loss parameter vector; The constraint mapping module is used to apply nonnegativity constraints, normalization constraints, and interval pruning constraints to the weight parameters and intrinsic parameters of the combined loss parameter vector, and map them into a finite-scale feasible search space. The outer search module is used to perform outer search of the genetic algorithm within the feasible search space, and to perform selection, crossover, mutation, and elite retention. The inner training and evaluation module is used to train the model on each candidate combination loss parameter vector λ and calculate the validation fitness. The output module updates the population based on fitness and outputs the optimal combination loss parameter vector.

Images