Multi-modal image segmentation and enhancement processing method

Abstract

The invention discloses a multi-modal image segmentation and enhancement processing method, and relates to the technical field of image processing. By constructing an image processing model, image enhancement and segmentation are established as a unified continuous process, a joint optimization objective function is constructed, and at least two optimization algorithms are adopted for parallel solving to obtain at least two groups of optimal segmentation parameters and optimal enhancement parameters corresponding to each optimization algorithm, so that the segmentation and enhancement of the image are realized. And based on the at least two groups of optimal segmentation parameters and optimal enhancement parameters, obtaining final segmentation parameters and enhancement parameters through a fusion strategy, processing the multi-modal target image by using the final segmentation parameters and enhancement parameters, and outputting a final enhanced image and a segmentation result. The multi-algorithm parallel and intelligent fusion strategy is utilized, local optimum is effectively avoided, the accuracy, stability and robustness of a processing result are remarkably improved, and the method is particularly suitable for high-quality processing requirement scenes such as multi-modal medical images.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for multimodal image segmentation and enhancement. Background Technology

[0002] Image segmentation and image enhancement are two core components of digital image processing, with wide applications in fields such as medical image analysis, remote sensing, autonomous driving, and industrial visual inspection. Image segmentation aims to divide an image into regions or objects with similar characteristics, thereby separating the target of interest; while image enhancement is used to improve the visual quality of an image, highlight key information, suppress noise, and provide higher-quality image data for subsequent recognition and analysis tasks.

[0003] In traditional image processing workflows, segmentation and enhancement are typically treated as two independent, sequential stages. A common approach is to first enhance the original image, such as through histogram equalization, contrast stretching, or filtering to improve image quality, and then perform segmentation on the enhanced image, using methods such as thresholding, region growing, active contour models, or graph theory-based approaches. However, this sequential processing model has inherent drawbacks: First, the parameter settings for the enhancement process often rely on prior knowledge or manual intervention, lacking co-optimization with the segmentation task. This can lead to over-enhancement introducing artifacts, or under-enhancement failing to effectively improve segmentation boundaries, ultimately resulting in decreased segmentation accuracy. Second, the quality of the segmentation results cannot guide the adjustment of enhancement parameters; the two processes are disconnected, making it difficult to achieve globally optimal processing results.

[0004] To overcome the aforementioned problems, some existing technologies have attempted to partially combine segmentation and enhancement. For example, some studies have proposed introducing local contrast enhancement during the iterative process of the segmentation algorithm, or considering image gradient information when constructing the segmentation energy function. However, most of these methods are local improvements within a fixed algorithmic framework and do not jointly optimize the parameters of segmentation and enhancement from a global perspective. They typically rely on a single optimization strategy (such as gradient descent) to solve for the model parameters. Such methods are prone to getting trapped in local optima, are sensitive to initial values, and lack stability and robustness.

[0005] Furthermore, in multimodal image processing, due to different imaging mechanisms and significant differences in image features, a single optimization algorithm is difficult to apply to all modalities, thus limiting the universality of such methods. Therefore, there is an urgent need in this field for a new method that can deeply couple image segmentation and enhancement, and stably and efficiently solve for globally optimal or near-optimal processing parameters, to overcome the limitations of existing serial processing modes and single optimization frameworks, thereby achieving more accurate and robust image analysis and processing results. To this end, a multimodal image segmentation and enhancement processing method is proposed. Summary of the Invention

[0006] The main objective of this invention is to provide a multimodal image segmentation and enhancement method. By combining multiple optimization algorithms and jointly optimizing the objective function, the method considers both the accuracy of segmentation and the quality of the enhanced image to solve for the optimal segmentation and enhancement parameters. This method can fully utilize the advantages of different optimization algorithms and effectively solve the problems in the background technology.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A multimodal image segmentation and enhancement method includes the following steps: S1: Acquire the multimodal target image to be processed; S2: Construct an image processing model, which defines image enhancement and image segmentation as a continuous process with enhancement parameters and segmentation parameters as optimization variables; S3: At least two different optimization algorithms are used to perform parallel optimization of the image processing model to obtain at least two sets of optimal segmentation parameters and optimal enhancement parameters corresponding to each optimization algorithm; S4: Based on the at least two sets of optimal segmentation parameters and optimal enhancement parameters, the final segmentation parameters and enhancement parameters are obtained through a fusion strategy; S5: Process the multimodal target image using the final segmentation and enhancement parameters, and output the final enhanced image and segmentation result.

[0008] A multimodal image segmentation and enhancement processing system, comprising: The image acquisition module is used to acquire the multimodal target image to be processed; The model building module is used to build an image processing model, which defines image enhancement and image segmentation as a continuous process with enhancement parameters and segmentation parameters as optimization variables; The parallel optimization solution module is used to perform parallel optimization solution on the image processing model using at least two different optimization algorithms to obtain at least two sets of optimal segmentation parameters and optimal enhancement parameters corresponding to each optimization algorithm. The fusion decision module is used to fuse the at least two sets of optimal segmentation parameters and optimal enhancement parameters using a weighted average method or game theory method to obtain the final segmentation parameters and enhancement parameters. The image processing output module is used to process the multimodal target image using the final segmentation parameters and enhancement parameters, and output the final enhanced image and segmentation result.

[0009] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of a multimodal image segmentation and enhancement processing method.

[0010] Furthermore, the objective function of the image processing model in step S2 is a joint objective function, the expression of which is: = +

[0011] in, For the joint objective function, Let be the image segmentation loss function. To split the parameter set, For image enhancement loss function, To enhance the parameter set, For the segmentation weight coefficients, To enhance the weighting coefficients, , All are constants within the interval (0,1), and + =1.

[0012] Furthermore, the image segmentation loss function is constructed based on the segmentation result and the Dice coefficient of the true label, and is expressed as: =

[0013] in, For the first Segmentation parameters for mid-modal images, For the first The input image of the intermediate modality, For the first Segmentation labels for mid-modal images, This is the Dice loss function.

[0014] Furthermore, the image enhancement loss function is constructed based on the structural similarity index measure SSIM, and is expressed as: =

[0015] in, For the first Enhancement parameters for mid-modal images, For the first The input image of the intermediate modality, This is the SSIM loss function.

[0016] Furthermore, the fusion strategy described in step S4 is a weighted average method, specifically including: Calculate the first Group Optimal Parameters , The corresponding joint objective function value ; Calculate the first based on the objective function value. Weights of the optimal parameters The calculation formula is: = ,in A scaling factor greater than zero. To optimize the number of algorithms; The final optimal parameters are obtained by taking a weighted average of the at least two sets of optimal parameters. , The calculation formulas are as follows: = , = .

[0017] Furthermore, the fusion strategy described in step S4 is a game theory method, specifically a fusion method based on Shapley values, which includes: Treat each set of optimal parameters as a game participant, and define a parameter to measure the alliance of any participant. Characteristic function of performance ; Calculate the first Shapley value of each participant The calculation formula is: =

[0018] in, The factorial symbol; According to the The Shapley value of each participant Calculate the first Weights of the optimal parameters The calculation formula is: = ; Based on the obtained weight The at least two sets of optimal parameters are weighted and fused to obtain the final optimal parameters. , The calculation formulas are as follows: = , = .

[0019] Furthermore, the at least two optimization algorithms mentioned in step S3 include a population-based global optimization algorithm and a gradient-based local optimization algorithm; wherein, the population-based global optimization algorithm is a particle swarm optimization algorithm, a genetic algorithm, or a differential evolution algorithm; and the gradient-based local optimization algorithm is a quasi-Newton method or a conjugate gradient method.

[0020] Furthermore, the multimodal target image includes images of at least two different modalities, each modality having different imaging principles and characteristics.

[0021] The present invention has the following beneficial effects: Compared with existing technologies, this solution breaks through the limitations of traditional serial processing by establishing a joint optimization model of segmentation and enhancement. This allows the enhancement process to directly serve the segmentation task, avoiding suboptimal results caused by independent processing and achieving synergistic optimization of the two stages.

[0022] Compared with existing technologies, this scheme employs a parallel solution strategy using multiple optimization algorithms, effectively overcoming the tendency of a single algorithm to get trapped in local optima. By combining the advantages of various algorithms through weighted averaging or game theory fusion methods, the accuracy of parameter solving and the robustness of the system are significantly improved.

[0023] Compared with existing technologies, this solution directly solves the segmentation and augmentation parameters through an optimized algorithm without relying on a large-scale labeled dataset, thereby reducing the computational resource consumption during the data preprocessing and model training stages.

[0024] Compared with existing technologies, this solution finds the most suitable segmentation and enhancement parameters for the current image conditions through continuous iterative optimization, and can resist the influence of noise, illumination changes and complex background light interference factors that may be contained in multimodal images to a certain extent.

[0025] Compared with existing technologies, this solution, through its modular system design, can flexibly adapt to different image modalities and processing requirements. It is particularly suitable for professional scenarios with high processing accuracy requirements, such as multimodal medical imaging, and has broad application value.

[0026] Compared with existing technologies, this solution requires no manual intervention in parameter adjustment throughout the entire processing process. It automatically finds the optimal solution through algorithms, which greatly reduces the difficulty of operation and the threshold for use, while ensuring the consistency and repeatability of the processing results. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating the multimodal image segmentation and enhancement processing method of the present invention; Figure 2 This is a schematic diagram of the structure of the multimodal image segmentation and enhancement processing system of the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0029] Example 1: See Figure 1 The flowchart of a multimodal image segmentation and enhancement method of the present invention shown herein includes the following steps: S1: Acquire the multimodal target image to be processed; S2: Construct an image processing model, which defines image enhancement and image segmentation as a continuous process with enhancement parameters and segmentation parameters as optimization variables; S3: Use at least two different optimization algorithms to perform parallel optimization of the image processing model to obtain at least two sets of optimal segmentation parameters and optimal enhancement parameters corresponding to each optimization algorithm; S4: Based on at least two sets of optimal segmentation parameters and optimal enhancement parameters, the final segmentation parameters and enhancement parameters are obtained through a fusion strategy; S5: Process the multimodal target image using the final segmentation and enhancement parameters, and output the final enhanced image and segmentation result.

[0030] The following are application examples of this invention in medical image processing and remote sensing image processing.

[0031] Application Example 1: Medical Image Segmentation and Enhancement Application Background: In medical image processing, multimodal images (such as CT and MRI) provide different tissue contrasts and anatomical information. By segmenting and enhancing these images, lesion areas can be identified more accurately, assisting doctors in diagnosis and treatment planning.

[0032] Specific process steps Step 1, Data Preparation Input Images: Acquire CT and MRI images, ensuring image alignment and registration.

[0033] Real labels: Prepare segmentation labels for CT and MRI images for training and validation of the segmentation model.

[0034] Step 2, Define the joint optimization objective function Objective function: = +

[0035] in, For the joint objective function, Let be the image segmentation loss function. To split the parameter set, For image enhancement loss function, To enhance the parameter set, For the segmentation weight coefficients, To enhance the weighting coefficients, , All are constants within the interval (0,1), and + =1.

[0036] Step 3, Define the image segmentation loss function The image segmentation loss function is constructed based on the Dice coefficients of the segmentation results and the ground truth labels, and is expressed as follows: =

[0037] in, For the first Segmentation parameters for mid-modal images, For the first The input image of the intermediate modality, For the first Segmentation labels for mid-modal images, The Dice loss function is as follows: = .

[0038] Step 4, Define the image enhancement loss function The image enhancement loss function is constructed based on the structural similarity index measure SSIM, and is expressed as: =

[0039] in, For the first Enhancement parameters for mid-modal images, For the first The input image of the intermediate modality, The SSIM loss function is as follows: =1- .

[0040] Step 5, Optimize Algorithm Selection The optimization algorithms include a population-based global optimization algorithm and a gradient-based local optimization algorithm; the population-based global optimization algorithm is a particle swarm optimization algorithm, a genetic algorithm, or a differential evolution algorithm; the gradient-based local optimization algorithm is a quasi-Newton method or a conjugate gradient method. In this application example, a genetic algorithm and a quasi-Newton method are selected to perform parallel optimization of the image processing model, wherein: The specific steps of a genetic algorithm include: Step 5-1.1), Initialization: Randomly generate the initial population. ; Step 5-1.2): Fitness assessment: Calculate the fitness of each individual. ; Step 5-1.3), selection operation: select individuals for breeding based on fitness; Step 5-1.4), crossover operation: Perform a crossover operation on the selected individuals to generate new individuals; Step 5-1.5): Mutation operation: Perform mutation operation on the newly generated individuals to increase the diversity of the population.

[0041] Step 5-1.6), Iteration: Repeat steps 5-1.1 to 5-1.5 until the maximum number of iterations is reached or the fitness converges.

[0042] The specific steps of the quasi-Newton method include: Step 5-1.1), Initialization Initial parameters: Select initial segmentation and enhancement parameters = ; Initial inverse Hessian matrix approximation: Choose a positive definite matrix Typically, an identity matrix can be chosen. .

[0043] Step 5-1.2), calculate the gradient. Calculate the objective function In the current parameters gradient The calculation formula is: = .

[0044] Step 5-1.3), update direction Calculate search direction , = .

[0045] Step 5-1.4), line search Step size selection: Determine a suitable step size using line search methods (such as the golden section method, backtracking line search, etc.). This causes the objective function value to decrease: = + .

[0046] Step 5-1.5), update the approximation of the inverse Hessian matrix. Difference vector: Calculate the difference vector and

[0047] = - ; = -

[0048] Inverse Hessian Matrix Update: Update the inverse Hessian matrix approximation using the BFGS formula. : = + - .

[0049] Step 5-1.6), convergence judgment Convergence condition: Check whether the convergence condition is met, such as the norm of the gradient being less than a certain threshold or the change in parameters being less than a certain threshold. If the convergence condition is met, stop the iteration; otherwise, return to step 5-1.2 to continue the iteration.

[0050] Step 6, optimize result fusion processing The optimal parameters obtained in step 5 can be obtained using any of the following fusion strategies. , Perform fusion processing.

[0051] Strategy 1, using a weighted average method, specifically includes: Calculate the first Group Optimal Parameters , The corresponding joint objective function value ; Calculate the first value based on the objective function value. Weights of the optimal parameters The calculation formula is: = ,in A scaling factor greater than zero. To optimize the number of algorithms; The final optimal parameters are obtained by taking a weighted average of at least two sets of optimal parameters. , The calculation formulas are as follows: = , = .

[0052] Strategy 2 employs a game theory approach, specifically a fusion method based on Shapley values, which includes: Treat each set of optimal parameters as a game participant, and define a parameter to measure the alliance of any participant. Characteristic function of performance ; Calculate the first Shapley value of each participant The calculation formula is: =

[0053] in, The factorial symbol; According to the Shapley value of each participant Calculate the first Weights of the optimal parameters The calculation formula is: = ; Based on the obtained weight We perform weighted fusion of at least two sets of optimal parameters to obtain the final optimal parameters. , The calculation formulas are as follows: = , = .

[0054] Step 7, Output the results Segmentation result: using the final segmentation parameters The input image is segmented to obtain the segmented image.

[0055] Enhancement Result: Using the final enhancement parameters The input image is enhanced to obtain the enhanced image.

[0056] Application effect: Segmentation accuracy: By combining multimodal information fusion and optimization algorithms, lesion areas can be identified more accurately, thus improving segmentation accuracy.

[0057] Enhancement effect: By optimizing the enhancement parameters, the visual quality of the image can be effectively improved, and the contrast and detail information of the image can be enhanced.

[0058] Robustness: Combining optimization algorithms can effectively resist interference from noise and complex backgrounds, thus improving the robustness of the algorithm.

[0059] Computational efficiency: Through the synergy of parallel computing and optimization algorithms, computational efficiency can be improved and processing time reduced.

[0060] Application Example 2: Remote Sensing Image Segmentation and Enhancement Application Background: In remote sensing monitoring, multimodal images (such as optical images and radar images) provide different information about the land surface. By segmenting and enhancing these images, land use types can be identified more accurately, and environmental changes can be monitored.

[0061] Specific process steps Step 1, Data Preparation Input Images: Acquire optical and radar images, ensuring image alignment and registration.

[0062] Real labels: Prepare segmentation labels for optical and radar images for training and validating the segmentation model.

[0063] Step 2, Define the joint optimization objective function Objective function: = +

[0064] in, For the joint objective function, Let be the image segmentation loss function. To split the parameter set, For image enhancement loss function, To enhance the parameter set, For the segmentation weight coefficients, To enhance the weighting coefficients, , All are constants within the interval (0,1), and + =1.

[0065] Step 3, Define the image segmentation loss function The image segmentation loss function is constructed based on the Dice coefficients of the segmentation results and the ground truth labels, and is expressed as follows: =

[0066] in, For the first Segmentation parameters for mid-modal images, For the first The input image of the intermediate modality, For the first Segmentation labels for mid-modal images, The Dice loss function is as follows: = .

[0067] Step 4, Define the image enhancement loss function The image enhancement loss function is constructed based on the structural similarity index measure SSIM, and is expressed as: =

[0068] in, For the first Enhancement parameters for mid-modal images, For the first The input image of the intermediate modality, The SSIM loss function is as follows: =1- .

[0069] Step 5, Optimize Algorithm Selection The optimization algorithms include a population-based global optimization algorithm and a gradient-based local optimization algorithm; the population-based global optimization algorithm is a particle swarm optimization algorithm, a genetic algorithm, or a differential evolution algorithm; the gradient-based local optimization algorithm is a quasi-Newton method or a conjugate gradient method. In this application example, the particle swarm optimization algorithm and the conjugate gradient method are selected to perform parallel optimization of the image processing model, wherein: The specific steps of particle swarm optimization include: Step 5-2.1), Initialization: Randomly generate an initial particle swarm {S(0),E(0)}.

[0070] Step 5-2.2), calculate the fitness J(S(t),E(t)) for each particle.

[0071] Step 5-2.3), velocity and position update: Update the velocity and position of the particles based on the individual and global optimal solutions.

[0072] Step 5-2.4), Iteration: Repeat steps 5-2.2 to 5-2.3 until the maximum number of iterations is reached or the fitness converges.

[0073] The specific steps of the conjugate gradient algorithm include: Step 5-2.1), Initialization Initial parameters: Select initial segmentation and enhancement parameters = ; Initial gradient: Calculate the objective function In initial parameters gradient ; Initial search direction: Select the initial search direction = .

[0074] Step 5-2.2), line search Step size selection: Determine a suitable step size using line search methods (such as the golden section method, backtracking line search, etc.). This causes the objective function value to decrease: = + .

[0075] Step 5-2.3), calculate the new gradient. Calculate the objective function In parameters gradient .

[0076] Step 5-2.4), update the search direction. Conjugate direction update: Update the search direction using the Polak-Ribière formula or the Fletcher-Reeves formula. : = +

[0077] = .

[0078] Step 5-2.5), convergence judgment Convergence condition: Check whether the convergence condition is met, such as the norm of the gradient being less than a certain threshold or the change in parameters being less than a certain threshold. If the convergence condition is met, stop the iteration; otherwise, return to step 5-1.2 to continue the iteration.

[0079] Step 6, optimize result fusion processing The optimal parameters obtained in step 5 can be obtained using any of the following fusion strategies. , Perform fusion processing.

[0080] Strategy 1, using a weighted average method, specifically includes: Calculate the first Group Optimal Parameters , The corresponding joint objective function value ; Calculate the first value based on the objective function value. Weights of the optimal parameters The calculation formula is: = ,in A scaling factor greater than zero. To optimize the number of algorithms; The final optimal parameters are obtained by taking a weighted average of at least two sets of optimal parameters. , The calculation formulas are as follows: = , = .

[0081] Strategy 2 employs a game theory approach, specifically a fusion method based on Shapley values, which includes: Treat each set of optimal parameters as a game participant, and define a parameter to measure the alliance of any participant. Characteristic function of performance ; Calculate the first Shapley value of each participant The calculation formula is: =

[0082] in, The factorial symbol; According to the Shapley value of each participant Calculate the first Weights of the optimal parameters The calculation formula is: = ; Based on the obtained weight We perform weighted fusion of at least two sets of optimal parameters to obtain the final optimal parameters. , The calculation formulas are as follows: = , = .

[0083] Step 7, Output the results Segmentation result: using the final segmentation parameters The input image is segmented to obtain the segmented image.

[0084] Enhancement Result: Using the final enhancement parameters The input image is enhanced to obtain the enhanced image.

[0085] Application effect: Segmentation accuracy: By combining multimodal information fusion and optimization algorithms, land use types can be identified more accurately, thus improving segmentation accuracy.

[0086] Enhancement effect: By optimizing the enhancement parameters, the visual quality of the image can be effectively improved, and the contrast and detail information of the image can be enhanced.

[0087] Robustness: Combining optimization algorithms can effectively resist interference from noise and complex backgrounds, thus improving the robustness of the algorithm.

[0088] Computational efficiency: Through the synergy of parallel computing and optimization algorithms, computational efficiency can be improved and processing time reduced.

[0089] Example 2: This invention also provides a multimodal image segmentation and enhancement processing system, the structural schematic of which is shown in the figure. Figure 2 ,include: The image acquisition module is used to acquire the multimodal target image to be processed; The model building module is used to build image processing models, which define image enhancement and image segmentation as continuous processes with enhancement parameters and segmentation parameters as optimization variables. The parallel optimization solution module is used to perform parallel optimization solution on the image processing model using at least two different optimization algorithms to obtain at least two sets of optimal segmentation parameters and optimal enhancement parameters corresponding to each optimization algorithm. The fusion decision module is used to fuse at least two sets of optimal segmentation parameters and optimal enhancement parameters using a weighted average method or game theory method to obtain the final segmentation parameters and enhancement parameters. The image processing output module is used to process the multimodal target image using the final segmentation and enhancement parameters, and output the final enhanced image and segmentation result.

[0090] Example 3: The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described multimodal image segmentation and enhancement processing method.

[0091] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A method of multi-modal image segmentation and enhancement processing, characterized in that, The method comprises the following steps: S1: obtaining a multi-modal target image to be processed; S2: constructing an image processing model, wherein the image processing model defines image enhancement and image segmentation as a continuous process with enhancement parameters and segmentation parameters as optimization variables; S3: using at least two different optimization algorithms to perform parallel optimization and solving on the image processing model to obtain at least two groups of optimal segmentation parameters and optimal enhancement parameters corresponding to each optimization algorithm; S4: performing fusion processing on the at least two groups of optimal segmentation parameters and optimal enhancement parameters through a fusion strategy to obtain final segmentation parameters and enhancement parameters; S5: processing the multi-modal target image using the final segmentation parameters and enhancement parameters to output a final enhanced image and segmentation result.

2. The multi-modal image segmentation and augmentation processing method of claim 1, wherein, The objective function of the image processing model in step S2 is a joint objective function, and its expression is: = + ； wherein, is a joint objective function, is an image segmentation loss function, is a set of segmentation parameters, is an image enhancement loss function, is a set of enhancement parameters, is a segmentation weight coefficient, is an enhancement weight coefficient, , are constants in the interval (0, 1), and + = 1.

3. The multi-modal image segmentation and augmentation processing method of claim 2, wherein, The image segmentation loss function is constructed based on the Dice coefficient of the segmentation result and the true label, and is expressed as: = ； wherein, is the first segmentation parameter for the image of the first modality, is the first input image of the first modality, is the first segmentation label for the image of the first modality, is the Dice loss function.

4. The multi-modal image segmentation and augmentation processing method of claim 2, wherein, The image enhancement loss function is constructed based on the structural similarity index measure SSIM, and is expressed as: = ； wherein, is the first enhancement parameter for the second modality image, is the first input image for the second modality, is the SSIM loss function.

5. The multi-modal image segmentation and augmentation processing method of claim 1, wherein, The fusion strategy in step S4 is a weighted average method, specifically including: Computing the first group of optimal parameters , corresponding joint objective function value ; The weight of the group optimal parameters is calculated according to the target function value The weight of the group optimal parameters is calculated according to the target function value The calculation formula is: = Wherein is a scaling factor greater than zero, is the number of optimization algorithms; weighting and averaging the at least two sets of optimal parameters to obtain final optimal parameters , , the calculation formulas are respectively: = , = .

6. The multi-modal image segmentation and augmentation processing method of claim 1, wherein, The fusion strategy in step S4 is a game theory method, specifically a fusion method based on Shapley value, specifically including: each set of optimal parameters is considered as a game player and a characteristic function is defined to measure the performance of any coalition of players ;​ The Sharpe value of the first participant is calculated The Sharpe value of the first participant is calculated​ = ； wherein is a factorial symbol; According to the Shapley value of the first participant , the weight of the first optimal parameter set is calculated , and the calculation formula is: = ; According to the acquired weight The at least two groups of optimal parameters are fused by weighting to obtain final optimal parameters 、 The calculation formulas are as follows: = , = .

7. The multi-modal image segmentation and augmentation processing method of claim 1, wherein, The at least two optimization algorithms in step S3 include a global optimization algorithm based on a population and a local optimization algorithm based on a gradient; wherein the global optimization algorithm based on the population is a particle swarm optimization algorithm, a genetic algorithm or a differential evolution algorithm; and the local optimization algorithm based on the gradient is a quasi-Newton method or a conjugate gradient method.

8. The multi-modal image segmentation and augmentation processing method of claim 1, wherein, The multi-modal target image includes at least two images of different modalities, and each modality of image has different imaging principles and characteristics.

9. A multi-modal image segmentation and enhancement processing system for implementing the multi-modal image segmentation and enhancement processing method according to any one of claims 1 to 8, characterized in that, The method comprises: An image acquisition module is configured to acquire a multi-modal target image to be processed; A model construction module is configured to construct an image processing model, wherein the image processing model defines image enhancement and image segmentation as a continuous process with enhancement parameters and segmentation parameters as optimization variables; A parallel optimization and solving module is configured to use at least two different optimization algorithms to perform parallel optimization and solving on the image processing model to obtain at least two groups of optimal segmentation parameters and optimal enhancement parameters corresponding to each optimization algorithm; A fusion decision module is configured to perform fusion through a weighted average method or a game theory method based on the at least two groups of optimal segmentation parameters and optimal enhancement parameters to obtain final segmentation parameters and enhancement parameters; An image processing output module is configured to process the multi-modal target image using the final segmentation parameters and enhancement parameters and output a final enhanced image and segmentation result.

10. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the multi-modal image segmentation and enhancement processing method of any one of claims 1-8.

Images