An underwater plankton image coloring method based on a diffusion model

Abstract

This invention relates to the field of underwater biological image enhancement technology, and in particular provides a method for colorizing underwater plankton images based on a diffusion model. The method includes: constructing an image latent variable extraction network based on a pre-trained stable diffusion model; building a low-frequency component suppression module based on the image latent variable extraction network; constructing an adaptive mask generation module based on the low-frequency component suppression module; constructing an image-text guided color information injection module, and connecting the image-text guided color information injection module with the image latent variable extraction network, the low-frequency component suppression module, and the adaptive mask generation module to construct a plankton colorization model; inferring a plankton detection model, and encapsulating and deploying the plankton model. This method achieves efficient colorization that integrates plankton semantic features and reference color information while maintaining realism and detail preservation, meeting the practical application needs of underwater ecological monitoring.

Description

Technical Field

[0001] This invention relates to the field of underwater biological image enhancement technology, and in particular to a method for colorizing underwater planktonic images based on a diffusion model. Background Technology

[0002] Underwater plankton images are a core data source for studying marine ecosystem structure and assessing aquatic environmental quality. Their color information directly affects the accuracy of plankton species identification, morphological feature extraction, and physiological state analysis. However, due to factors such as underwater light attenuation, water scattering / absorption effects, and limitations of imaging equipment, the actual plankton images often suffer from the following problems: First, most images are in grayscale mode, lacking color dimension information, resulting in low species identification efficiency; second, some color images suffer from color distortion, severe color cast (such as blue-green color cast), and blurred details, failing to truly reflect the natural color characteristics of plankton; third, existing colorization methods mostly rely on traditional image enhancement techniques or single-modal supervised learning models, making it difficult to balance color fidelity and detail preservation, and exhibiting poor color adaptability to different types of plankton (such as diatoms, dinoflagellates, and copepods).

[0003] In recent years, image generation and editing technologies based on diffusion models have made groundbreaking progress, and their powerful feature learning and generation capabilities have provided new solutions for image colorization tasks. For example, latent diffusion models such as stable diffusion models can achieve high-fidelity image colorization through text guidance and image condition constraints. However, existing diffusion model colorization methods have significant limitations in underwater plankton scenarios: on the one hand, they lack specific optimizations for the morphological characteristics and color properties of plankton, and the generated colors are prone to deviating from the natural attributes of the species (such as coloring transparent copepods with unnatural bright colors); on the other hand, they do not fully utilize the semantic information and reference color knowledge of plankton, resulting in poor coloring results in terms of species specificity and environmental adaptability, making it difficult to meet the needs of scientific research-level image analysis.

[0004] To address these issues, existing research has attempted to combine diffusion models with multimodal information for biological image colorization, but shortcomings remain. Some methods rely on a large number of paired grayscale-color plankton samples for fine-tuning, resulting in high data acquisition costs and limited generalization ability; other methods do not incorporate species semantic constraints and reference color guidance, leading to a lack of scientific rigor and consistency in the colorization results. Summary of the Invention

[0005] In view of this, the present invention provides a diffusion model-based underwater plankton image colorization method to achieve efficient colorization that integrates plankton semantic features and reference color information while maintaining realism and detail preservation, so as to meet the practical application needs in the field of underwater ecological monitoring.

[0006] In a first aspect, the present invention provides a method for colorizing underwater planktonic images based on a diffusion model, the method comprising:

[0007] Step S1: Construct an image latent variable extraction network based on a pre-trained stable diffusion model; Step S2: Construct a low-frequency component suppression module based on the image latent variable extraction network; Step S3: Based on the low-frequency component suppression module, construct an adaptive mask generation module; Step S4: Construct an image-text guided color information injection module, and connect the image-text guided color information injection module with the image latent variable extraction network, low-frequency component suppression module, and adaptive mask generation module to construct a plankton coloring model; Step S5: Reason the plankton detection model and encapsulate and deploy the plankton model.

[0008] Optionally, the image latent variable extraction network in step S1 includes an image feature extraction network and a text information extraction network; The image feature extraction network is used to backpropagate the grayscale image to the implicit noise space to extract latent variables, and it is composed of a U network; the text information extraction network is used to extract features of the grayscale image structure describing the text, and it includes 5 identical text prompt cross attention modules, a position encoder, 5 identical input instruction attention modules, and an output projection layer.

[0009] Optionally, the execution flow of the low-frequency component suppression module in step S2 includes: Step S21: Extract latent variables of grayscale images from the image feature extraction network. Perform a Fourier transform, where C represents the number of feature channels, H represents the height of the latent variable, and W represents the width of the latent variable; convert the spatial domain features into frequency domain features. Its expression is: ; in, FFT This represents a two-dimensional Fast Fourier Transform operation, used to implement the mapping operation from the spatial domain to the frequency domain; then, the frequency domain features are processed. Configure a high-pass filter and a corresponding low-pass filter, and extract the low-frequency components by element-wise multiplication. and high-frequency components Its expression is: ; ; in, This indicates element-wise multiplication; L represents a low-pass filter; and H represents a high-pass filter. Performing inverse fast Fourier transforms on the filtered low-frequency and high-frequency components respectively, mapping them back to the spatial domain, the expression is: ; ; Here, IFFT stands for Inverse Fast Fourier Transform. Represents low-frequency latent variables in the spatial domain. Represents high-frequency latent variables in the spatial domain; Step S22: Targeting low-frequency latent variables and high-frequency latent variables First, set the attenuation coefficient. ; for low-frequency latent variables To perform proportional attenuation, reducing its weight at the sampling starting point, the expression is: ; in, This represents the low-frequency latent variable after attenuation; Then generate and Gaussian noise of the same dimension and in accordance with The proportion is added to the attenuated low-frequency latent variable Its expression is: ; in, This indicates that the mean is 0 and the variance is 0. Gaussian noise is used to fill the feature gaps after low-frequency suppression; Finally, the noise-enhanced low-frequency latent variables are fused with the high-frequency latent variables to obtain the optimized sampling starting point latent variables. Its expression is: .

[0010] Optionally, the execution flow of the adaptive mask generation module in step S3 includes: Step S31: Calculate the pixel value of each pixel in the grayscale image and calculate the probability of each pixel value appearing in the entire grayscale image. For each different candidate threshold T, the foreground probability and background probability are first calculated, and their expressions are as follows: ; ; in, Represents the prospect probability. Let T represent the background probability and T represent the candidate threshold. i Indicates the index of the current pixel; Then, the foreground mean and background mean are calculated, and the global grayscale mean is obtained, which is expressed as: , ; , ; ; in, Indicates the mean of the foreground. This represents the background mean, and L is 255, representing all possible pixel values. This represents the global grayscale mean. The inter-class variance corresponding to the candidate threshold T is calculated using the following expression: ; in, This represents the inter-class variance of the corresponding candidate threshold T; the candidate threshold T with the largest inter-class variance value is selected as the optimal threshold. ; Step S32: Based on the optimal threshold Image segmentation, its expression is: ; in, Indicates the generated mask mask At coordinate point (x,y) The value, Indicates the original image at coordinate point (x,y) The value is obtained by the above process to get the mask. mask .

[0011] Optionally, the image-text guided color information injection module in step S4 includes a color feature extraction submodule and a directional fusion submodule; The color feature extraction submodule is used to extract the color features of the color reference image; the directional fusion submodule is used to combine the extracted color features with the sampling starting point latent variable. By fusing the data, latent variables containing color information are obtained, and the final colored image is inferred.

[0012] Optionally, the execution flow of the color feature extraction submodule includes: Using a pre-trained IP structure model Φ, a colored reference image I is received. s And color information extraction instruction prompts, outputting structured style features F style The style extraction command prompt is to extract the color characteristics and color distribution of an image, ignoring object outlines and spatial layout information. Its expression is: ; CLIP stands for Contrast Image Encoder, which acts as an image color feature extractor. The text embedding representing the color extraction instruction prompt is generated by a contrastive text encoder; MLP stands for Multilayer Perceptron, which consists of two fully connected layers and an intermediate GeLU activation function.

[0013] Optionally, the execution flow of the targeted fusion submodule includes: Step S41: Locate the first upsampling module in the stable diffusion model U network architecture as a style-specific injection layer. Its structure includes: Input feature dimension transformation layer: using a 3×3 convolutional kernel to transform the number of previous feature channels into... The convolution stride is 1, and the padding method is padding; the non-linear activation layer uses the GeLU activation function; the batch normalization layer normalizes the activated features; Step S42: Calculate the feature fusion weights. Based on the shading intensity requirements, calculate the color information fusion weights. The color information fusion weights are related to the number of diffusion sampling steps t, and the dynamically adjusted expression is: ; in, T The total number of sampling steps is set to 35; The initial fusion weight is set to 0.6; The final fusion weight is set to 0.2; Step S43: Targeted feature fusion, using a residual feature addition strategy to fuse style features. The expression for fusing with the intermediate features of the style-specific injection layer is as follows: ; in, The original features of the U-Net-style injection layer during t-step sampling; Norm is the feature normalization operation; Step S44: Post-processing of fused features. Perform a convolutional transformation to restore the feature dimensions required for the next level of U-Net. The expression is as follows: ; in, This represents a 3×3 convolution operation; After style feature injection, the colored plankton image is output.

[0014] In a second aspect, embodiments of the present invention provide a computer-readable storage medium comprising a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform the diffusion-based underwater plankton image colorization method in the first aspect or any possible implementation thereof.

[0015] Thirdly, embodiments of the present invention provide an electronic device, including: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the device, cause the device to perform the diffusion-model-based underwater plankton image colorization method in the first aspect or any possible implementation of the first aspect.

[0016] The technical solution provided by this invention includes a method that includes: constructing an image latent variable extraction network based on a pre-trained stable diffusion model; building a low-frequency component suppression module based on the image latent variable extraction network; constructing an adaptive mask generation module based on the low-frequency component suppression module; constructing an image-text guided color information injection module, and connecting the image-text guided color information injection module with the image latent variable extraction network, the low-frequency component suppression module, and the adaptive mask generation module to construct a plankton coloring model; inferring a plankton detection model, and encapsulating and deploying the plankton model. This method achieves efficient coloring that integrates plankton semantic features and reference color information while maintaining realism and detail preservation, meeting the practical application needs in the field of underwater ecological monitoring. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart of an underwater plankton image colorization method based on a diffusion model provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of the planktonic coloring model provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “the,” and “the” used in the embodiments of this invention are also intended to include the plural forms unless the context clearly indicates otherwise.

[0021] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0022] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0023] This invention provides a method for colorizing underwater planktonic images based on a diffusion model, such as... Figure 1 and Figure 2 As shown, the method includes: Step S1: Build an image latent variable extraction network based on a pre-trained stable diffusion model.

[0024] In this embodiment of the invention, the image latent variable extraction network in step S1 includes an image feature extraction network and a text information extraction network; The image feature extraction network is used to backpropagate the grayscale image to the implicit noise space to extract latent variables, and it is composed of a U network; the text information extraction network is used to extract features of the grayscale image structure describing the text, and it includes 5 identical text prompt cross attention modules, a position encoder, 5 identical input instruction attention modules, and an output projection layer.

[0025] Step S2: Construct a low-frequency component suppression module based on the image latent variable extraction network.

[0026] In this embodiment of the invention, the execution flow of the low-frequency component suppression module in step S2 includes: Step S21: Extract latent variables of grayscale images from the image feature extraction network. Perform a Fourier transform, where C represents the number of feature channels, H represents the height of the latent variable, and W represents the width of the latent variable; convert the spatial domain features into frequency domain features. Its expression is: ; in, FFT This represents a two-dimensional Fast Fourier Transform operation, used to implement the mapping operation from the spatial domain to the frequency domain; then, the frequency domain features are processed. Configure a high-pass filter and a corresponding low-pass filter, and extract the low-frequency components by element-wise multiplication. and high-frequency components Its expression is: ; ; in, This indicates element-wise multiplication; L represents a low-pass filter; and H represents a high-pass filter. Performing inverse fast Fourier transforms on the filtered low-frequency and high-frequency components respectively, mapping them back to the spatial domain, the expression is: ; ; Here, IFFT stands for Inverse Fast Fourier Transform. Represents low-frequency latent variables in the spatial domain. Represents high-frequency latent variables in the spatial domain; Step S22: Targeting low-frequency latent variables and high-frequency latent variables First, set the attenuation coefficient. ; for low-frequency latent variables To perform proportional attenuation, reducing its weight at the sampling starting point, the expression is: ; in, This represents the low-frequency latent variable after attenuation; Then generate and Gaussian noise of the same dimension and in accordance with The proportion is added to the attenuated low-frequency latent variable Its expression is: ; in, This indicates that the mean is 0 and the variance is 0. Gaussian noise is used to fill the feature gaps after low-frequency suppression; Finally, the noise-enhanced low-frequency latent variables are fused with the high-frequency latent variables to obtain the optimized sampling starting point latent variables. Its expression is: .

[0027] Step S3: Construct an adaptive mask generation module based on the low-frequency component suppression module.

[0028] In this embodiment of the invention, the execution flow of the adaptive mask generation module in step S3 includes: Step S31: Calculate the pixel value of each pixel in the grayscale image and calculate the probability of each pixel value appearing in the entire grayscale image. For each different candidate threshold T, the foreground probability and background probability are first calculated, and their expressions are as follows: ; ; in, Represents the prospect probability. Let T represent the background probability and T represent the candidate threshold. i Indicates the index of the current pixel; Then, the foreground mean and background mean are calculated, and the global grayscale mean is obtained, which is expressed as: , ; , ; ; in, Indicates the mean of the foreground. This represents the background mean, and L is 255, representing all possible pixel values. This represents the global grayscale mean. The inter-class variance corresponding to the candidate threshold T is calculated using the following expression: ; in, This represents the inter-class variance of the corresponding candidate threshold T; the candidate threshold T with the largest inter-class variance value is selected as the optimal threshold. ; Step S32: Based on the optimal threshold Image segmentation, its expression is: ; in, Indicates the generated mask maskAt coordinate point (x,y) The value, Indicates the original image at coordinate point (x,y) The value is obtained by the above process to get the mask. mask .

[0029] Step S4: Construct an image-text guided color information injection module, and connect the image-text guided color information injection module with the image latent variable extraction network, the low-frequency component suppression module, and the adaptive mask generation module to construct a plankton coloring model.

[0030] In this embodiment of the invention, the image-text guided color information injection module in step S4 includes a color feature extraction submodule and a directional fusion submodule; The color feature extraction submodule is used to extract the color features of the color reference image; the directional fusion submodule is used to combine the extracted color features with the sampling starting point latent variable. By fusing the data, latent variables containing color information are obtained, and the final colored image is inferred.

[0031] In this embodiment of the invention, the execution flow of the color feature extraction submodule includes: Using a pre-trained IP structure model Φ, a colored reference image I is received. s And color information extraction instruction prompts, outputting structured style features F style The style extraction command prompt is to extract the color characteristics and color distribution of an image, ignoring object outlines and spatial layout information. Its expression is: ; CLIP stands for Contrast Image Encoder, which acts as an image color feature extractor. The text embedding representing the color extraction instruction prompt is generated by a contrastive text encoder; MLP stands for Multilayer Perceptron, which consists of two fully connected layers and an intermediate GeLU activation function.

[0032] In this embodiment of the invention, the execution flow of the targeted fusion submodule includes: Step S41: Locate the first upsampling module in the stable diffusion model U network architecture as a style-specific injection layer. Its structure includes: Input feature dimension transformation layer: using a 3×3 convolutional kernel to transform the number of previous feature channels into... The convolution stride is 1, and the padding method is padding; the non-linear activation layer uses the GeLU activation function; the batch normalization layer normalizes the activated features; Step S42: Calculate the feature fusion weights. Based on the shading intensity requirements, calculate the color information fusion weights. The color information fusion weights are related to the number of diffusion sampling steps t, and the dynamically adjusted expression is: ; in, T The total number of sampling steps is set to 35; The initial fusion weight is set to 0.6; The final fusion weight is set to 0.2; Step S43: Targeted feature fusion, using a residual feature addition strategy to fuse style features. The expression for fusing with the intermediate features of the style-specific injection layer is as follows: ; in, The original features of the U-Net-style injection layer during t-step sampling; Norm is the feature normalization operation; Step S44: Post-processing of fused features. Perform a convolutional transformation to restore the feature dimensions required for the next level of U-Net. The expression is as follows: ; in, This represents a 3×3 convolution operation; After style feature injection, the colored plankton image is output.

[0033] Step S5: Reason the plankton detection model and encapsulate and deploy the plankton model.

[0034] The technical solution provided by this invention includes a method that includes: constructing an image latent variable extraction network based on a pre-trained stable diffusion model; building a low-frequency component suppression module based on the image latent variable extraction network; constructing an adaptive mask generation module based on the low-frequency component suppression module; constructing an image-text guided color information injection module, and connecting the image-text guided color information injection module with the image latent variable extraction network, the low-frequency component suppression module, and the adaptive mask generation module to construct a plankton coloring model; inferring a plankton detection model, and encapsulating and deploying the plankton model. This method achieves efficient coloring that integrates plankton semantic features and reference color information while maintaining realism and detail preservation, meeting the practical application needs in the field of underwater ecological monitoring.

[0035] The various steps in the embodiments of the present invention can be performed by an electronic device. This electronic device includes, but is not limited to, tablet computers, portable PCs, and desktop computers.

[0036] This invention provides a computer-readable storage medium including a stored program, wherein, when the program is running, it controls the electronic device containing the computer-readable storage medium to execute the above-described embodiment of the underwater plankton image colorization method based on a diffusion model.

[0037] Figure 3 A schematic diagram of an electronic device provided in an embodiment of the present invention, such as... Figure 3 As shown, the electronic device 21 includes a processor 211, a memory 212, and a computer program 213 stored in the memory 212 and executable on the processor 211. When the computer program 213 is executed by the processor 211, it implements the underwater plankton image coloring method based on the diffusion model in the embodiment. To avoid repetition, it will not be described in detail here.

[0038] Electronic device 21 includes, but is not limited to, processor 211 and memory 212. Those skilled in the art will understand that... Figure 3 This is merely an example of electronic device 21 and does not constitute a limitation on electronic device 21. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device may also include input / output devices, network access devices, buses, etc.

[0039] The processor 211 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0040] The memory 212 can be an internal storage unit of the electronic device 21, such as a hard disk or RAM of the electronic device 21. The memory 212 can also be an external storage device of the electronic device 21, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or FlashCard equipped on the electronic device 21. Furthermore, the memory 212 can include both internal and external storage units of the electronic device 21. The memory 212 is used to store computer programs and other programs and data required by network devices. The memory 212 can also be used to temporarily store data that has been output or will be output.

[0041] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0042] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A colorization method for underwater plankton images based on a diffusion model, characterized in that, The method includes: Step S1: Construct an image latent variable extraction network based on a pre-trained stable diffusion model; Step S2: Construct a low-frequency component suppression module based on the image latent variable extraction network; Step S3: Based on the low-frequency component suppression module, construct an adaptive mask generation module; Step S4: Construct an image-text guided color information injection module, and connect the image-text guided color information injection module with the image latent variable extraction network, low-frequency component suppression module, and adaptive mask generation module to construct a plankton coloring model; Step S5: Reason the plankton detection model and encapsulate and deploy the plankton model.

2. The method according to claim 1, characterized in that, The image latent variable extraction network in step S1 includes an image feature extraction network and a text information extraction network; The image feature extraction network is used to backpropagate the grayscale image to the implicit noise space to extract latent variables, and it is composed of a U network; the text information extraction network is used to extract features of the grayscale image structure describing the text, and it includes 5 identical text prompt cross attention modules, a position encoder, 5 identical input instruction attention modules, and an output projection layer.

3. The method according to claim 2, characterized in that, The execution flow of the low-frequency component suppression module in step S2 includes: Step S21: Extract latent variables of grayscale images from the image feature extraction network. Perform a Fourier transform, where C represents the number of feature channels, H represents the height of the latent variable, and W represents the width of the latent variable; convert the spatial domain features into frequency domain features. Its expression is: ； in, FFT This represents a two-dimensional Fast Fourier Transform operation, used to implement the mapping operation from the spatial domain to the frequency domain; then, the frequency domain features are processed. Configure a high-pass filter and a corresponding low-pass filter, and extract the low-frequency components by element-wise multiplication. and high-frequency components Its expression is: ； ； in, This indicates element-wise multiplication; L represents a low-pass filter; and H represents a high-pass filter. Performing inverse fast Fourier transforms on the filtered low-frequency and high-frequency components respectively, mapping them back to the spatial domain, the expression is: ； ； Here, IFFT stands for Inverse Fast Fourier Transform. Represents low-frequency latent variables in the spatial domain. Represents high-frequency latent variables in the spatial domain; Step S22: Targeting low-frequency latent variables and high-frequency latent variables First, set the attenuation coefficient. ; for low-frequency latent variables To perform proportional attenuation and reduce its weight at the sampling starting point, the expression is: ； in, This represents the low-frequency latent variable after attenuation; Then generate and Gaussian noise of the same dimension and in accordance with The proportion is added to the attenuated low-frequency latent variable Its expression is: ； in, This indicates that the mean is 0 and the variance is 0. Gaussian noise is used to fill the feature gaps after low-frequency suppression; Finally, the noise-enhanced low-frequency latent variables are fused with the high-frequency latent variables to obtain the optimized sampling starting point latent variables. Its expression is: 。 4. The method according to claim 3, characterized in that, The execution flow of the adaptive mask generation module in step S3 includes: Step S31: Calculate the pixel value of each pixel in the grayscale image and calculate the probability of each pixel value appearing in the entire grayscale image. For each different candidate threshold T, the foreground probability and background probability are first calculated, and their expressions are as follows: ； ； in, Represents the prospect probability. Let T represent the background probability and T represent the candidate threshold. i Indicates the index of the current pixel; Then, the foreground mean and background mean are calculated, and the global grayscale mean is obtained, which is expressed as: ， ； ， ； ； in, Indicates the mean of the foreground. This represents the background mean, and L is 255, representing all possible pixel values. This represents the global grayscale mean. The inter-class variance corresponding to the candidate threshold T is calculated using the following expression: ； in, This represents the inter-class variance of the corresponding candidate threshold T; the candidate threshold T with the largest inter-class variance value is selected as the optimal threshold. ; Step S32: Based on the optimal threshold Image segmentation, its expression is: ； in, Indicates the generated mask mask At coordinate point (x,y) The value, Indicates the original image at coordinate point (x,y) The value is obtained by the above process to get the mask. mask .

5. The method according to claim 4, characterized in that, The image-text guided color information injection module in step S4 includes a color feature extraction submodule and a directional fusion submodule; The color feature extraction submodule is used to extract the color features of the color reference image; The directional fusion submodule is used to combine the proposed color features with the sampling starting point latent variable. By fusing the data, latent variables containing color information are obtained, and the final colored image is inferred.

6. The method according to claim 5, characterized in that, The execution flow of the color feature extraction submodule includes: Using a pre-trained IP structure model Φ, a colored reference image I is received. s And color information extraction instruction prompts, outputting structured style features F style The style extraction command prompt is to extract the color characteristics and color distribution of an image, ignoring object outlines and spatial layout information. Its expression is: ； CLIP stands for Contrast Image Encoder, which acts as an image color feature extractor. The text embedding representing the color extraction instruction prompt is generated by a contrastive text encoder; MLP stands for Multilayer Perceptron, which consists of two fully connected layers and an intermediate GeLU activation function.

7. The method according to claim 5, characterized in that, The execution flow of the targeted fusion submodule includes: Step S41: Locate the first upsampling module in the stable diffusion model U network architecture as a style-specific injection layer. Its structure includes: Input feature dimension transformation layer: using a 3×3 convolutional kernel to transform the number of previous feature channels into... The convolution stride is 1, and the padding method is padding; the non-linear activation layer uses the GeLU activation function; the batch normalization layer normalizes the activated features; Step S42: Calculate the feature fusion weights. Based on the shading intensity requirements, calculate the color information fusion weights. The color information fusion weights are related to the number of diffusion sampling steps t, and the dynamically adjusted expression is: ； in, T The total number of sampling steps is set to 35. The initial fusion weight is set to 0.6; The final fusion weight is set to 0.2; Step S43: Targeted feature fusion, using a residual feature addition strategy to fuse style features. The expression for fusing with the intermediate features of the style-specific injection layer is as follows: ； in, The original features of the U-Net-style injection layer during t-step sampling; Norm is the feature normalization operation; Step S44: Post-processing of fused features. Perform a convolutional transformation to restore the feature dimensions required for the next level of U-Net. The expression is as follows: ； in, This represents a 3×3 convolution operation; After style feature injection, the colored plankton image is output.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device containing the computer-readable storage medium to perform the underwater plankton image colorization method based on any one of claims 1 to 7.

9. An electronic device, characterized in that, include: One or more processors; Memory; And one or more computer programs, wherein the one or more computer programs are stored in the memory, the one or more computer programs including instructions that, when executed by the device, cause the device to perform the diffusion-model-based underwater planktonic image colorization method according to any one of claims 1 to 7.

Images