Ultrasonic image quality enhancement and segmentation joint optimization method

Abstract

The invention discloses a joint optimization method for ultrasonic image quality enhancement and segmentation, and belongs to the technical field of ultrasonic image quality optimization. Comprising the following steps: S1, constructing an image processing model; s2, calculating through the third ultrasonic image and the second ultrasonic image to obtain image enhancement loss; s3, inputting the third ultrasonic image into an image segmentation module to obtain a segmented image, and calculating segmentation loss through the segmented image and the marked image; s4, calculating a loss function based on the image enhancement loss and the segmentation loss; and S5, training the image processing model through the loss function, and jointly optimizing the image enhancement module and the image segmentation module. The technical scheme has the beneficial effects that the image quality enhancement and the image segmentation are jointly optimized, the image segmentation is taken as a target, the ultrasonic image quality is improved, the deviation of the segmentation module caused by noise is reduced, and a more accurate image segmentation result is obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of ultrasonic image quality optimization, and in particular to a joint optimization method for ultrasonic image quality enhancement and segmentation. Background Art

[0002] Handheld ultrasound devices are widely used in various medical scenarios that require rapid diagnosis because of their portability and ease of use. However, compared with large high-end ultrasound devices, the image resolution and quality collected by handheld ultrasound devices may be lower. Moreover, handheld ultrasound devices are usually equipped with fewer types and specifications of probes, which cannot meet all clinical needs. In addition, due to the miniaturization of the equipment, the performance of the handheld ultrasound processor is limited, making the image processing (such as denoising, 3D reconstruction, etc.) speed unable to meet diagnostic requirements. Therefore, most of the existing equipment and systems adopt a combination of edge and cloud, implementing simple image processing tasks on the edge and completing complex diagnostic tasks on the cloud.

[0003] In the existing technology, the image resolution of handheld ultrasound is low and the noise is high due to the limitation of the equipment. When the image is transmitted to the cloud for subsequent diagnostic tasks such as organ segmentation or lesion identification, the segmentation or identification algorithm is easily misled by noise, resulting in low accuracy of the result. In addition, the use of methods such as generative adversarial networks or super-resolution to improve image quality will introduce distortion or aliasing, so that the scanned information cannot be truly observed, resulting in missed diagnosis or misdiagnosis. Summary of the invention

[0004] The purpose of the present invention is to provide a joint optimization method for ultrasound image quality enhancement and segmentation to solve the above technical problems;

[0005] A joint optimization method for ultrasound image quality enhancement and segmentation, comprising:

[0006] Step S1, constructing an image processing model, wherein the image processing model includes an image enhancement module and an image segmentation module;

[0007] Step S2, processing the first ultrasonic image through the image enhancement module to obtain a third ultrasonic image, and calculating the image enhancement loss through the third ultrasonic image and the second ultrasonic image;

[0008] Step S3, inputting the third ultrasound image into the image segmentation module to obtain a segmented image, and calculating the segmentation loss through the segmented image and the annotated image;

[0009] Step S4, calculating a loss function based on the image enhancement loss and the segmentation loss;

[0010] Step S5: training the image processing model through the loss function, and jointly optimizing the image enhancement module and the image segmentation module.

[0011] Preferably, the calculation formula for the image enhancement loss in step S2 is:

[0012]

[0013] Among them, L enhance represents the image enhancement loss, I′ represents the second ultrasound image, represents the third ultrasound image.

[0014] Preferably, the calculation formula for the segmentation loss in step S3 is:

[0015] L seg =f(M′-M);

[0016] Among them, L seg represents the segmentation loss, f represents the segmentation loss function, M′ represents the labeled image, and M represents the segmented image.

[0017] Preferably, the loss function in step S4 is calculated as:

[0018] L=L seg +α×L enhance ;

[0019] Wherein, L represents the loss function, L seg represents the segmentation loss, α represents the weight coefficient, L enhance represents the image enhancement loss.

[0020] Preferably, in step S2, the first ultrasound image is acquired by a handheld ultrasound device.

[0021] Preferably, the image quality of the first ultrasonic image is lower than that of the second ultrasonic image, and the image quality of the first ultrasonic image is lower than that of the third ultrasonic image.

[0022] Preferably, in step S2, the first ultrasonic image is obtained by adding noise, style transfer, and blurring processing to the second ultrasonic image.

[0023] Preferably, in step S2, the second ultrasound image is an expected image of the image enhancement module, and the second ultrasound image is obtained by a large-scale ultrasound device.

[0024] Preferably, the image segmentation module is a probabilistic generative adversarial network or a U-type network.

[0025] Preferably, the image enhancement module includes a denoising submodule and a resolution enhancement submodule.

[0026] The beneficial effects of the present invention are: jointly optimizing image quality enhancement and image segmentation, taking image segmentation as the goal, improving the quality of ultrasound images, reducing the deviation of the segmentation module caused by noise, and obtaining a more accurate image segmentation result. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a step diagram of the joint optimization method for ultrasound image quality enhancement and segmentation of the present invention;

[0028] Figure 2 is a structural diagram of the joint optimization method for ultrasound image quality enhancement and segmentation of the present invention;

[0029] Figure 3 It is a structural diagram of the joint optimization method of the present invention taking carotid artery plaque segmentation as an example. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0032] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0033] A joint optimization method for ultrasound image quality enhancement and segmentation, such as Figure 1 , Figure 2 As shown, including

[0034] Step S1, constructing an image processing model, the image processing model includes an image enhancement module 1 and an image segmentation module 2;

[0035] Step S2, processing the first ultrasonic image through the image enhancement module 1 to obtain a third ultrasonic image, and calculating the image enhancement loss through the third ultrasonic image and the second ultrasonic image;

[0036] Step S3, inputting the third ultrasound image into the image segmentation module 2 to obtain a segmented image, and calculating the segmentation loss through the segmented image and the labeled image;

[0037] Step S4, calculating a loss function based on the image enhancement loss and the segmentation loss;

[0038] Step S5, training the image processing model through the loss function, and jointly optimizing the image enhancement module 1 and the image segmentation module 2.

[0039] Specifically, the present invention provides a joint optimization method for ultrasound image quality enhancement and segmentation, which is used for image optimization of handheld ultrasound equipment. Image quality enhancement and image segmentation are jointly optimized based on an artificial intelligence model, with image segmentation as the goal, reducing the deviation of the segmentation module caused by noise, improving the quality of ultrasound images, and obtaining more accurate image segmentation results.

[0040] In a preferred embodiment, the calculation formula of the image enhancement loss in step S2 is:

[0041]

[0042] Among them, L enhance represents the image enhancement loss, I′ represents the second ultrasound image, represents a third ultrasound image.

[0043] Specifically, the image enhancement loss quantifies the effect of the image enhancement module 1 by calculating the difference between the second ultrasound image and the third ultrasound image processed by the image enhancement module. This enables the system to evaluate and optimize the performance of the image enhancement module, ensuring that the processed image is closer to the standard second ultrasound image, thereby improving the visual quality and usability of the image.

[0044] Specifically, the overall image quality is improved, and important image features and details are enhanced. The reliability and usability of ultrasound images in clinical applications are improved. The optimized images are easier to interpret and analyze.

[0045] In a preferred embodiment, the calculation formula for the segmentation loss in step S3 is:

[0046] L seg =f(M′-M);

[0047] Among them, L seg represents the segmentation loss, f represents the segmentation loss function, M′ represents the labeled image, and M represents the segmented image.

[0048] Specifically, the calculation of segmentation loss can directly reflect the difference between the segmented image and the annotated image. The optimization system can guide the image segmentation model to learn how to more accurately distinguish different structures and tissues in the image. This helps to improve the segmentation algorithm's ability to recognize various complex structures in medical images.

[0049] More specifically, it ensures that the image enhancement module 1 and the image segmentation module 2 can simultaneously focus on improving image quality and improving segmentation accuracy during training, thereby improving the stability and reliability of the model in practical applications.

[0050] In a preferred embodiment, the loss function in step S4 is calculated as:

[0051] L=L seg +α×L enhance ;

[0052] Among them, L represents the loss function, L seg represents the segmentation loss, α represents the weight coefficient, L enhance represents the image enhancement loss.

[0053] Specifically, the loss function combines segmentation loss and image enhancement loss together, and controls their relative importance in the optimization process through weight coefficients. This enables the system to not only improve accuracy in segmentation tasks, but also ensure the improvement of image quality after enhancement. The design of the overall loss function enables the system to strike a balance between multiple objectives, improving the global performance of the system.

[0054] Specifically, by jointly optimizing the segmentation loss and the image enhancement loss, the system is able to simultaneously improve the accuracy of the segmentation results and the visual effect of the image quality. Optimizing the segmentation loss and the image enhancement loss together helps to enhance the robustness of the entire image processing module. Even in complex medical images, the system is able to effectively process and improve the image quality while maintaining accurate segmentation results.

[0055] In a preferred embodiment, the first ultrasound image in step S2 is acquired by a handheld ultrasound device.

[0056] Specifically, the first ultrasound image is a low-quality ultrasound image obtained by scanning with a handheld ultrasound device. After being processed by the image enhancement module 1, the noise can be effectively reduced, and the image contrast, clarity and details can be enhanced, so that the quality of the third ultrasound image is significantly improved, which helps to more accurately identify the anatomical structure or lesion area.

[0057] In a preferred embodiment, the image quality of the first ultrasonic image is lower than that of the second ultrasonic image, and the image quality of the first ultrasonic image is lower than that of the third ultrasonic image.

[0058] Specifically, the image quality of the first ultrasound image is low, and it can be processed into a third ultrasound image through the image enhancement module 1 to improve the visual quality and information richness of the image. The quality of the third ultrasound image is significantly higher than that of the first ultrasound image, which has an important impact on the subsequent image segmentation task. The purpose of the image enhancement module 1 is to enhance the contrast, clarity and details of the image without adding noise or distortion, so that the subsequent image segmentation is more accurate and reliable.

[0059] In a preferred embodiment, the first ultrasonic image in step S2 is obtained by adding noise, style transfer, and blurring to the second ultrasonic image.

[0060] Specifically, the first ultrasound image is training data obtained by adding noise, style transfer, and blurring to the second ultrasound image. Adding noise, style transfer, and blurring can expand the diversity of the training data set and make the image processing model more generalizable. The model is exposed to ultrasound images of different styles and qualities during the training process, which helps it learn a wider range of features and image processing strategies, thereby improving the actual application effect.

[0061] More specifically, adding noise means introducing some random pixel values ​​into the image to simulate the noise or signal interference that may exist in the real world. Common noise adding methods include Gaussian noise (by adding random numbers that obey Gaussian distribution), salt and pepper noise (randomly setting pixel values ​​to 0 or 255), etc. Style transfer is a technology that applies the style features of one image to another image. Style transfer can make the first ultrasound image visually have certain artistic style or structural features of the second ultrasound image. Including the use of pre-trained deep learning models, such as neural style transfer networks. Blurring is to reduce the details and clarity of the image, making the image blurred or soft. Including Gaussian blur, mean blur, motion blur, etc.

[0062] In a preferred embodiment, the second ultrasonic image in step S2 is an expected image of the image enhancement module 1, and the second ultrasonic image is obtained by a large ultrasonic device.

[0063] Specifically, the large-scale ultrasound equipment is a high-precision imaging device, and the quality and characteristics of the generated second ultrasound image as the expected image are the standards that the image enhancement module 1 should reach, so that the output image is closer to the high-quality, high-resolution second ultrasound image, and the quality and usability of the image in subsequent image segmentation tasks are improved.

[0064] In a preferred embodiment, the image segmentation module 2 is a probabilistic generative adversarial network (SAM) or a U-type network (UNet).

[0065] Specifically, SAM and UNet can segment images at the pixel level and produce fine segmentation results. UNet can use the feature pyramid structure to gradually extract and integrate feature information of different scales from shallow to deep layers, which helps to understand the image content more comprehensively and perform accurate segmentation. SAM can not only generate detailed segmentation results, but also generate more realistic and lifelike images through adversarial training, which can improve the performance of the segmentation model.

[0066] Specifically, it supports end-to-end training and can directly perform joint optimization from the original image to the segmentation result, which helps to improve the overall performance and generalization ability of the model, while simplifying the training process and parameter tuning. It can be applied to various medical image segmentation tasks, such as tumor detection, organ segmentation, and vascular analysis.

[0067] In a preferred embodiment, the image enhancement module 1 includes a denoising submodule and a resolution enhancement submodule.

[0068] Specifically, handheld ultrasound devices often generate noise in images due to their small size and sensor limitations. Image enhancement module 1 is specifically designed to remove this noise, thereby improving the clarity and quality of the image. The resolution of handheld ultrasound images is usually affected by the hardware limitations of the device and may not meet the standards for high-resolution imaging. Image enhancement module 1 uses advanced image processing technology to improve the resolution of images without increasing hardware costs, making the image clearer and showing more details.

[0069] More specifically, the image enhancement module 1 can also enhance the visual effect of the image by adjusting contrast, color, brightness, etc., making the image more vivid and easier to observe. The image enhancement module 1 is optimized for the special needs of the handheld ultrasound device to meet the needs of operators in real-time ultrasound scanning.

[0070] Reference Figure 3 , taking carotid artery plaque segmentation as an example, the specific implementation steps are as follows:

[0071] Step 1: A low-quality carotid artery scan image is obtained by handheld ultrasound scanning, and the low-quality carotid artery scan image is input into the image enhancement module 1 to obtain an image after quality improvement, and the image enhancement loss of step S2 of the present invention is calculated based on the pre-prepared high-quality image and the image after quality improvement;

[0072] Step 2: Input the image after quality improvement into the image segmentation module 2 to obtain a carotid plaque segmentation mask, and calculate the segmentation loss of step S3 of the present invention by using the carotid plaque segmentation mask and the pre-prepared carotid plaque annotation mask;

[0073] Step three, the loss function is calculated based on the image enhancement loss and the segmentation loss, and the entire model is trained using the loss function.

[0074] A low-quality carotid artery scan image is obtained by inputting a handheld ultrasound scan into the trained image processing model, and a more accurate carotid artery plaque segmentation image is obtained after passing through the trained image enhancement module 1 and the image segmentation module 2.

[0075] Specifically, the image enhancement module 1 generates a third ultrasound image by processing the first ultrasound image, making it closer to the prepared second ultrasound image, which can significantly improve the visual quality, contrast and detail clarity of the ultrasound image. The image segmentation module 2 receives the enhanced third ultrasound image and generates a segmented image. Based on the loss function calculated based on the image enhancement loss and the segmentation loss, the system can comprehensively consider the dual optimization goals of image quality and segmentation accuracy. The image processing model is prompted to better balance the relationship between the two during the training process, thereby achieving better overall performance.

[0076] Specifically, the entire method is an end-to-end training process, that is, from the first ultrasound image to the final segmented image, the image enhancement module and the image segmentation module are directly optimized through the loss function, which can improve the integration and overall efficiency of the system and reduce the interference of manual adjustments and intermediate steps on the results. The joint optimization method is more flexible and can adapt to different types of ultrasound images and a variety of medical application scenarios.

[0077] In summary, the present application provides a joint optimization method for ultrasound image quality enhancement and segmentation. When the image quality obtained by handheld ultrasound scanning is not high, the user experience of the handheld ultrasound device is improved with the help of an artificial intelligence model, and an implicit high-quality image space is established to remove the influence of interference factors such as noise on image segmentation. In the inference stage, the model accepts low-quality handheld ultrasound images, obtains quality-enhanced images through the trained image quality enhancement module, and then enters the image segmentation module 2 to obtain more accurate image segmentation results. The segmentation results and the quality-enhanced images can be displayed to the user, and the results are more accurate.

[0078] The above description is only a preferred embodiment of the present invention, and does not limit the implementation mode and protection scope of the present invention. For those skilled in the art, it should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the description and illustrations of the present invention should be included in the protection scope of the present invention.

Claims

1. A joint optimization method for ultrasound image quality enhancement and segmentation, characterized in that: include, Step S1, constructing an image processing model, wherein the image processing model includes an image enhancement module and an image segmentation module; Step S2, processing the first ultrasonic image through the image enhancement module to obtain a third ultrasonic image, and calculating the image enhancement loss through the third ultrasonic image and the second ultrasonic image; Step S3, inputting the third ultrasound image into the image segmentation module to obtain a segmented image, and calculating the segmentation loss through the segmented image and the annotated image; Step S4, calculating a loss function based on the image enhancement loss and the segmentation loss; Step S5: training the image processing model through the loss function, and jointly optimizing the image enhancement module and the image segmentation module.

2. The joint optimization method for ultrasound image quality enhancement and segmentation according to claim 1, characterized in that: The calculation formula of the image enhancement loss in step S2 is: Among them, L enhance represents the image enhancement loss, I′ represents the second ultrasound image, represents the third ultrasound image.

3. The joint optimization method for ultrasound image quality enhancement and segmentation according to claim 1, characterized in that: The calculation formula of the segmentation loss in step S3 is: L seg =f(M′-M); Among them, L seg represents the segmentation loss, f represents the segmentation loss function, M′ represents the labeled image, and M represents the segmented image.

4. The joint optimization method for ultrasound image quality enhancement and segmentation according to claim 1, characterized in that: The calculation formula of the loss function in step S4 is: L=L seg +α×L enhance ; Wherein, L represents the loss function, L seg represents the segmentation loss, α represents the weight coefficient, L enhance represents the image enhancement loss.

5. The joint optimization method for ultrasound image quality enhancement and segmentation according to claim 1, characterized in that: In step S2, the first ultrasound image is acquired by a handheld ultrasound device.

6. The joint optimization method for ultrasound image quality enhancement and segmentation according to claim 1, characterized in that: The image quality of the first ultrasonic image is lower than that of the second ultrasonic image, and the image quality of the first ultrasonic image is lower than that of the third ultrasonic image.

7. The joint optimization method for ultrasound image quality enhancement and segmentation according to claim 1, characterized in that: In step S2, the first ultrasonic image is obtained by adding noise, style transfer, and blurring to the second ultrasonic image.

8. The joint optimization method for ultrasound image quality enhancement and segmentation according to claim 1, characterized in that: In step S2, the second ultrasonic image is the expected image of the image enhancement module, and the second ultrasonic image is obtained by a large ultrasonic device.

9. The joint optimization method for ultrasound image quality enhancement and segmentation according to claim 1, characterized in that: The image segmentation module is a probabilistic generative adversarial network or a U-type network.

10. The joint optimization method for ultrasound image quality enhancement and segmentation according to claim 1, characterized in that: The image enhancement module includes a denoising submodule and a resolution enhancement submodule.

Images