Controllable mobile ultrasound image quality enhancement method, system, device and medium

Abstract

The application discloses a controllable mobile ultrasonic image quality improvement method, system and device and a medium, wherein the method is to input the collected real medical mobile ultrasonic image into a trained super-resolution reconstruction model and adjust a blur parameter; the super-resolution reconstruction model converts the input real medical mobile ultrasonic image into a reconstructed medical mobile ultrasonic image with a corresponding super-resolution degree according to the adjusted resolution parameter. The super-resolution reconstruction model adopts the framework of a Cycle generative adversarial network and introduces a controllable super-parameter convolution kernel. A diversified data set is adopted, and a dynamic random image damage technology is used to process part of the data set. The model trains the multi-scale features of the data set together with the blur parameter, and optimizes the model by using a composite loss function. Therefore, the application greatly improves the imaging quality, has flexibility in adjusting the super-resolution, and simplifies the deep learning training process.

Description

Technical Field

[0001] This invention belongs to the field of medical imaging and image processing technology, specifically relating to a controllable method, system, device, and medium for improving the quality of mobile ultrasound images. Background Technology

[0002] Improving the quality of primary healthcare services has become a common goal in many developing countries. Mobile ultrasound devices, due to their portability and ease of use, are particularly important in environments where traditional large medical equipment is unavailable, playing a crucial role in the diagnosis of complex conditions requiring immediate attention. The widespread adoption of mobile ultrasound devices is expected to increase significantly in the future, highlighting their importance as an indispensable tool in the evolution of primary healthcare services.

[0003] However, existing mobile ultrasound devices are limited in image quality by hardware, such as probe frequency and imaging technology. As a result, the mobile ultrasound images often have problems such as low resolution, insufficient contrast, and unclear organ boundaries. These problems seriously affect the accuracy and efficiency of clinical diagnosis, making it difficult to meet clinical needs.

[0004] Traditional methods for improving the quality of mobile ultrasound images include modulation and demodulation techniques as well as improvements to ultrasound transmission, reception, and beamforming modules. While effective, these methods all require a trade-off between image quality and equipment cost / portability. Furthermore, while existing deep learning methods have demonstrated superior performance in super-resolution reconstruction, they often fail to effectively improve image texture and may even lead to texture alterations, which is unacceptable in medical imaging.

[0005] In super-resolution reconstruction of ultrasound images, the lack of paired high- and low-resolution datasets for medical ultrasound images makes training with deep learning methods exceptionally difficult. Furthermore, existing deep learning methods have limited effectiveness in improving image texture, sometimes even leading to undesirable changes in image texture.

[0006] Furthermore, in actual clinical examinations, doctors often wish to adjust the degree of super-resolution according to the specific medical condition in order to obtain more accurate diagnostic information. However, current technology does not provide sufficient flexibility to meet this need, enabling doctors to adjust the degree of super-resolution as needed in order to obtain diagnostic information more accurately.

[0007] Therefore, there is an urgent need to develop a new low-cost method that not only maintains the compactness of the equipment but also significantly improves the quality of ultrasound imaging. Summary of the Invention

[0008] To overcome the shortcomings of existing noise reduction methods, this invention provides a controllable mobile ultrasound image quality improvement method, system, device, and medium that can meet the balance between high-quality imaging and equipment portability, while also providing the ability to dynamically adjust super-resolution to adapt to different clinical needs.

[0009] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution: A controllable method for improving the quality of mobile ultrasound images, comprising: Step 1) Acquire real medical mobile ultrasound images of the patient, wherein the real medical mobile ultrasound images are formed by mobile ultrasound equipment; Step 2) Input the acquired real medical mobile ultrasound images into the trained super-resolution reconstruction model; Step 3) Adjust the fuzzy parameters of the super-resolution reconstruction model to control the degree of super-resolution in the reconstructed medical mobile ultrasound image; Step 4) The super-resolution reconstruction model converts the input real medical mobile ultrasound image into a corresponding reconstructed medical mobile ultrasound image with a corresponding degree of super-resolution based on the adjusted resolution parameters. Step 5) Output the reconstructed medical mobile ultrasound image with the corresponding super-resolution level generated by the super-resolution reconstruction model.

[0010] Furthermore, the super-resolution reconstruction model adopts a framework based on Cycle Generative Adversarial Network (CycleGAN), whose structure includes at least a generator G. AB Generator G BA And the discriminator; among which, The generator G AB It is responsible for converting the input low-resolution real image into a corresponding high-resolution reconstructed image, so as to achieve super-resolution reconstruction of the input low-resolution real image; The discriminator is responsible for using a high-resolution real image paired with the input low-resolution real image and comparing it with the generator G. AB The generated high-resolution reconstructed images are compared to optimize the parameters of the super-resolution reconstruction model; The generator G BA responsible for generating the generator G AB The generated high-resolution reconstructed image is converted into a low-resolution restored image corresponding to the input low-resolution real image, and compared with the input low-resolution real image to optimize the parameters of the super-resolution reconstruction model. The generator G AB and the generator G BA All decoders incorporate controllable hyperparameters, which are fuzzy parameters. These hyperparameters are adjusted by modifying the generator G.AB The fuzzy parameters can control the generator G. AB The degree of super-resolution of the generated high-definition reconstructed image is determined by adjusting the generator G. BA The fuzzy parameters can control the generator. BA The degree of blur restoration in the generated low-resolution restored image; The generator G AB and the generator G BA The decoders in the network are all hyperparameter decoders controlled by hyperparameter cues. The hyperdecoder uses a hyperparameter-tunable hyperparameter convolution kernel (HyperConv). The hyperparameter convolution kernel is responsible for extracting latent features (the number of times manual blurring is performed) from the blurring parameters through a multilayer perceptron (MLP), and mapping them along with trainable weights to a specific convolution kernel, so that the hyperdecoder can maintain the convolution kernel parameters under different blurring parameters, which facilitates the network to switch cores.

[0011] Furthermore, the training method for the super-resolution reconstruction model includes the following steps: Step 2.1) Construct a diverse dataset that includes natural image datasets, paired image datasets, and unpaired image datasets; Step 2.2) Input diverse datasets into the super-resolution reconstruction model, which automatically distinguishes dataset types; Step 2.3) The super-resolution reconstruction model extracts multi-scale features of the input image based on the type of the input dataset; Step 2.4) The super-resolution reconstruction model obtains the blur parameter values ​​of the input image according to the type of the input dataset; Step 2.5) The super-resolution reconstruction model is trained by combining the extracted multi-scale features with the obtained fuzzy parameter values ​​to train the super-resolution reconstruction capability. Step 2.6) Optimize the super-resolution reconstruction model using a composite loss function that integrates pixel-level loss, adversarial loss, perceptual loss, and cycle consistency loss.

[0012] Furthermore, the method for constructing the diverse dataset is as follows: First, collect a certain number of "high-definition natural images", "high-definition real medical ultrasound report images", "low-definition real medical mobile ultrasound images", and "low-definition real medical mobile ultrasound images". Then, dynamic random image destruction technology is used to artificially blur the "high-definition natural image" and "high-definition real medical ultrasound report image" to generate "low-definition simulated natural image" paired with "high-definition natural image" and "low-definition simulated medical ultrasound report image" paired with "high-definition real medical ultrasound report image". Finally, “low-resolution simulated natural images” are paired with “high-resolution natural images” to form a natural image dataset, “low-resolution simulated medical ultrasound report images” are paired with “high-resolution real medical ultrasound report images” to form a paired image dataset, and “low-resolution real medical mobile ultrasound images” are used as an unpaired image dataset, thus forming the diverse dataset.

[0013] Furthermore, the processing method of the dynamic random image destruction technique is as follows: The image to be processed is subjected to a random number of manual blurring operations, and each time the manual blurring operation is performed, one of several specific blurring methods is randomly selected. The number of manual blurring operations is the blurring parameter value that can be obtained during the training of the super-resolution reconstruction model, in order to simulate the dynamic adjustment of the degree of super-resolution in actual applications; The artificial blurring techniques include Gaussian blur, mesh blur, mosaic blur, and orientation blur; among them, The Gaussian blur is used to simulate the smoothing effect caused by rapid patient movement or hand tremors during image acquisition; The blurred box is used to indicate pixelation that may occur in low-resolution displays, which are often found on compact, portable devices. The mosaic blur is used to simulate sensor noise effects; The directional blur is used to simulate the imaging distortion caused by the scanner moving along the anatomical surface of the curved surface.

[0014] Furthermore, the natural image dataset, the paired image dataset, and the unpaired image dataset each undergo the following training process: If the input is a natural image dataset, the super-resolution reconstruction model first performs multi-scale feature extraction on the "low-resolution simulated natural image" in the natural image dataset, and obtains the corresponding blur parameter values ​​based on the number of times it has undergone manual blurring. Then, based on the extracted multi-scale features and the obtained blur parameter values, a corresponding "high-resolution reconstructed natural image" is generated. The resolution improvement of the generated "high-resolution reconstructed natural image" relative to the input "low-resolution simulated natural image" corresponds to the obtained blur parameter values. Next, feature extraction is performed on the generated "high-resolution reconstructed natural image," and based on the extracted features and the obtained blur parameter values, a corresponding "low-resolution restored natural image" is generated. Finally, the generated "high-resolution reconstructed natural image" is compared with the "high-resolution simulated natural image" in the natural image dataset, and the generated "low-resolution restored natural image" is compared with the "low-resolution simulated natural image" in the natural image dataset. The parameters of the super-resolution reconstruction model are optimized through a composite loss function. If the input is a paired image dataset, the super-resolution reconstruction model first performs multi-scale feature extraction on the "low-resolution simulated medical ultrasound report image" in the paired image dataset, and obtains the corresponding blur parameter values ​​based on the number of times it has undergone manual blurring. Then, based on the extracted multi-scale features and the obtained blur parameter values, a corresponding "high-resolution reconstructed medical ultrasound report image" is generated. The resolution improvement of the generated "high-resolution reconstructed medical ultrasound report image" relative to the input "low-resolution simulated medical ultrasound report image" corresponds to the obtained blur parameter values. Next, feature extraction is performed on the generated "high-resolution reconstructed medical ultrasound report image," and based on the extracted features and the obtained blur parameter values, a corresponding "low-resolution restored medical ultrasound report image" is generated. Finally, the generated "high-resolution reconstructed medical ultrasound report image" is compared with the "high-resolution real medical ultrasound report image" in the paired image dataset, and the generated "low-resolution restored medical ultrasound report image" is compared with the "low-resolution simulated medical ultrasound report image" in the paired image dataset. The parameters of the super-resolution reconstruction model are optimized through a composite loss function. If the input is an unpaired image dataset, the super-resolution reconstruction model first performs multi-scale feature extraction on the "low-resolution real medical mobile ultrasound image" in the unpaired image dataset and specifies its fuzzy parameter value; then, based on the extracted multi-scale features and the specified fuzzy parameter value, it generates the corresponding "high-resolution reconstructed medical mobile ultrasound image". The resolution improvement of the generated "high-resolution reconstructed medical mobile ultrasound image" relative to the input "low-resolution real medical mobile ultrasound image" corresponds to the specified fuzzy parameter value; next, it extracts features from the generated "high-resolution reconstructed medical mobile ultrasound image" and generates the corresponding "low-resolution restored medical mobile ultrasound image" based on the extracted features and the specified fuzzy parameter value; finally, it compares the generated "low-resolution restored medical mobile ultrasound image" with the "low-resolution real medical mobile ultrasound image" in the unpaired image dataset and optimizes the parameters of the super-resolution reconstruction model through a composite loss function.

[0015] A controllable mobile ultrasound image quality improvement system includes: a data receiving unit, a data processing unit, and a data output unit; The data receiving unit is responsible for acquiring real medical mobile ultrasound images captured by the mobile ultrasound device; The data processing unit is responsible for executing the operations corresponding to the above-mentioned controllable mobile ultrasound image quality improvement method, and finally generating a reconstructed medical mobile ultrasound image with a corresponding super-resolution level. The data output unit is responsible for displaying the generated reconstructed medical mobile ultrasound image with the corresponding super-resolution level; The controllable mobile ultrasound image quality enhancement system is either a standalone image processing device or integrated into a mobile ultrasound device.

[0016] A computer device includes a processor, a memory, a communication interface, and a communication bus. The processor, the memory, and the communication interface communicate with each other via the communication bus. The memory stores at least one executable instruction, which causes the processor to perform the operation corresponding to the controllable mobile ultrasound image quality improvement method described above.

[0017] A computer-readable storage medium storing at least one executable instruction that causes a processor to perform operations corresponding to the above-described controllable mobile ultrasound image quality improvement method.

[0018] Compared with the prior art, the present invention has the following beneficial effects: Improving Image Quality: The super-resolution reconstruction method based on Generative Adversarial Networks (GANs) proposed in this invention can significantly improve the image quality of mobile ultrasound equipment, especially in terms of contrast, resolution, and boundary sharpness. By optimizing the generator of the CycleGAN network and using hyperparameter convolutional kernels to extract multi-scale features, higher-quality image reconstruction is achieved.

[0019] Furthermore, this invention employs multiple loss functions (including pixel-level loss, perceptual loss, and cycle consistency loss) to optimize the training of the super-resolution network, thereby improving the quality and texture realism of the generated images. This ensures that the super-resolution reconstruction model possesses pixel-level accuracy and perceptual similarity, thus optimizing the training of the super-resolution reconstruction model and improving the quality and texture realism of the generated images.

[0020] 2. Providing flexibility in adjusting super-resolution: This invention innovatively introduces controllable hyperparameters, allowing physicians to dynamically adjust the degree of super-resolution according to different clinical needs. This function is achieved by adjusting the hyperparameters in the decoder, thereby providing greater application flexibility and personalized diagnostic support.

[0021] 3. Simplified deep learning training process: This invention adopts a hybrid training strategy, using a combination of natural images and medical ultrasound images, as well as pseudo-paired datasets generated by random injury technology, which can effectively train deep learning models even when paired data is scarce.

[0022] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the invention and to implement it according to the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. Specific embodiments of the present invention are given in detail below with reference to the accompanying drawings. Attached Figure Description

[0023] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings: Figure 1 This is a schematic diagram of the CycleGAN network used in this invention; Figure 2 This is a schematic diagram of the controllable hyperparameter convolution kernel introduced in the CycleGAN network of this invention; Figure 3 This is the pseudocode for the controllable hyperparameter convolution kernel of this invention; Figure 4 This is a schematic diagram illustrating various artificial modulus processing techniques used in the dynamic random image destruction technology employed in this invention. Detailed Implementation

[0024] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings to provide a clearer understanding of the invention's purpose, features, and advantages. It should be understood that the embodiments shown in the drawings are not intended to limit the scope of the invention, but are merely illustrative of the essential spirit of the invention's technical solution.

[0025] In the following description, certain specific details are set forth for the purpose of illustrating various disclosed embodiments in order to provide a thorough understanding of the various disclosed embodiments. However, those skilled in the art will recognize that embodiments may be practiced without one or more of these specific details. In other instances, well-known apparatuses, structures, and techniques associated with this application may not have been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments.

[0026] Unless the context requires otherwise, throughout the specification and claims, the word “comprising” and its variations, such as “including” and “having”, shall be understood to have an open, inclusive meaning, that is, to be interpreted as “including, but not limited to”.

[0027] Throughout this specification, references to "an embodiment" or "an embodiment" indicate that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Therefore, the appearance of "in an embodiment" or "an embodiment" in various places throughout the specification does not necessarily refer to the same embodiment. Furthermore, a particular feature, structure, or characteristic may be combined in any manner in one or more embodiments.

[0028] The singular forms “a” and “the” used in this specification and the appended claims include plural references unless otherwise expressly stated herein. It should be noted that the term “or” is generally used to mean “and / or” unless otherwise expressly stated herein.

[0029] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0030] This invention first proposes a super-resolution reconstruction model, which is essentially a complex image-to-image translation framework that uses a generative adversarial network (GAN) architecture to emphasize structural integrity.

[0031] See Figure 1 As shown, the super-resolution reconstruction model of this invention preferably employs an optimized Cycle Generative Adversarial Network (CycleGAN) framework to achieve super-resolution reconstruction of ultrasound images. The CycleGAN framework is an important architecture for unsupervised image-to-image translation, learning two mapping functions (derived by the generator G...) AB Generator GBA (representation), and its associated discriminator, to convert between different image domains A and B.

[0032] The main translation process of the super-resolution reconstruction model of this invention is as follows: starting from a low-resolution real image P1 (real medical moving ultrasound image) in image domain A, and through generator G... AB Convert it into a high-resolution reconstructed image P3 (reconstructed medical mobile ultrasound image) in image domain B.

[0033] To improve the quality of these generated images, the super-resolution reconstruction model of this invention proposes a composite loss function that integrates pixel-level loss, adversarial loss, perceptual loss, and cycle consistency loss. This function aims to ensure pixel-level accuracy and perceptual similarity, as well as maintain semantic (image texture, content, etc.) consistency between the source and generated images.

[0034] During training, the generator G AB The generator G is responsible for converting the low-resolution ground truth image P1 (located in image domain A) in the dataset into the corresponding high-resolution reconstructed image P3 (located in image domain B); BA Responsible for generating G AB The generated high-resolution reconstructed image P3 (located in image domain B) is converted into a low-resolution restored image P4 corresponding to the input low-resolution ground image P1; the discriminator and composite loss function are responsible for using the high-resolution ground image P2 (located in image domain B) in the dataset to evaluate the generator G. AB The generated high-resolution reconstructed image P3 (located in image domain B) is compared, and the generator G is used... BA The generated low-resolution restored image P4 is compared with the low-resolution real image P1 (located in image domain A) in the dataset, thereby optimizing the parameters of the super-resolution reconstruction model of the present invention and improving the quality of the generated image.

[0035] To improve the flexibility of model super-resolution reconstruction, see [link to relevant documentation]. Figure 2As shown, this invention proposes a controllable hyperparameter convolutional kernel (HyperConv) to dynamically adjust the generator's behavior to adapt to different image content and clinical needs. In deep learning networks, the convolutional kernel is a fundamental structure. Traditional decoders use fixed convolutional kernels, which cannot be dynamically adjusted according to actual conditions, leading to the loss of detailed information in the reconstructed image. This invention, while retaining the original encoder's convolutional kernel (Conv) and residual block structure in the generator, replaces the fixed convolutional kernel of the decoder in the generator with a novel hyperparameter-tunable hyperparameter convolutional kernel (HyperConv). This hyperparameter convolutional kernel can extract latent features from the blurring parameters through a multilayer perceptron (MLP), and then map them along with trainable weights to a specific convolutional kernel. This allows the superdecoder to maintain the convolutional kernel parameters under different blurring parameters, facilitating network kernel switching.

[0036] The PyTorch-style pseudocode for the hyperparameter convolution kernel (HyperConv) is as follows: Figure 3 As shown.

[0037] Figure 3 The pseudocode shown demonstrates how to implement a HyperConv convolutional layer in the PyTorch framework. This layer can dynamically adjust the convolution kernel based on the input blur parameters to optimize the quality of image reconstruction.

[0038] 1. Parameter Initialization: First, create a parameter `param`, whose structure is defined by `kshape` (the shape of the convolution kernel) and `cw` (the dimension of the weight library). This allows the shape and size of the convolution kernel to be adjusted as needed.

[0039] 2. Linear Transformation: A linear layer, Linear, is used to process the fuzzy parameters s. This step maps the fuzzy parameters to a new space corresponding to the dimension of the weight library. The output is then reshaped into a (cw, 1) shape using the view function, ready for multiplication with the parameter matrix. Here, the Linear layer, which processes the fuzzy parameters s, is essentially a single-layer neural network responsible for mapping the fuzzy parameters from their original dimension cs to the new spatial dimension cw. For simplicity, only one linear layer is shown in the pseudocode; in reality, multiple such linear layers are stacked, with activation functions between each layer to increase the model's non-linear expressiveness.

[0040] 3. Convolution Kernel Generation: The result of the linear transformation is multiplied by the initialization parameters using matrix multiplication (matmul) to generate a dynamically adjusted convolution kernel. The output shape is then adjusted again via view to match kshape, ensuring compatibility of the convolution operation.

[0041] 4. Convolution Operation: The generated convolution kernel is used to perform a convolution operation on the input feature map x to obtain the output feature map y. This step is the actual image processing operation, adjusting feature extraction based on the dynamically generated convolution kernel to adapt to different levels of blur.

[0042] The advantage of this method lies in its dynamism, which allows it to adjust the processing strategy according to the specific degradation of the image, thereby ensuring the quality and accuracy of image reconstruction in various application scenarios (such as different levels of image blur parameter requirements).

[0043] Generator G AB and generator G BA All decoders incorporate controllable hyperparameter kernels, which are adjusted by the generator G. AB The hyperparameters (fuzzy parameters) can control the generator G. AB The degree of super-resolution of the generated high-resolution reconstructed image P3 is determined by adjusting the generator G. BA The hyperparameters (fuzzy parameters) can control the generator G. BA The degree of blur restoration of the generated low-resolution restored image P4.

[0044] It should be noted that the aforementioned blur parameter actually refers to a parameter value extracted by the model from the input image during training. This parameter value reflects the degree of blur in the input image. Based on the introduction of a controllable hyperparameter convolution kernel, this invention uses this parameter value as a parameter for the degree of super-resolution reconstruction performed by the model. It is trained together with the multi-scale features of the input image extracted by the model to simulate the adjustment of hyperparameters during super-resolution reconstruction, thereby enabling the trained model to generate reconstructed images with a super-resolution degree corresponding to the blur parameter.

[0045] This invention proposes a super-resolution reconstruction method based on CycleGAN networks, which can significantly improve the image quality of mobile ultrasound equipment, particularly in terms of contrast, resolution, and boundary sharpness. By optimizing the generator of the CycleGAN network and employing hyperparameter convolution kernels to extract multi-scale features, higher-quality image reconstruction is achieved.

[0046] Furthermore, by innovatively introducing controllable hyperparameters, the super-resolution reconstruction model of this invention allows physicians to dynamically adjust the degree of super-resolution according to different clinical needs. This function is achieved by adjusting the hyperparameters (fuzzy parameters) in the decoder, thereby providing greater application flexibility and personalized diagnostic support.

[0047] To simulate mobile ultrasound imaging on a large scale, this invention proposes a sophisticated dynamic random image destruction technique that artificially blurs the dataset, thereby replicating the typical artifacts and low-resolution problems encountered in clinical settings when using mobile ultrasound equipment.

[0048] The dynamic random image destruction technique includes the following four artificial modulo methods: Gaussian blur is used to simulate the smoothing effect caused by rapid patient movement or hand tremors during image acquisition.

[0049] The box blur is used to indicate pixelation that may occur in low-resolution displays, which are often found on compact, portable devices.

[0050] The mosaic blurring is used to simulate sensor noise effects, which is most common in portable ultrasonic machines that may lack the advanced hardware features of their fixed-position pairing devices.

[0051] The directional ambiguity, used to simulate the imaging distortion caused by the movement of a scanner along a curved anatomical surface, is a common challenge in emergency scenarios or bedside cases.

[0052] See Figure 4 As shown, Figure 4 Image a shows the ultrasound image before blurring. Figure 4 b shows the ultrasound image after Gaussian blurring. Figure 4 c shows the ultrasound image after the box has been blurred. Figure 4 d shows a low-resolution moving ultrasound image. Figure 4 e shows the ultrasound image after it has been pixelated and blurred. Figure 4 e shows the ultrasound image after directional blurring.

[0053] The dynamic random image destruction technique involves performing a random number of artificial blurring operations (1-19 times) on the image to be processed, with each blurring operation randomly selecting one of the four specific artificial blurring methods mentioned above. This invention, by employing dynamic random image destruction technology, simulates the low-resolution situation when using mobile ultrasound imaging equipment, providing numerous paired samples for a limited unpaired dataset.

[0054] Here, the number of times the manual blurring is applied (1-19 times) refers to the blurring parameters of the input image that the hyperparameter convolution kernel can obtain during model training. The larger the value of the blurring parameter, the higher the degree of blurring (damage) of the input image, and consequently, the higher the degree of image quality improvement required by the model during super-resolution reconstruction, in order to simulate the dynamic adjustment of blurring parameters made by doctors due to the requirements of super-resolution in real-world applications.

[0055] Based on this dynamic random image destruction technique, this invention also proposes a training method or strategy for the aforementioned super-resolution reconstruction model. The training method or strategy specifically includes the following steps: 1) Construct diverse datasets.

[0056] Collect a certain number of "high-definition natural images", "high-definition real medical ultrasound report images", "low-definition real medical mobile ultrasound images", and "low-definition real medical mobile ultrasound images".

[0057] The aforementioned dynamic random image destruction technique is used to artificially blur the "high-definition natural image" and the "high-definition real medical ultrasound report image" to generate "low-definition simulated natural image" paired with the "high-definition natural image" and "low-definition simulated medical ultrasound report image" paired with the "high-definition real medical ultrasound report image".

[0058] The diverse dataset is constructed by pairing "low-resolution simulated natural images" with "high-resolution natural images" as a natural image dataset, pairing "low-resolution simulated medical ultrasound report images" with "high-resolution real medical ultrasound report images" as a paired image dataset, and using "low-resolution real medical mobile ultrasound images" as an unpaired image dataset.

[0059] A super-resolution reconstruction model was trained using a diverse dataset.

[0060] A diverse dataset is input into the super-resolution reconstruction model, which automatically distinguishes the dataset types in the diverse dataset, classifying the currently trained dataset as a natural image dataset, a paired image dataset, or an unpaired image dataset.

[0061] If the input is a natural image dataset, the super-resolution reconstruction model first performs multi-scale feature extraction on the "low-resolution simulated natural image" in the dataset and obtains the corresponding blur parameter values ​​(1-19 times) based on the number of times it has undergone manual blurring. Then, based on the extracted multi-scale features and the obtained blur parameter values ​​(1-19 times), the corresponding "high-resolution reconstructed natural image" is generated. The resolution improvement of the generated "high-resolution reconstructed natural image" relative to the input "low-resolution simulated natural image" corresponds to the obtained blur parameter values. That is, if the blur parameter value is 10 times, it means that the input "low-resolution simulated natural image" has been manually blurred 10 times. Therefore, when reconstructing the "high-resolution reconstructed natural image," the generator needs to improve the resolution of the input "low-resolution simulated natural image" 10 times. Next, feature extraction is performed on the generated "high-resolution reconstructed natural image," and based on the extracted features and the obtained blur parameter values, the corresponding "low-resolution restored natural image" is generated. Finally, the generated "high-resolution reconstructed natural image" is compared with the "high-resolution simulated natural image" in the natural image dataset, and the generated "low-resolution restored natural image" is compared with the "low-resolution simulated natural image" in the natural image dataset. The parameters of the super-resolution reconstruction model are optimized by using a composite loss function.

[0062] Natural image datasets capture a variety of real-world scenes, providing a rich foundation for feature extraction and helping models learn diverse textures and structural details that are not specific to medical imaging.

[0063] If the input is a paired image dataset, the super-resolution reconstruction model first performs multi-scale feature extraction on the "low-resolution simulated medical ultrasound report image" in the paired image dataset, and obtains the corresponding blur parameter value (1-19 times) based on the number of times it has undergone manual blurring. Then, based on the extracted multi-scale features and the obtained blur parameter value (1-19 times), the corresponding "high-resolution reconstructed medical ultrasound report image" is generated. The resolution improvement of the generated "high-resolution reconstructed medical ultrasound report image" relative to the input "low-resolution simulated medical ultrasound report image" corresponds to the obtained blur parameter value. That is, if the blur parameter value is 8 times, it means that the input "low-resolution simulated medical ultrasound report image" has been manually blurred 8 times. Therefore, when generating the "high-resolution reconstructed medical ultrasound report image", the generator needs to perform 8 resolution improvements on the input "low-resolution simulated medical ultrasound report image". Next, feature extraction is performed on the generated "high-resolution reconstructed medical ultrasound report image", and based on the extracted features and the obtained blur parameter value, the corresponding "low-resolution restored medical ultrasound report image" is generated. Finally, the generated "high-definition reconstructed medical ultrasound report images" are compared with the "high-definition real medical ultrasound report images" in the paired image dataset, and the generated "low-definition restored medical ultrasound report images" are compared with the "low-definition simulated medical ultrasound report images" in the paired image dataset. The parameters of the super-resolution reconstruction model are optimized by using a composite loss function.

[0064] The paired image dataset constitutes an important part of our dataset. These images were carefully created by cropping frames from detailed ultrasound reports and collected during clinical diagnosis. Furthermore, these high-resolution images were converted to low-resolution images by introducing the aforementioned dynamic random image distortion techniques (such as Gaussian blur, box blur, mosaic blur, and orientation blur), resulting in a large amount of paired data simulating real ultrasound imaging conditions.

[0065] If the input is an unpaired image dataset, the super-resolution reconstruction model first performs multi-scale feature extraction on the "low-resolution real medical mobile ultrasound images" in the unpaired image dataset and obtains their blur parameter values. For the unpaired dataset, since the images are already of low quality, the blur parameter values ​​are manually set (e.g., 5 times). Then, based on the extracted multi-scale features and the specified blur parameter values, the corresponding "high-resolution reconstructed medical mobile ultrasound image" is generated. The resolution improvement of the generated "high-resolution reconstructed medical mobile ultrasound image" relative to the input "low-resolution real medical mobile ultrasound image" corresponds to the specified blur parameter value. That is, if the blur parameter value is 5 times, it means that the input "low-resolution real medical mobile ultrasound image" is equivalent to being manually blurred 5 times by default. Therefore, when generating the "high-resolution reconstructed medical mobile ultrasound image", the generator needs to improve the resolution of the input "low-resolution real medical mobile ultrasound image" by 5 times. Next, feature extraction is performed on the generated "high-resolution reconstructed medical mobile ultrasound image", and based on the extracted features and the specified blur parameter values, the corresponding "low-resolution restored medical mobile ultrasound image" is generated. Finally, the generated "low-resolution restored medical mobile ultrasound images" are compared with the "low-resolution real medical mobile ultrasound images" in the unpaired image dataset, and the parameters of the super-resolution reconstruction model are optimized by using a composite loss function.

[0066] Conversely, unpaired image datasets consist of low-resolution scans typically obtained using mobile ultrasound equipment, lacking pixel-level correspondence with high-resolution images captured from advanced clinical ultrasound systems. This difference introduces unique challenges during training but is invaluable for improving the model's ability to interpolate high-resolution details from significantly degraded inputs.

[0067] This invention utilizes diverse datasets, including natural image datasets, paired image datasets, and unpaired image datasets, significantly enhancing the model's generalization ability under different imaging conditions. Furthermore, to effectively handle this heterogeneous data, this invention implements an automated training strategy. This automated training strategy enables the model to cleverly distinguish between paired and unpaired images, and applies a comprehensive loss function to the paired dataset to optimize pixel-level accuracy. For the unpaired dataset, this invention specifically omits consideration of pixel-level loss, as the lack of direct high-resolution counterparts could lead to misinterpretations. Through this hybrid training scheme, the model can cleverly learn the mapping relationship between low-resolution and high-resolution images, thereby promoting a more accurate super-resolution process under different imaging conditions.

[0068] This invention also provides a controllable method for improving the quality of moving ultrasound images based on the above-mentioned super-resolution reconstruction model, comprising: Acquire real medical mobile ultrasound images of the patient, wherein the real medical mobile ultrasound images are imaged by mobile ultrasound equipment; The acquired real medical mobile ultrasound images are input into the trained super-resolution reconstruction model; Adjust the fuzzy parameters of the super-resolution reconstruction model to control the degree of super-resolution in the reconstructed medical mobile ultrasound image; The super-resolution reconstruction model converts the input real medical mobile ultrasound image into a corresponding reconstructed medical mobile ultrasound image with a corresponding degree of super-resolution based on the adjusted resolution parameters. Output the reconstructed medical mobile ultrasound image with the corresponding super-resolution level generated by the super-resolution reconstruction model.

[0069] The present invention also provides a controllable mobile ultrasound image quality improvement system, comprising: a data receiving unit, a data processing unit, and a data output unit; The data receiving unit is responsible for acquiring real medical mobile ultrasound images captured by the mobile ultrasound device; The data processing unit is responsible for executing the operations corresponding to the above-mentioned controllable mobile ultrasound image quality improvement method, and finally generating a reconstructed medical mobile ultrasound image with a corresponding super-resolution level. The data output unit is responsible for displaying the generated reconstructed medical mobile ultrasound image with the corresponding super-resolution level; The controllable mobile ultrasound image quality enhancement system is either a standalone image processing device or integrated into a mobile ultrasound device.

[0070] The present invention also provides a computer device, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus, and the memory is used to store at least one executable instruction, wherein the executable instruction causes the processor to perform the operation corresponding to the above-described controllable mobile ultrasound image quality improvement method.

[0071] The present invention also provides a computer-readable storage medium storing at least one executable instruction that causes a processor to perform operations corresponding to the above-described controllable mobile ultrasound image quality improvement method.

[0072] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A controllable method for improving the quality of mobile ultrasound images, characterized in that, include: Step 1) Acquire real medical mobile ultrasound images of the patient, wherein the real medical mobile ultrasound images are formed by mobile ultrasound equipment; Step 2) Input the acquired real medical mobile ultrasound images into the trained super-resolution reconstruction model; The super-resolution reconstruction model adopts a framework based on Cycle adversarial generative networks, and its structure includes at least a generator G. AB Generator G BA And the discriminator; among which, The generator G AB It is responsible for converting the input low-resolution real image (P1) into a corresponding high-resolution reconstructed image (P3) to achieve super-resolution reconstruction of the input low-resolution real image (P1); The discriminator is responsible for using a high-resolution real image (P2) paired with the input low-resolution real image (P1) and the generator G. AB The generated high-resolution reconstructed image (P3) is compared to optimize the parameters of the super-resolution reconstruction model; The generator G BA responsible for generating the generator G AB The generated high-resolution reconstructed image (P3) is converted into a low-resolution restored image (P4) corresponding to the input low-resolution real image, and compared with the input low-resolution real image (P1) to optimize the parameters of the super-resolution reconstruction model; The generator G AB and the generator G BA All decoders incorporate controllable hyperparameters, which are fuzzy parameters. These hyperparameters are adjusted by modifying the generator G. AB The fuzzy parameters can control the generator G. AB The degree of super-resolution of the generated high-resolution reconstructed image (P3) is determined by adjusting the generator G. BA The fuzzy parameters can control the generator G. BA The degree of blur restoration in the generated low-resolution restored image (P4); The generator G AB and the generator G BA The decoders in the network are all hyperparameter decoders controlled by hyperparameter cues. The hyperparameter decoders use hyperparameter-adjustable hyperparameter convolution kernels. The hyperparameter convolution kernels are responsible for extracting latent features from the fuzzy parameters through the multilayer perceptron and mapping them to specific convolution kernels along with trainable weights, so that the hyperparameter decoders can maintain the convolution kernel parameters under different fuzzy parameters, which facilitates the network to switch cores. Step 3) Adjust the fuzzy parameters of the super-resolution reconstruction model to control the degree of super-resolution in the reconstructed medical mobile ultrasound image; Step 4) The super-resolution reconstruction model converts the input real medical mobile ultrasound image into a corresponding reconstructed medical mobile ultrasound image with a corresponding degree of super-resolution based on the adjusted fuzzy parameters. Step 5) Output the reconstructed medical mobile ultrasound image with the corresponding super-resolution level generated by the super-resolution reconstruction model.

2. The controllable mobile ultrasound image quality improvement method according to claim 1, characterized in that, The training method for the super-resolution reconstruction model includes the following steps: Step 2.1) Construct a diverse dataset that includes natural image datasets, paired image datasets, and unpaired image datasets; Step 2.2) Input diverse datasets into the super-resolution reconstruction model, which automatically distinguishes dataset types; Step 2.3) The super-resolution reconstruction model extracts multi-scale features of the input image based on the type of the input dataset; Step 2.4) The super-resolution reconstruction model obtains the blur parameter values ​​of the input image according to the type of the input dataset; Step 2.5) The super-resolution reconstruction model is trained by combining the extracted multi-scale features with the obtained fuzzy parameter values ​​to train the super-resolution reconstruction capability. Step 2.6) Optimize the super-resolution reconstruction model using a composite loss function that integrates pixel-level loss, adversarial loss, perceptual loss, and cycle consistency loss.

3. The controllable mobile ultrasound image quality improvement method according to claim 2, characterized in that, The method for constructing the diverse dataset is as follows: First, collect a certain number of "high-definition natural images", "high-definition real medical ultrasound report images", "low-definition real medical mobile ultrasound images", and "low-definition real medical mobile ultrasound images". Then, dynamic random image destruction technology is used to artificially blur the "high-definition natural image" and the "high-definition real medical ultrasound report image" to generate "low-definition simulated natural image" paired with the "high-definition natural image" and "low-definition simulated medical ultrasound report image" paired with the "high-definition real medical ultrasound report image". Finally, "low-resolution simulated natural images" are paired with "high-resolution natural images" to form a natural image dataset, "low-resolution simulated medical ultrasound report images" are paired with "high-resolution real medical ultrasound report images" to form a paired image dataset, and "low-resolution real medical mobile ultrasound images" are used as an unpaired image dataset, thus forming the diverse dataset.

4. The controllable mobile ultrasound image quality improvement method according to claim 3, characterized in that, The processing method of the dynamic random image destruction technique is as follows: The image to be processed is subjected to a random number of manual blurring operations, and each time the manual blurring operation is performed, one of several specific blurring methods is randomly selected. The number of manual blurring operations is the blurring parameter value that can be obtained during the training of the super-resolution reconstruction model, so as to simulate the dynamic adjustment of the degree of super-resolution in actual applications; The artificial blurring techniques include Gaussian blur, mesh blur, mosaic blur, and orientation blur; among them, The Gaussian blur is used to simulate the smoothing effect caused by rapid patient movement or hand tremors during image acquisition; The mesh blur is used to represent pixelation that may occur in low-resolution displays, which are often found on compact, portable devices. The mosaic blur is used to simulate sensor noise effects; The directional ambiguity is used to simulate the imaging distortion caused by the scanner moving along the anatomical surface of a curved surface.

5. The controllable mobile ultrasound image quality improvement method according to claim 4, characterized in that, The natural image dataset, the paired image dataset, and the unpaired image dataset each have the following training processes: If the input is a natural image dataset, the super-resolution reconstruction model first performs multi-scale feature extraction on the "low-resolution simulated natural image" in the natural image dataset, and obtains the corresponding blur parameter values ​​based on the number of times it has undergone manual blurring. Then, based on the extracted multi-scale features and the obtained blur parameter values, a corresponding "high-resolution reconstructed natural image" is generated. The resolution improvement of the generated "high-resolution reconstructed natural image" relative to the input "low-resolution simulated natural image" corresponds to the obtained blur parameter values. Next, feature extraction is performed on the generated "high-resolution reconstructed natural image," and based on the extracted features and the obtained blur parameter values, a corresponding "low-resolution restored natural image" is generated. Finally, the generated "high-resolution reconstructed natural image" is compared with the "high-resolution simulated natural image" in the natural image dataset, and the generated "low-resolution restored natural image" is compared with the "low-resolution simulated natural image" in the natural image dataset. The parameters of the super-resolution reconstruction model are optimized through a composite loss function. If the input is a paired image dataset, the super-resolution reconstruction model first performs multi-scale feature extraction on the "low-resolution simulated medical ultrasound report image" in the paired image dataset, and obtains the corresponding blur parameter values ​​based on the number of times it has undergone manual blurring. Then, based on the extracted multi-scale features and the obtained blur parameter values, a corresponding "high-resolution reconstructed medical ultrasound report image" is generated. The resolution improvement of the generated "high-resolution reconstructed medical ultrasound report image" relative to the input "low-resolution simulated medical ultrasound report image" corresponds to the obtained blur parameter values. Next, feature extraction is performed on the generated "high-resolution reconstructed medical ultrasound report image," and based on the extracted features and the obtained blur parameter values, a corresponding "low-resolution restored medical ultrasound report image" is generated. Finally, the generated "high-resolution reconstructed medical ultrasound report image" is compared with the "high-resolution real medical ultrasound report image" in the paired image dataset, and the generated "low-resolution restored medical ultrasound report image" is compared with the "low-resolution simulated medical ultrasound report image" in the paired image dataset. The parameters of the super-resolution reconstruction model are optimized through a composite loss function. If the input is an unpaired image dataset, the super-resolution reconstruction model first performs multi-scale feature extraction on the "low-resolution real medical mobile ultrasound image" in the unpaired image dataset and specifies its fuzzy parameter value; then, based on the extracted multi-scale features and the specified fuzzy parameter value, it generates the corresponding "high-resolution reconstructed medical mobile ultrasound image". The resolution improvement of the generated "high-resolution reconstructed medical mobile ultrasound image" relative to the input "low-resolution real medical mobile ultrasound image" corresponds to the specified fuzzy parameter value; next, it extracts features from the generated "high-resolution reconstructed medical mobile ultrasound image" and generates the corresponding "low-resolution restored medical mobile ultrasound image" based on the extracted features and the specified fuzzy parameter value; finally, it compares the generated "low-resolution restored medical mobile ultrasound image" with the "low-resolution real medical mobile ultrasound image" in the unpaired image dataset and optimizes the parameters of the super-resolution reconstruction model through a composite loss function.

6. A controllable mobile ultrasound image quality improvement system, characterized in that, include: Data receiving unit, data processing unit, and data output unit; The data receiving unit is responsible for acquiring real medical mobile ultrasound images captured by the mobile ultrasound device; The data processing unit is responsible for performing the operation corresponding to the controllable mobile ultrasound image quality improvement method as described in any one of claims 1-5, and finally generating a reconstructed medical mobile ultrasound image with a corresponding super-resolution level. The data output unit is responsible for displaying the generated reconstructed medical mobile ultrasound image with the corresponding super-resolution level; The controllable mobile ultrasound image quality enhancement system is either a standalone image processing device or integrated into a mobile ultrasound device.

7. A computer device, characterized in that, include: The system includes a processor, a memory, a communication interface, and a communication bus. The processor, the memory, and the communication interface communicate with each other via the communication bus. The memory stores at least one executable instruction, which causes the processor to perform the operation corresponding to the controllable mobile ultrasound image quality improvement method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one executable instruction that causes a processor to perform an operation corresponding to the controllable mobile ultrasound image quality improvement method as described in any one of claims 1-5.

Images