A method and system for evaluating organ ultrasound image quality based on multi-layer activation response fusion deep neural network

Abstract

The present invention provides a method and system for evaluating organ ultrasound image quality based on a multi-layer activation response fusion deep neural network, comprising S1, acquiring a captured organ ultrasound image; S2, inputting the organ ultrasound image into a pre-trained deep neural network model to obtain an activation response map, which represents the fusion of activation response features of the image at different layers; and S3, performing a reduction calculation on the activation response map to obtain a quality evaluation estimate of the organ ultrasound image. The organ ultrasound image quality evaluation method of the present invention uses a deep neural network model structure and efficient data expansion methods to enable the model to capture more comprehensive features to evaluate image quality. At the same time, the model structure and a large number of positive and negative samples ensure sufficient generalization and stability of the model.

Description

Technical Field

[0001] The present invention relates to the field of image quality evaluation, and in particular to a quality evaluation method for organ ultrasound images. Background Art

[0002] In organ ultrasound image processing, after doctors complete ultrasound scans of a patient's organs and tissues, they often need to further analyze these acquired ultrasound images. Consequently, quickly selecting valuable images (e.g., how much organ and tissue information is contained) from the numerous acquired images is a valuable and practical image analysis task.

[0003] Previous image quality assessment methods were based on traditional image algorithms, resulting in relatively slow computational processes and low accuracy and stability. Some quality assessment algorithms based on deep neural networks often include more parameters or employ crude training processes. For example, some networks require performing operations such as region-of-interest (ROI) pooling on images before performing quality assessment on the ROI region. Such operations are inherently non-end-to-end and can result in low accuracy and stability.

[0004] In clinical applications, low-quality ultrasound images can lead to misdiagnosis or missed diagnosis. Therefore, establishing a scientific and reliable quality assessment mechanism is crucial to improving ultrasound diagnosis. However, researchers face a complex and pressing technical challenge: ensuring accuracy while adapting to ultrasound images collected by different organs and equipment, and ensuring the versatility and robustness of the assessment method. Summary of the Invention

[0005] The purpose of the present invention is to address the problems existing in existing image quality assessment methods and propose an organ ultrasound image quality evaluation method based on a multi-layer activation response fusion deep neural network.

[0006] The technical solution of the present invention is:

[0007] The present invention provides a method for evaluating organ ultrasound image quality based on a multi-layer activation response fusion deep neural network, the method comprising the following steps:

[0008] S1. Obtaining ultrasound images of collected organs;

[0009] S2. Inputting the organ ultrasound image into a pre-trained deep neural network model to obtain an activation response map, where the activation response map represents a fusion of activation response features of the image at different levels;

[0010] S3. Performing a reduction calculation on the activation response map to obtain a quality evaluation estimate of the organ ultrasound image.

[0011] Furthermore, S1 further includes: acquiring an organ ultrasonic segmentation image corresponding to the organ ultrasonic image, wherein the organ ultrasonic segmentation image is used to provide a mask for subsequent reduction calculation.

[0012] Furthermore, before inputting the organ ultrasound image into the pre-trained deep neural network model in S2, the method further includes: scaling and cropping the organ ultrasound image, and the size of the processed organ ultrasound image is the same as the output size of the activation response map.

[0013] Furthermore, the deep neural network model described in S2 is designed with an encoder-decoder structure, including a compression path and an expansion path;

[0014] providing a plurality of downsampling modules in the compression path to extract multi-scale features of the organ ultrasound image;

[0015] Multiple upsampling modules and skip connections are set in the expansion path to gradually restore the image resolution and fuse multi-scale features.

[0016] Furthermore, S2 specifically includes:

[0017] Inputting the preprocessed ultrasound image of the organ into a pre-trained deep neural network model, and extracting multi-scale features through an encoder compression path;

[0018] The extracted multi-scale features are restored to image resolution through the upsampling module and skip connection of the expansion path to obtain an activation response map output at each scale, wherein the activation response map represents the image features extracted by the model at this layer;

[0019] The output activation response map is fused to obtain a final activation response region.

[0020] Furthermore, in S3, the multi-layer activation response is reduced and calculated, including:

[0021] S31, performing sigmoid activation on the activation response map output by the deep neural network model to obtain a sigmoid activation value;

[0022] S32. If an organ ultrasound segmentation image exists, use the organ ultrasound segmentation image as a mask, perform probability normalization on the activation response map within the mask area, and calculate softmax to obtain a probability heat map within the mask area.

[0023] If there is no organ ultrasound segmentation image, the activation response map of the entire image is probability normalized and softmax is calculated to obtain the probability heat map of the entire image;

[0024] S33. Perform reduction calculation based on sigmoid activation value and probability heat map, and use the result as the quality evaluation estimation parameter.

[0025] Furthermore, the sigmoid activation in S31 uses the following formula:

[0026]

[0027] in, , Represents activation-response graph.

[0028] Furthermore, the calculation in S33 adopts the following formula:

[0029]

[0030] in: Indicates the pixel number, Refers to the number The sigmoid activation value of the pixel point, Refers to the number The probability of the pixel.

[0031] Furthermore, after obtaining the quality evaluation estimate of the organ ultrasound image, the following steps are performed:

[0032] Comparing the quality evaluation estimate with a preset threshold, and determining if the estimated value is greater than the threshold as a high-quality image, otherwise determining the image as a low-quality image;

[0033] Repeat the above steps for all organ ultrasound images in an examination sequence to obtain a quality evaluation estimate for each frame of the image, and calculate the average quality score of the entire sequence as the overall quality evaluation result of the examination;

[0034] The quality evaluation estimate is compared with a preset threshold value. If the quality evaluation estimate is greater than the preset threshold value, it is considered that the organ ultrasound image quality is qualified.

[0035] A system used in an organ ultrasound image quality assessment method based on a multi-layer activation response fusion deep neural network comprises:

[0036] An image acquisition module, used for acquiring ultrasound images of organs;

[0037] a deep neural network processing module, configured to input the organ ultrasound image into a pre-trained deep neural network model to obtain an activation response map, wherein the activation response map represents a fusion of activation response features of the image at different levels;

[0038] The quality evaluation calculation module is used to perform a reduction calculation on the activation response map to obtain a quality evaluation estimate of the organ ultrasound image.

[0039] Beneficial effects of the present invention:

[0040] The organ ultrasound image quality assessment method of the present invention utilizes a deep neural network model structure and efficient data augmentation methods to enable the model to capture more comprehensive features for image quality assessment. Furthermore, the model structure and the large number of positive and negative samples ensure sufficient generalization and stability.

[0041] The method of the present invention obtains ultrasound images of organs, inputs them into a pre-trained deep neural network model, generates activation response maps, and performs reduction calculations on these maps to obtain image quality estimates. Specifically, the deep neural network model, designed using an encoder-decoder structure, includes compression and expansion paths to extract and fuse multi-scale features. A reduction calculation is performed by setting a mask region to obtain a quality estimate. Finally, the quality estimate is compared with a preset threshold to determine image quality and perform a quality assessment on the entire examination sequence.

[0042] This method can effectively assess the quality of organ ultrasound images, providing reliable image quality assurance for medical diagnosis and improving the accuracy and reliability of ultrasound examinations. The method also boasts high computational speed, with the quantized model taking only 8ms to infer on an NVIDIA graphics card.

[0043] The method of the present invention directly performs activation response fusion in the decoding stage from low resolution to high resolution without introducing any additional operations such as ROI (full-image operation). The method only requires the network model to learn the representation from fuzzy 0 to clear 1, involves few parameters, and the training process is efficient and reliable.

[0044] Other features and advantages of the present invention will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The above and other objects, features and advantages of the present invention will become more apparent through a more detailed description of exemplary embodiments of the present invention with reference to the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present invention.

[0046] Figure 1 A flow chart of the organ ultrasound image quality assessment method based on a multi-layer activation response fusion deep neural network of the present invention is shown;

[0047] Figure 2 A schematic diagram of the structure of the deep neural network model of the present invention is shown;

[0048] Figure 3 A schematic diagram of reduced calculation in an embodiment of the present invention is shown;

[0049] Figure 4 A schematic diagram showing the quality evaluation results of a batch of continuously acquired ultrasound images in an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0050] The preferred embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the preferred embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0051] Example 1

[0052] Figure 1 The flowchart of the organ ultrasound image quality assessment method based on a multi-layer activation response fusion deep neural network of the present invention is shown.

[0053] like Figure 1 As shown, the present invention provides a method for evaluating organ ultrasound image quality based on a multi-layer activation response fusion deep neural network, comprising:

[0054] S1. Obtaining ultrasound images of collected organs;

[0055] S2. Inputting the organ ultrasound image into a pre-trained deep neural network model to obtain an activation response map, where the activation response map represents a fusion of activation response features of the image at different levels;

[0056] S3. Performing a reduction calculation on the activation response map to obtain a quality evaluation estimate of the organ ultrasound image.

[0057] In this embodiment, by acquiring high-quality ultrasound images, a reliable data source can be provided for subsequent deep neural network processing; the organ ultrasound image is input into a pre-trained deep neural network model to obtain a multi-layer activation response. The activation response is the response of the network to the input image at each level, reflecting the feature representation of the image at that level. Through the multi-layer activation response, the network can fully capture the feature information of the image. The activation response graph is reduced and a quality evaluation estimate of the organ ultrasound image is obtained. The reduction calculation is a process of summarizing or compressing the multi-layer activation response, the purpose of which is to convert the complex features of multiple levels into a single quality score, which can reasonably integrate the features of different levels to ensure the accuracy and stability of the score. Through the reduction calculation, the quality of the ultrasound image can be quickly evaluated, helping doctors to screen out valuable images.

[0058] In one example, an organ ultrasound segmentation image corresponding to the organ ultrasound image is obtained, and the organ ultrasound segmentation image is used to provide a template for subsequent reduction calculation.

[0059] In this embodiment, the organ ultrasound segmentation image serves as a template for subsequent reduction calculations. The segmented image precisely delineates the organ region within the original ultrasound image, clearly identifying the organ's boundaries and internal structure. This distinction not only provides a clear regional scope for subsequent quality assessment but also prevents non-target regions from interfering with the evaluation results.

[0060] Using the segmented image as a template provides a foundation for reduction calculations. Reduction calculations require statistical or feature extraction of specific regions within the image. Segmented images ensure these calculations are performed only on the target organ region, eliminating the influence of background noise and significantly improving the accuracy and stability of quality assessments.

[0061] In one example, the organ ultrasound image is scaled and cropped, and the size of the processed organ ultrasound image is the same as the output size of the activation response map.

[0062] In this embodiment, scaling ensures spatial alignment between the input image and the network model's output, facilitating subsequent activation response calculations. By uniformly processing ultrasound images of varying sizes and resolutions to the same input size, the model's sensitivity to varying input sizes is reduced, thereby improving its performance on diverse datasets. Furthermore, cropping removes noise and irrelevant information from the image, allowing the model to focus on learning features in the target region, further enhancing its stability and robustness.

[0063] In one example, the deep neural network model is designed using an encoder-decoder structure, including a compression path and an expansion path;

[0064] providing a plurality of downsampling modules in the compression path to extract multi-scale features of the organ ultrasound image;

[0065] Multiple upsampling modules and skip connections are set in the expansion path to gradually restore the image resolution and fuse multi-scale features. The processing performed includes:

[0066] Inputting the preprocessed ultrasound image of the organ into a pre-trained deep neural network model, and extracting multi-scale features through an encoder compression path;

[0067] The extracted multi-scale features are restored to image resolution through the upsampling module and skip connection of the expansion path to obtain an activation response map output at each scale, wherein the activation response map represents the image features extracted by the model at this layer;

[0068] The output activation response map is fused to obtain a final activation response region.

[0069] In this embodiment, when preprocessed organ ultrasound images are fed into a pre-trained deep neural network model, the model first extracts multi-scale features through an encoder compression path. The encoder consists of multiple convolutional layers and pooling layers. The convolutional layers extract local features from the image, while the pooling layers gradually reduce the resolution of the feature map through downsampling while retaining important feature information. This multi-scale feature extraction method captures detailed information at different levels of the image, providing a rich feature foundation for subsequent quality assessment.

[0070] After extracting multi-scale features, the image resolution is restored through the upsampling module and skip connections in the extended path. The upsampling module uses deconvolution or interpolation operations to gradually restore the low-resolution feature map to the original image resolution. The skip connection fuses the feature maps from the encoder path with those from the decoder path, allowing the model to restore the image resolution while preserving the multi-scale features extracted in the encoder path. This effectively avoids information loss and enhances the model's ability to capture image details.

[0071] When fusing the output activation response maps, activation response maps at different scales are combined using a weighted summation or cascading approach. This multi-layer fusion model comprehensively considers both local details and global structure in the image, resulting in the final activation response region. The final activation response region is represented as a heat map, with highlighted areas in the heat map indicating areas that the model considers to be most significant for image quality assessment.

[0072] In one example, performing a reduction calculation on the multi-layer activation response in S3 includes:

[0073] S31, perform sigmoid activation on the activation response map output by the deep neural network model, obtain the sigmoid activation value, and perform full-map operation, with the value ranging from 0 to 1;

[0074]

[0075] in, , represents the activation response map;

[0076] S32: If there is an organ ultrasound segmentation image, use the organ ultrasound segmentation image as a mask and perform probability normalization calculation on the activation response map within the mask area. softmax , get the probability heat map in the mask area;

[0077] If there is no organ ultrasound segmentation image, the activation response map of the whole image is calculated by probability normalization. softmax , get the probability heat map of the whole image;

[0078] S33, perform reduction calculation based on sigmoid activation value and probability heat map, and use the result as the quality evaluation estimation parameter;

[0079]

[0080] in: Indicates the pixel number, Refers to the number The sigmoid activation value of the pixel point, Refers to the number The probability of the pixel.

[0081] In this embodiment, if an organ ultrasound segmentation image exists, the calculation is performed based on the pre-provided segmentation image. The mask area can focus on the key area in the image and exclude interference from irrelevant areas, thereby improving the accuracy of the evaluation.

[0082] If there is no organ ultrasound segmentation image, automatic analysis is performed based on the activation response of the entire image, and quality evaluation can be completed even without a segmented image.

[0083] In real-world applications, the presence and absence of segmented images may alternate, and the system must be able to flexibly switch between them. For example, in a hospital's ultrasound image processing workflow, some images may already have segmentation annotations, while others lack segmentation information. The system needs to be able to automatically perform this evaluation based on the characteristics of the input images, ensuring consistency and accuracy in quality assessment, significantly improving the algorithm's applicability and practicality.

[0084] In one example, after obtaining the quality evaluation estimate of the organ ultrasound image, the following steps are performed:

[0085] Comparing the quality evaluation estimate with a preset threshold, and determining if the estimated value is greater than the threshold as a high-quality image, otherwise determining the image as a low-quality image;

[0086] Repeat the above steps for all organ ultrasound images in an examination sequence to obtain a quality evaluation estimate for each frame of the image, and calculate the average quality score of the entire sequence as the overall quality evaluation result of the examination;

[0087] The quality evaluation estimate is compared with a preset threshold value. If the quality evaluation estimate is greater than the preset threshold value, it is considered that the organ ultrasound image quality is qualified.

[0088] In this embodiment, after obtaining an estimated quality evaluation of an organ ultrasound image, it is first compared with a preset threshold. The preset threshold is typically set based on medical imaging quality standards and actual application requirements. For example, a threshold of 0.7 is set. If the estimated quality evaluation value is greater than 0.7, the image is considered high-quality; otherwise, it is considered low-quality. This allows for rapid screening of images that meet quality requirements for subsequent analysis and processing.

[0089] The above comparison steps are repeated for all organ ultrasound images in an examination sequence to obtain a quality evaluation estimate for each frame. The average quality score for the entire sequence is calculated as the overall quality assessment result for the examination. This helps to comprehensively evaluate the quality of the entire examination sequence, ensure the reliability and accuracy of the examination results, and provide doctors with more accurate diagnosis basis.

[0090] While various embodiments of the present invention have been described above, the above description is intended to be illustrative, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

Claims

1. A method for organ ultrasound image quality assessment based on multi-layer activation response fusion deep neural network, characterized in that: The method comprises the following steps: S1. Obtaining ultrasound images of collected organs; S2. Inputting the organ ultrasound image into a pre-trained deep neural network model to obtain an activation response map, where the activation response map represents a fusion of activation response features of the image at different levels; S3. performing a reduction calculation on the activation response map to obtain a quality evaluation estimate of the organ ultrasound image; In S2, the deep neural network model adopts an encoder-decoder structure design, including a compression path and an expansion path; multiple downsampling modules are set in the compression path to extract multi-scale features of the organ ultrasound image; multiple upsampling modules and skip connections are set in the expansion path to gradually restore image resolution and fuse multi-scale features; S2 specifically includes: Inputting the preprocessed ultrasound image of the organ into a pre-trained deep neural network model, and extracting multi-scale features through an encoder compression path; The extracted multi-scale features are restored to image resolution through the upsampling module and skip connection of the expansion path to obtain an activation response map output at each scale, wherein the activation response map represents the image features extracted by the model at this layer; Performing fusion processing on the output activation response map to obtain a final activation response area; In S3, the multi-layer activation response is reduced and calculated, including: S31. Perform sigmoid activation on the activation response graph output by the deep neural network model to obtain a sigmoid activation value. The sigmoid activation uses the following formula: ； in, , represents the activation response map; S32: If there is an organ ultrasound segmentation image, use the organ ultrasound segmentation image as a mask and perform probability normalization calculation on the activation response map within the mask area. softmax , get the probability heat map in the mask area; If there is no organ ultrasound segmentation image, the activation response map of the whole image is calculated by probability normalization. softmax , get the probability heat map of the whole image; S33. Perform a reduction calculation based on the sigmoid activation value and the probability heat map, and use the result as the quality evaluation estimation parameter; the reduction calculation uses the following formula: ； in: Indicates the pixel number, Refers to the number The sigmoid activation value of the pixel point, Refers to the number The probability of the pixel.

2. The organ ultrasound image quality assessment method based on a multi-layer activation response fusion deep neural network as claimed in claim 1, characterized in that S1 also includes: acquiring an organ ultrasonic segmentation image corresponding to the organ ultrasonic image, wherein the organ ultrasonic segmentation image is used to provide a mask for subsequent reduction calculation.

3. The organ ultrasound image quality assessment method based on a multi-layer activation response fusion deep neural network as claimed in claim 1, characterized in that In S2, before inputting the organ ultrasound image into the pre-trained deep neural network model, the method further includes: scaling and cropping the organ ultrasound image, and the size of the processed organ ultrasound image is the same as the output size of the activation response map.

4. The organ ultrasound image quality assessment method based on a multi-layer activation response fusion deep neural network as claimed in claim 1, characterized in that After obtaining the quality evaluation estimate of the organ ultrasound image, the following steps are performed: Comparing the quality evaluation estimate with a preset threshold, and determining if the estimated value is greater than the threshold as a high-quality image, otherwise determining the image as a low-quality image; Repeat the above steps for all organ ultrasound images in an examination sequence to obtain a quality evaluation estimate for each frame of the image, and calculate the average quality score of the entire sequence as the overall quality evaluation result of the examination; The quality evaluation estimate is compared with a preset threshold value. If the quality evaluation estimate is greater than the preset threshold value, it is considered that the organ ultrasound image quality is qualified.

5. A system used in the organ ultrasound image quality assessment method based on a multi-layer activation response fusion deep neural network according to any one of claims 1 to 4, characterized in that: include: An image acquisition module, used for acquiring ultrasound images of organs; a deep neural network processing module, configured to input the organ ultrasound image into a pre-trained deep neural network model to obtain an activation response map, wherein the activation response map represents a fusion of activation response features of the image at different levels; The quality evaluation calculation module is used to perform a reduction calculation on the activation response map to obtain a quality evaluation estimate of the organ ultrasound image.

Images