Ultrasonic image quality evaluation method, program product and electronic equipment

Abstract

The invention provides an ultrasonic image quality evaluation method, a program product and electronic equipment, which are applied to the technical field of image processing, and the method comprises the following steps: obtaining a frame of 2D ultrasonic image, multiple frames of 3D ultrasonic images and clinical text data of a patient; inputting the 2D ultrasonic image, the 3D ultrasonic image and the clinical text data of the patient into a quality evaluation model to obtain a quality evaluation score corresponding to a target ultrasonic image output by the quality evaluation model; wherein the quality evaluation model comprises a plurality of evaluation sub-networks, each evaluation sub-network is used for determining an initial score of the 3D ultrasonic image in one dimension, and the quality evaluation score is determined based on the plurality of initial scores. The multi-modal data is processed, so that the accuracy of quality evaluation is improved; besides, the quality evaluation network can realize multi-dimension-based image quality scoring, and quality evaluation is carried out on each image by integrating multiple dimensions, so that a more comprehensive image quality reference can be provided.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method for evaluating the quality of ultrasound images, a program product, and an electronic device. Background Art

[0002] In recent years, ultrasound imaging, as a non-invasive, real-time diagnostic technology, has been widely used in clinical medicine, especially in obstetrics and gynecology, cardiology, hepatobiliary, urology and other fields. Ultrasound examination has become one of the important diagnostic methods in primary medical institutions because of its simple operation, low cost and real-time feedback of diagnostic information. However, the difference in ultrasound image quality is a key factor affecting the diagnostic effect. The quality of ultrasound images is affected by many factors, including the doctor's operating skills, scanning angle, equipment performance and patient's body shape, which makes the success rate of standard section acquisition low in actual clinical operations, resulting in unstable image quality, which in turn affects the final diagnosis.

[0003] In primary care institutions, due to doctors' relatively inexperience, improper operation or inadequate understanding of standard sections, misdiagnosis or missed diagnosis often occurs, especially in standard section acquisition and image quality assessment. Doctors may miss the best section or fail to find key anatomical structures in the image in time, thus affecting the early diagnosis and treatment of the disease. Although ultrasound examination technology itself is constantly developing, how to ensure image quality, reduce interference from human factors and improve work efficiency is still a huge challenge facing primary care institutions.

[0004] Traditional ultrasound image quality assessment mostly relies on the doctor's personal experience and subjective judgment, which is easily affected by the doctor's proficiency and technical level, and the evaluation of image quality during the diagnosis process often takes a long time. In addition, the acquisition of standard sections also mainly relies on the doctor's manual adjustment and shooting, which not only increases the patient's examination burden in actual operation, but also makes the diagnosis and treatment process cumbersome and inefficient. Therefore, the accuracy of ultrasound image quality assessment in the existing technology is low. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a method, a program product, and an electronic device for evaluating the quality of an ultrasound image, so as to solve the technical problem of low accuracy in evaluating the quality of an ultrasound image in the prior art.

[0006] In a first aspect, an embodiment of the present application provides a method for quality assessment of an ultrasound image, comprising: acquiring a frame of 2D ultrasound image, multiple frames of 3D ultrasound images, and patient clinical text data; inputting the 2D ultrasound image, the 3D ultrasound image, and the patient clinical text data into a quality assessment model to obtain a quality assessment score corresponding to a target ultrasound image output by the quality assessment model; wherein the quality assessment model comprises multiple evaluation subnetworks, each evaluation subnetwork being used to determine an initial score of the 3D ultrasound image in one dimension, and the quality assessment score being determined based on multiple initial scores.

[0007] In the above scheme, when performing quality assessment on ultrasound images, 2D ultrasound images, multi-frame 3D ultrasound images and patient clinical text data can be simultaneously input into the quality assessment network, and the accuracy of quality assessment can be improved by processing multimodal data; in addition, the quality assessment network can realize multi-dimensional image quality scoring, and perform quality assessment on each image by comprehensively evaluating clarity, anatomical integrity, noise level, contrast and other dimensions, thereby providing a more comprehensive image quality reference.

[0008] In an optional embodiment, the quality assessment model includes: a feature extraction module, a feature fusion module and a quality assessment module; the 2D ultrasound image, the 3D ultrasound image and the patient clinical text data are input into the quality assessment model to obtain the quality assessment score corresponding to the target ultrasound image output by the quality assessment model, including: using the feature extraction module to extract the first feature corresponding to the 2D ultrasound image, the second feature corresponding to the 3D ultrasound image and the third feature corresponding to the patient clinical text data; using the feature fusion module to fuse the first feature, the second feature and the third feature to obtain a fusion feature; using the quality assessment module to determine the quality assessment score corresponding to each frame of the 3D ultrasound image according to the fusion feature; determining the 3D ultrasound image with the highest quality assessment score as the target ultrasound image, and outputting the quality assessment score corresponding to the target ultrasound image. In the above scheme, the fusion feature can be obtained by extracting the features of the 2D ultrasound image, the 3D ultrasound image and the patient clinical text data and fusing the above features; based on the above fusion feature, it is convenient to perform quality assessment on the image to obtain a quality assessment result with a high accuracy rate.

[0009] In an optional embodiment, the feature extraction module includes: a convolutional neural network, a 3D convolutional neural network and a word embedding model; the use of the feature extraction module to extract the first feature corresponding to the 2D ultrasound image, the second feature corresponding to the 3D ultrasound image and the third feature corresponding to the patient's clinical text data includes: inputting the 2D ultrasound image into the convolutional neural network to obtain the first feature output by the convolutional neural network; and, inputting the 3D ultrasound image into the 3D convolutional neural network to obtain the second feature output by the 3D convolutional neural network; and, inputting the patient's clinical text data into the word embedding model to obtain the third feature output by the word embedding model. In the above scheme, a convolutional neural network can be used to extract the first feature corresponding to the 2D ultrasound image. The convolutional neural network can capture the shape, boundary and structural information of the organ through layer-by-layer convolution, and further provide support for subsequent organ classification and key anatomical structure segmentation; a 3D convolutional neural network can be used to extract the second feature corresponding to the 3D ultrasound image. By applying the 3D convolutional layer, it can not only identify the organs in a single frame image, but also learn the change pattern of the organs in continuous video frames, thereby improving the accuracy of quality assessment; the word embedding model can be used to extract the third feature corresponding to the patient's clinical text data, thereby extracting key information that is helpful for image quality assessment.

[0010] In an optional embodiment, the fusing the first feature, the second feature and the third feature to obtain a fused feature includes: fusing the first feature, the second feature and the third feature based on a cross-modal attention mechanism. In the above scheme, the cross-modal attention mechanism can effectively learn how to weight and select features of each modality by modeling the relationship between data of different modalities, thereby improving the efficiency of feature fusion.

[0011] In an optional embodiment, the quality assessment module includes: an organ classification model, an image segmentation model and the multiple assessment subnetworks; the quality assessment score corresponding to each frame of 3D ultrasound image is determined according to the fusion feature, including: using the organ classification model to classify the 3D ultrasound image according to the fusion feature to obtain the organ category corresponding to each frame of 3D ultrasound image; using the image segmentation model to segment the key anatomical structure of the 3D ultrasound image according to the fusion feature and the organ category to obtain the segmentation feature corresponding to each frame of 3D ultrasound image; using the assessment subnetwork to determine the initial score of each frame of 3D ultrasound image according to the segmentation feature; and determining the quality assessment score corresponding to each frame of 3D ultrasound image based on the initial score. In the above scheme, the quality assessment module can first classify the input ultrasound image, accurately identify the specific organs involved in the image, and provide a basis for subsequent quality assessment and anatomical structure segmentation; based on the organ classification, the quality assessment module can further segment the key anatomical structure to improve the accuracy and reliability of quality assessment; finally, through multi-dimensional image quality scoring, a more comprehensive image quality reference is provided.

[0012] In an optional embodiment, the ultrasound image quality assessment method further includes: training a neural network model using the following steps to obtain the quality assessment model: obtaining a sample data set, wherein the sample data set includes a 2D sample image, a 3D sample image, and sample text data; inputting the sample data set into the neural network model to obtain a loss value corresponding to the neural network model; optimizing the neural network model according to the loss value to obtain the quality assessment model. In the above scheme, the sample data set can be used to train the neural network model to obtain a quality assessment model, and then the above quality assessment model can be used to perform quality assessment on the image.

[0013] In an optional embodiment, the loss value includes at least one of the following: an organ classification loss value, an anatomical structure segmentation loss value, an image retention comparison loss value, a quality score loss value, and an organ position loss value. In the above scheme, by calculating the above organ classification loss value, anatomical structure segmentation loss value, image retention comparison loss value, quality score loss value, and organ position loss value, the neural network model can be optimized to obtain a better quality assessment model.

[0014] In a second aspect, an embodiment of the present application provides an ultrasound image quality assessment device, comprising: an acquisition module, used to acquire a frame of 2D ultrasound image, multiple frames of 3D ultrasound images and patient clinical text data; an input module, used to input the 2D ultrasound image, the 3D ultrasound image and the patient clinical text data into a quality assessment model to obtain a quality assessment score corresponding to a target ultrasound image output by the quality assessment model; wherein the quality assessment model comprises multiple evaluation subnetworks, each evaluation subnetwork is used to determine an initial score of the 3D ultrasound image in one dimension, and the quality assessment score is determined based on multiple initial scores.

[0015] In the above scheme, when performing quality assessment on ultrasound images, 2D ultrasound images, multi-frame 3D ultrasound images and patient clinical text data can be simultaneously input into the quality assessment network, and the accuracy of quality assessment can be improved by processing multimodal data; in addition, the quality assessment network can realize multi-dimensional image quality scoring, and perform quality assessment on each image by comprehensively evaluating clarity, anatomical integrity, noise level, contrast and other dimensions, thereby providing a more comprehensive image quality reference.

[0016] In an optional embodiment, the quality assessment model includes: a feature extraction module, a feature fusion module and a quality assessment module; the input module is specifically used to: use the feature extraction module to extract the first feature corresponding to the 2D ultrasound image, the second feature corresponding to the 3D ultrasound image and the third feature corresponding to the patient's clinical text data; use the feature fusion module to fuse the first feature, the second feature and the third feature to obtain a fusion feature; use the quality assessment module to determine the quality assessment score corresponding to each frame of the 3D ultrasound image according to the fusion feature; determine the 3D ultrasound image with the highest quality assessment score as the target ultrasound image, and output the quality assessment score corresponding to the target ultrasound image. In the above scheme, the fusion feature can be obtained by extracting the features of the 2D ultrasound image, the 3D ultrasound image and the patient's clinical text data and fusing the above features; based on the above fusion feature, it is convenient to perform quality assessment on the image to obtain a quality assessment result with a higher accuracy rate.

[0017] In an optional embodiment, the feature extraction module includes: a convolutional neural network, a 3D convolutional neural network and a word embedding model; the input module is also used to: input the 2D ultrasound image into the convolutional neural network to obtain the first feature output by the convolutional neural network; and, input the 3D ultrasound image into the 3D convolutional neural network to obtain the second feature output by the 3D convolutional neural network; and, input the patient's clinical text data into the word embedding model to obtain the third feature output by the word embedding model. In the above scheme, the first feature corresponding to the 2D ultrasound image can be extracted by using a convolutional neural network, which can capture the shape, boundary and structural information of the organ through layer-by-layer convolution, and further provide support for subsequent organ classification and key anatomical structure segmentation; the second feature corresponding to the 3D ultrasound image can be extracted by using a 3D convolutional neural network, and by applying a 3D convolutional layer, not only can the organ in a single frame be identified, but also the change pattern of the organ in continuous video frames can be learned, thereby improving the accuracy of quality assessment; the third feature corresponding to the patient's clinical text data can be extracted by using a word embedding model, thereby extracting key information that helps to assess image quality.

[0018] In an optional embodiment, the input module is further used to: fuse the first feature, the second feature and the third feature based on a cross-modal attention mechanism. In the above scheme, the cross-modal attention mechanism can effectively learn how to weight and select features of each modality by modeling the relationship between different modal data, thereby improving the efficiency of feature fusion.

[0019] In an optional embodiment, the quality assessment module includes: an organ classification model, an image segmentation model and the multiple assessment subnetworks; the input module is also used to: use the organ classification model to classify the 3D ultrasound image according to the fusion features to obtain the organ category corresponding to each frame of the 3D ultrasound image; use the image segmentation model to segment the key anatomical structure of the 3D ultrasound image according to the fusion features and the organ category to obtain the segmentation features corresponding to each frame of the 3D ultrasound image; use the assessment subnetwork to determine the initial score of each frame of the 3D ultrasound image according to the segmentation features; and determine the quality assessment score corresponding to each frame of the 3D ultrasound image based on the initial score. In the above scheme, the quality assessment module can first classify the input ultrasound image, accurately identify the specific organs involved in the image, and provide a basis for subsequent quality assessment and anatomical structure segmentation; based on the organ classification, the quality assessment module can further segment the key anatomical structure to improve the accuracy and reliability of the quality assessment; finally, through multi-dimensional image quality scoring, a more comprehensive image quality reference is provided.

[0020] In an optional embodiment, the ultrasound image quality assessment device further includes: a training module for training a neural network model using the following steps to obtain the quality assessment model: obtaining a sample data set, wherein the sample data set includes a 2D sample image, a 3D sample image, and sample text data; inputting the sample data set into the neural network model to obtain a loss value corresponding to the neural network model; optimizing the neural network model according to the loss value to obtain the quality assessment model. In the above scheme, the sample data set can be used to train the neural network model to obtain a quality assessment model, and then the above quality assessment model can be used to perform quality assessment on the image.

[0021] In an optional embodiment, the loss value includes at least one of the following: an organ classification loss value, an anatomical structure segmentation loss value, an image retention comparison loss value, a quality score loss value, and an organ position loss value. In the above scheme, by calculating the above organ classification loss value, anatomical structure segmentation loss value, image retention comparison loss value, quality score loss value, and organ position loss value, the neural network model can be optimized to obtain a better quality assessment model.

[0022] In a third aspect, an embodiment of the present application provides a computer program product, comprising computer program instructions, which, when read and executed by a processor, execute the ultrasound image quality assessment method as described in the first aspect.

[0023] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a memory and a bus; the processor and the memory communicate with each other through the bus; the memory stores computer program instructions that can be executed by the processor, and the processor calls the computer program instructions to execute the ultrasound image quality assessment method as described in the first aspect.

[0024] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer program instructions. When the computer program instructions are executed by a computer, the computer executes the ultrasound image quality assessment method as described in the first aspect.

[0025] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the following specifically cites the embodiments of the present application and describes them in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0027] Figure 1 A flowchart of a method for evaluating the quality of an ultrasound image provided in an embodiment of the present application; Figure 2 A structural block diagram of an ultrasound image quality assessment device provided in an embodiment of the present application; Figure 3 A structural block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0029] Please refer to Figure 1 , Figure 1 A flowchart of a method for evaluating the quality of an ultrasound image provided in an embodiment of the present application, the method for evaluating the quality of an ultrasound image may specifically include the following steps: Step S101: Acquire a frame of 2D ultrasound image, multiple frames of 3D ultrasound images, and clinical text data of the patient.

[0030] Step S102: Input the 2D ultrasound image, the 3D ultrasound image and the patient's clinical text data into the quality assessment model to obtain a quality assessment score corresponding to the target ultrasound image output by the quality assessment model.

[0031] Specifically, in the above step S101, the 2D ultrasound image is a two-dimensional plane image collected from the patient using ultrasound technology, and the 3D ultrasound image is a three-dimensional stereoscopic image collected from the patient using ultrasound technology; wherein the above 3D ultrasound image may be a number of video frames extracted from a 3D ultrasound video, and the above 2D ultrasound image may be a cross-sectional image of a certain video frame.

[0032] The patient clinical text data is clinical data related to the patient, such as patient medical history, patient age, patient gender, etc. Among them, the 2D ultrasound image, 3D ultrasound image and patient clinical text data are all data corresponding to the same patient.

[0033] As an implementation method, the above-mentioned 2D ultrasound images, 3D ultrasound images and patient clinical text data may be preprocessed. The present application embodiment does not specifically limit the specific implementation method of the preprocessing, and those skilled in the art may make appropriate adjustments according to actual conditions. The implementation method of the preprocessing is introduced by way of example below.

[0034] For a 2D ultrasound image, preprocessing may include at least one of the following: denoising, image standardization, image enhancement, size adjustment, cropping, and region of interest (ROI) extraction.

[0035] Among them, denoising: 2D ultrasound images usually contain noise, which may be caused by factors such as equipment and patient position changes. Therefore, filters (such as Gaussian filtering, mean filtering, etc.) can be used to remove noise and smooth the image to improve the effect of subsequent feature extraction. Image standardization: Since different equipment, settings, and patient conditions may cause differences in image contrast and brightness, 2D ultrasound images can be standardized; for example, through mean-variance normalization, pixel values ​​are normalized to a certain range (such as between 0 and 1) to reduce the difference between different images. Image enhancement: For images with low contrast or uneven illumination, image enhancement methods (such as histogram equalization, CLAHE, etc.) can be applied to improve image details and contrast, especially the performance of key anatomical structures. Resizing: The image sizes of different devices and different patients may be inconsistent. Therefore, all images can be resized to a uniform size (for example, 224x224, etc.) so that the network can process them. Cropping and region of interest extraction: In order to reduce the interference of irrelevant background information, cropping methods can be used to extract regions of interest from images, focusing on organs and their key structural areas.

[0036] For 3D ultrasound images, preprocessing may include at least one of the following: frame extraction, alignment and resampling, spatial normalization, image enhancement and standardization.

[0037] Among them, frame extraction: 3D ultrasound video consists of a series of continuous frames, so several frames can be extracted from the video (such as 5-10 frames per second) to generate candidate 3D ultrasound images for evaluation. Alignment and resampling: 3D ultrasound images generated by different devices may have slight spatial inconsistencies, so these frames can be aligned; this process includes operations such as rotation, translation, and scaling to make the spatial distribution of all frames as consistent as possible, so as to better perform subsequent frame evaluation and comparison. Spatial normalization: In 3D ultrasound images, the viewing angle and shooting distance may be different, resulting in changes in the position of organs in the image. Therefore, the position of organs in 3D space can be unified by using standardization techniques (such as stereo geometric models or spatial registration algorithms, etc.) to ensure the comparability of multi-frame data. Image enhancement and standardization: Like 2D ultrasound images, 3D ultrasound images can also be denoised, contrast enhanced, and standardized to ensure clear organ details and uniform lighting in the frame. The enhancement process is particularly important for low-quality frames or frames with severe motion blur.

[0038] For patient clinical text data, preprocessing may include at least one of the following: text cleaning, text encoding, normalization and standardization.

[0039] Among them, text cleaning: The patient's clinical text data may be input in free text or structured form, so these data can be cleaned, for example, removing irrelevant information (such as extra spaces, punctuation marks, etc.), standardizing medical terms, unifying data formats (such as unifying "hypertension" as "Hypertension"), etc. Text encoding: Since text data cannot be directly input into the neural network, it can be converted into a format that the machine can understand, for example, using natural language processing (NLP) technology (such as word embedding (Word2Vec, GloVe), BERT, etc.), or using simple One-Hot encoding or TF-IDF methods to convert text data into numerical features. Normalization and standardization: For numerical ultrasound clinical text data (such as age, weight, etc.), normalization processing (such as Z-score standardization, etc.) can be performed to eliminate the impact of different data ranges on model training, and to ensure that the features of each modality are on the same scale, avoiding the adverse effects of data range differences on model training, and improving training efficiency and stability.

[0040] It should be noted that the embodiments of the present application do not specifically limit the specific implementation methods for obtaining the above-mentioned 2D ultrasound images, 3D ultrasound images, and patient clinical text data, and those skilled in the art may make appropriate adjustments according to actual conditions. For example, a 2D ultrasound image, 3D ultrasound image, or patient clinical text data sent by an external device may be received; or a 2D ultrasound image, 3D ultrasound image, or patient clinical text data pre-stored locally or in the cloud may be read; or a 2D ultrasound image, 3D ultrasound image, or patient clinical text data may be collected in real time.

[0041] It can be understood that in the embodiment of the present application, the methods of acquiring the above-mentioned 2D ultrasound images, 3D ultrasound images and patient clinical text data can be the same or different.

[0042] In the above step S102, the quality assessment model includes multiple assessment subnetworks, each of which is used to determine an initial score of the 3D ultrasound image in one dimension, and the quality assessment score is determined based on the multiple initial scores. The embodiment of the present application does not specifically limit the specific dimensions of the above quality assessment, and those skilled in the art can make appropriate adjustments according to actual conditions, such as: the number and quality of anatomical structures, organ location, image clarity, etc.

[0043] It should be noted that the embodiments of the present application do not specifically limit the specific implementation method of determining the quality assessment score based on multiple initial scores, and those skilled in the art can make appropriate adjustments according to actual conditions. For example, the sum of multiple initial scores can be calculated as the quality assessment score; or the average of multiple initial scores can be calculated as the quality assessment score; or the weighted average of multiple initial scores can be calculated as the quality assessment score, etc.

[0044] By inputting the 3D ultrasound image, 3D ultrasound image and patient clinical text data obtained in step S101 into the above-mentioned quality assessment model, the quality assessment score corresponding to the target ultrasound image output by the quality assessment model can be obtained. As an implementation method, the above-mentioned target ultrasound image can be a frame of 3D ultrasound image with the highest quality assessment score among multiple frames of 3D ultrasound images.

[0045] In the above scheme, when performing quality assessment on ultrasound images, 2D ultrasound images, multi-frame 3D ultrasound images and patient clinical text data can be simultaneously input into the quality assessment network, and the accuracy of quality assessment can be improved by processing multimodal data; in addition, the quality assessment network can realize multi-dimensional image quality scoring, and perform quality assessment on each image by comprehensively evaluating clarity, anatomical integrity, noise level, contrast and other dimensions, thereby providing a more comprehensive image quality reference.

[0046] Further, based on the above embodiment, the above quality assessment model may include: a feature extraction module, a feature fusion module and a quality assessment module, wherein the above feature extraction module is used to extract features of 2D ultrasound images, 3D ultrasound images and patient clinical text data, the feature fusion module is used to fuse the above extracted features, and the quality assessment module is used to perform quality assessment on 3D ultrasound images based on the fused features. Specifically, the above step S102 may include the following steps: Step 1) Use a feature extraction module to extract a first feature corresponding to the 2D ultrasound image, a second feature corresponding to the 3D ultrasound image, and a third feature corresponding to the patient's clinical text data.

[0047] Step 2) Use the feature fusion module to fuse the first feature, the second feature and the third feature to obtain a fused feature.

[0048] Step 3) Use the quality assessment module to determine the quality assessment score corresponding to each frame of the 3D ultrasound image according to the fusion features.

[0049] In step 4, the 3D ultrasound image with the highest quality assessment score is determined as the target ultrasound image, and the quality assessment score corresponding to the target ultrasound image is output.

[0050] Specifically, in the above step 1), the embodiment of the present application does not specifically limit the specific implementation method of extracting the first feature corresponding to the 2D ultrasound image, the second feature corresponding to the 3D ultrasound image, and the third feature corresponding to the patient's clinical text data. Those skilled in the art can make appropriate adjustments according to actual conditions. For example, a convolutional neural network can be used to extract the above features; or, NLP technology can be used to extract the above features, etc.

[0051] In the above step 2), the embodiment of the present application does not specifically limit the specific implementation method of fusing the above first feature, the second feature and the third feature, and those skilled in the art can make appropriate adjustments according to actual conditions. For example, the above first feature, the second feature and the third feature can be spliced; or the above first feature, the second feature and the third feature can be weighted fused; or the above first feature, the second feature and the third feature can be fused based on the cross-modal attention mechanism, etc.

[0052] In the above steps 3) and 4), the quality assessment module can be used to determine the quality assessment score corresponding to each frame of the 3D ultrasound image according to the fusion features, and then the 3D ultrasound image with the highest quality assessment score is determined as the target ultrasound image, and the quality assessment score corresponding to the target ultrasound image is output.

[0053] In the above scheme, the features of 2D ultrasound images, 3D ultrasound images and patient clinical text data can be extracted and fused to obtain fused features; based on the fused features, the image quality can be easily evaluated to obtain a quality evaluation result with a higher accuracy.

[0054] Further, based on the above embodiment, the feature extraction module may include: a convolutional neural network, a 3D convolutional neural network and a word embedding model, wherein the convolutional neural network is used to extract a first feature corresponding to a 2D ultrasound image, the 3D convolutional neural network is used to extract a second feature corresponding to a 3D ultrasound image, and the word embedding model is used to extract a third feature corresponding to the patient's clinical text data. Specifically, the step of using the feature extraction module to extract the first feature corresponding to the 2D ultrasound image, the second feature corresponding to the 3D ultrasound image, and the third feature corresponding to the patient's clinical text data may include the following steps: Step 1) Input the 2D ultrasound image into the convolutional neural network to obtain the first feature of the convolutional neural network output.

[0055] Step 2), input the 3D ultrasound image into the 3D convolutional neural network to obtain the second feature output by the 3D convolutional neural network.

[0056] Step 3) Input the patient's clinical text data into the word embedding model to obtain the third feature output by the word embedding model.

[0057] Specifically, in step 1) above, the classic Convolutional Neural Network (CNN) structure (such as ResNet, DenseNet, etc.) can be used to extract low-level and high-level features of 2D ultrasound images. By stacking multiple convolutional layers, pooling layers, and fully connected layers, the network can automatically learn features such as texture, edges, and anatomical structures in the image. Among them, for organs in 2D ultrasound images, the network can capture the shape, boundary, and structural information of the organs through layer-by-layer convolution, further providing support for subsequent organ classification and key anatomical structure segmentation.

[0058] It should be noted that the embodiments of the present application do not specifically limit the specific structure of the above-mentioned convolutional neural network, and those skilled in the art can make appropriate adjustments based on the existing technology.

[0059] In step 2) above, for 3D ultrasound images, since the video data contains the time dimension (i.e., continuous frames), the feature extraction module needs to be able to extract spatial and temporal features from the video. Therefore, a 3D convolutional neural network (3D CNN) can be used to process this type of data. 3D convolution can capture both spatial and temporal information at the same time, thereby extracting dynamic features of organs in the video.

[0060] By applying 3D convolutional layers, the feature extraction module can not only identify organs in a single frame, but also learn the changing patterns of organs in consecutive video frames, thereby determining whether the video frames are clear and complete, and helping to generate suitable candidate images.

[0061] It should be noted that the embodiment of the present application does not specifically limit the specific structure of the above-mentioned 3D convolutional neural network, and those skilled in the art can make appropriate adjustments in combination with existing technologies.

[0062] In step 3) above, the patient's clinical text data (such as text description, age, gender, etc.) can be processed using natural language processing (NLP) technology. The patient's clinical text data is converted into numerical features through a pre-trained word embedding model (such as BERT or Word2Vec, etc.). These numerical features are then further processed through a fully connected layer or a long short-term memory network (Long Short-Term Memory, LSTM) to extract key information that helps to evaluate image quality.

[0063] In the above scheme, a convolutional neural network can be used to extract the first feature corresponding to the 2D ultrasound image. The convolutional neural network can capture the shape, boundary and structural information of the organ through layer-by-layer convolution, and further provide support for subsequent organ classification and key anatomical structure segmentation; a 3D convolutional neural network can be used to extract the second feature corresponding to the 3D ultrasound image. By applying the 3D convolutional layer, it can not only identify the organs in a single frame image, but also learn the change pattern of the organs in continuous video frames, thereby improving the accuracy of quality assessment; the word embedding model can be used to extract the third feature corresponding to the patient's clinical text data, thereby extracting key information that is helpful for image quality assessment.

[0064] Further, based on the above embodiment, the step of fusing the first feature, the second feature and the third feature to obtain the fused feature may specifically include the following steps: The first feature, the second feature and the third feature are fused based on the cross-modal attention mechanism.

[0065] Specifically, after extracting the features of each modality, the network needs to effectively fuse the features of these different modalities together so that they can be processed uniformly and make a final evaluation. This process can be achieved through several different strategies. As an implementation method, feature-level fusion can be adopted; or, it can be fused based on a cross-modal attention mechanism.

[0066] In feature-level fusion, the first feature, the second feature, and the third feature are concatenated together to form a unified feature vector as the input for subsequent tasks. The concatenation operation can be implemented by a simple concatenation operation or weighted fusion to give different weights to different modalities according to their importance.

[0067] In cross-modal attention mechanism fusion, the cross-modal attention mechanism can effectively learn how to weight and select features of each modality by modeling the relationship between different modal data, thereby improving the efficiency of feature fusion. For example, by designing the attention mechanism, the network can automatically select modal features that are more important in certain tasks and give them higher weights when evaluating image quality.

[0068] The cross-modal attention mechanism can automatically learn and weight the features of each modality by modeling the relationship between data of different modalities, thereby improving the effect of feature fusion. In multimodal learning, different types of data such as images, videos, and text are usually processed simultaneously, and the information carried by each modality may have different importance in different tasks. The cross-modal attention mechanism assigns weights to each modality, allowing the model to adaptively select more important modal features and ignore less relevant parts, thereby improving the efficiency and accuracy of fusion.

[0069] The role of this mechanism is that it does not simply concatenate the features of each modality, but automatically determines which features should be emphasized and which can be suppressed through the attention mechanism. For example, in image quality assessment, the model may automatically increase the weight of images or physiological data and reduce the influence of other modalities according to the requirements of the task. Unlike traditional feature fusion methods, the cross-modal attention mechanism can dynamically adjust the importance of each modality feature by learning the relationship between modalities, helping the model to more accurately utilize information from different modalities when processing complex tasks.

[0070] In the above scheme, the cross-modal attention mechanism can effectively learn how to weight and select the features of each modality and improve the efficiency of feature fusion by modeling the relationship between data of different modalities.

[0071] Further, based on the above embodiment, the above quality assessment module may include: an organ classification model, an image segmentation model and multiple assessment sub-networks, wherein the above organ classification model is used to classify organs in 3D ultrasound images, the image segmentation model is used to segment 3D ultrasound images, and each assessment sub-network is used to determine an initial score of the 3D ultrasound image in one dimension. Specifically, the step of determining the quality assessment score corresponding to each frame of the 3D ultrasound image according to the fusion features may include the following steps: Step 1) Use the organ classification model to classify the organs of the 3D ultrasound image according to the fusion features to obtain the organ category corresponding to each frame of the 3D ultrasound image.

[0072] Step 2) Use the image segmentation model to segment the key anatomical structures of the 3D ultrasound image according to the fusion features and organ categories, and obtain the segmentation features corresponding to each frame of the 3D ultrasound image.

[0073] In step 3, the evaluation subnetwork is used to determine the initial score of each frame of 3D ultrasound image based on the segmentation features.

[0074] Step 4), based on the initial score, a quality assessment score corresponding to each frame of the 3D ultrasound image is determined.

[0075] Specifically, in the above step 1), the organ classification module can classify organs of the 3D ultrasound image based on the fusion features through a fully connected layer or a convolutional layer. The task of the organ classification module is to determine which organ (for example, liver, kidney, heart, etc.) is displayed in the input image or video frame.

[0076] In the above step 2), the image segmentation model can adopt the U-Net segmentation architecture, using convolutional layers and jump connections to perform anatomical structure segmentation on 3D ultrasound images. Among them, the task of the image segmentation module is to segment the key anatomical structures of organs from images or videos, such as the liver lobules of the liver and the renal tubules of the kidneys, to ensure that high-quality anatomical information is extracted.

[0077] In step 3) above, the subnetwork can adopt an independent branch structure, each branch is responsible for calculating the score of a specific quality dimension. For example, one branch can focus on the clarity of the image, while another branch may evaluate the integrity of the anatomical structure. These scores will eventually be aggregated to form a comprehensive quality assessment score.

[0078] As an implementation method, the above branches can be simply understood as adding a regression head to the fused features to calculate the score of each specific dimension. The design of these regression heads is similar to multi-task learning. Each head performs independent regression processing on the fused features and calculates its own score.

[0079] In the above scheme, the quality assessment module can first classify the organs of the input ultrasound image and accurately identify the specific organs involved in the image, providing a basis for subsequent quality assessment and anatomical structure segmentation; based on the organ classification, the quality assessment module can further segment the key anatomical structures to improve the accuracy and reliability of quality assessment; finally, through multi-dimensional image quality scoring, a more comprehensive image quality reference is provided.

[0080] Furthermore, based on the above embodiments, the ultrasound image quality assessment method provided in the embodiments of the present application may further include the following steps: Step 1) Obtain the doctor selected image.

[0081] In step 2, the doctor-selected image is input into the quality assessment model to obtain the doctor-assessment score corresponding to the doctor-selected image output by the quality assessment model.

[0082] Step 3) Output image retention recommendations based on the doctor's evaluation score and quality assessment score.

[0083] Specifically, in the above step 1), the doctor selecting an image means that the doctor selects a frame of image from multiple frames of ultrasound images when performing ultrasound diagnosis, and the frame of image is the best ultrasound image considered by the doctor.

[0084] It should be noted that the embodiments of the present application do not specifically limit the specific implementation method of obtaining the doctor selection image, and those skilled in the art can make appropriate adjustments according to actual conditions. For example, the doctor selection image sent by an external device can be received; or the doctor selection image pre-stored locally or in the cloud can be read; or the doctor selection image can be collected in real time, etc.

[0085] In the above step 2), the doctor selected image can be input into the quality assessment model to obtain the doctor evaluation score corresponding to the above doctor selected image. The specific implementation method of determining the above doctor evaluation score is similar to the specific implementation method of determining the quality assessment score, which will not be repeated here.

[0086] In the above step 3), a recommendation to keep the image can be output based on the doctor's evaluation score and the quality evaluation score. If the doctor's evaluation score is higher than the quality evaluation score, a recommendation to the doctor to keep the doctor's selected image can be output; if the doctor's evaluation score is lower than the quality evaluation score, a recommendation to the doctor to replace it with the target ultrasound image can be output.

[0087] In the above scheme, based on the image quality assessment score and the comparison result with the doctor's selected image, it can output whether to recommend replacing the current doctor's image. Through this module, the work efficiency of primary care doctors can be further improved, and misdiagnosis or missed diagnosis caused by poor image quality can be reduced.

[0088] Further, based on the above embodiment, the ultrasound image quality assessment method provided in the embodiment of the present application may also include the following steps: training a neural network model using the following steps to obtain a quality assessment model: Step 1) Obtain a sample data set, wherein the sample data set includes 2D sample images, 3D sample images, and sample text data.

[0089] Step 2) Input the sample data set into the neural network model to obtain the loss value corresponding to the neural network model.

[0090] Step 3) Optimize the neural network model according to the loss value to obtain the quality assessment model.

[0091] In the above scheme, the sample data set can be used to train the neural network model to obtain a quality assessment model, and then the quality assessment model is used to assess the quality of the image.

[0092] Further, based on the above embodiment, the loss value includes at least one of the following: an organ classification loss value, an anatomical structure segmentation loss value, an image contrast loss value, a quality score loss value, and an organ position loss value.

[0093] Specifically, organ classification loss: Use the cross-entropy loss function to train the organ classification module to ensure that the network can correctly classify different organs. In order to adapt to the challenges of organ classification, weighted cross-entropy loss is used to deal with the problem of imbalanced categories. As an implementation method, the organ classification loss value can be calculated using the following formula: ; in, is the organ classification loss value, is the number of categories, Yes Category The weight of and The categories are The true labels and predicted probabilities.

[0094] Anatomical structure segmentation loss: A combination of Dice Loss and Binary Cross Entropy (BCE) is used as the loss function to optimize the segmentation module and ensure segmentation accuracy. As an implementation method, the anatomical structure segmentation loss value can be calculated using the following formula: ; in, is the anatomical structure segmentation loss, is the pixel value in the predicted image, is the pixel value in the true label image, is a small constant (such as ), to prevent division by zero errors, is the total number of pixels, and is a hyperparameter that controls the weight of each loss function.

[0095] Image-retaining contrast loss: By comparing the candidate frame with the image selected by the doctor, the difference in image quality is calculated to optimize the network output. Mean Squared Error (MSE) loss and Structural Similarity Index (SSIM Loss) are used to ensure the multi-dimensionality of image quality assessment, making it more suitable for the quality evaluation of medical images. As an implementation method, the image-retaining contrast loss value can be calculated using the following formula: ; in, is the image retention comparison loss value, and They are the predicted image and the doctor’s calibration image, is the number of pixels in the image, and are the means of the predicted image and the real image respectively, and Variance is the covariance, and is a stability constant.

[0096] Quality score loss: The quality score loss is designed to evaluate image quality based on multiple dimensions. A weighted regression loss function can be used to consider multiple scoring dimensions (such as anatomical integrity, clarity, noise, etc.) to optimize the accuracy of the image scoring module. As an implementation method, the quality score loss value can be calculated using the following formula: ; in, is the quality score loss value, is the weight of each dimension, For real rating, Score the predictions, is the number of scoring dimensions.

[0097] Organ position loss: Ensure that the position of the organ in the image is close to the central position, and use the coordinate regression loss function to optimize. As an implementation method, the organ position loss value can be calculated using the following formula: ; in, is the organ position loss value, are the coordinates of the center of the image, are the predicted coordinates of the organ's center of gravity, is the penalty coefficient, It is an indicator function that indicates whether the organ deviates from the center of the image.

[0098] As an implementation method, the overall loss value can be calculated using the following formula: ; in, , , , , is a hyperparameter that controls the weight of each loss function.

[0099] In the above scheme, by calculating the above-mentioned organ classification loss value, anatomical structure segmentation loss value, image retention comparison loss value, quality score loss value and organ position loss value, the neural network model can be optimized to obtain a better quality assessment model.

[0100] The embodiment of the present application provides a MultiModal Ultrasound Image Quality Assessment Network (MUIQAN). In the embodiment of the present application, the automatic recognition technology of the standard section of the ultrasound image uses a deep learning model to automatically determine whether the image meets the standard section requirements by inputting the ultrasound image. This technology can automatically identify qualified sections during the ultrasound examination, reduce doctor's operation errors, and ensure that the acquired images meet the diagnostic requirements. In this way, the deep learning system can improve the efficiency and quality of image acquisition and reduce the uncertainty caused by human operation, which is particularly important for inexperienced doctors in primary medical environments.

[0101] In terms of ultrasound image quality assessment, deep learning technology can automatically evaluate image quality through multi-dimensional analysis (such as resolution, noise level, image clarity, etc.) and give a quality score. This function can help doctors quickly determine the effectiveness of images and avoid using poor quality images for diagnosis. The system can not only detect the integrity of anatomical structures in images, but also identify motion artifacts, noise and other adverse factors, thereby improving the diagnostic value of images.

[0102] Automatic image retention technology can automatically select and save images that meet diagnostic requirements according to set standards during ultrasound examinations. This function relies on the image recognition capabilities of deep learning models, which can intelligently screen out standard section images to prevent doctors from failing to obtain optimal images due to improper operation or lack of experience. Through the automated screening process, doctors can reduce unnecessary operation steps, optimize the diagnosis and treatment process, and improve work efficiency.

[0103] The intelligent feedback system provides doctors with image quality scores and diagnostic suggestions through real-time evaluation of ultrasound image quality. The system can provide feedback on different image quality dimensions (such as resolution, clarity, structural integrity, etc.) to help doctors quickly obtain high-quality images and make timely diagnostic decisions. For less experienced primary care doctors, the system's intelligent prompts and feedback can effectively make up for their shortcomings in standard section recognition and image quality assessment, thereby improving the accuracy and efficiency of diagnosis.

[0104] Please refer to Figure 2 , Figure 2 A structural block diagram of an ultrasound image quality assessment device provided in an embodiment of the present application, the ultrasound image quality assessment device 200 includes: an acquisition module 201, used to acquire a frame of 2D ultrasound image, multiple frames of 3D ultrasound images and patient clinical text data; an input module 202, used to input the 2D ultrasound image, the 3D ultrasound image and the patient clinical text data into a quality assessment model to obtain a quality assessment score corresponding to the target ultrasound image output by the quality assessment model; wherein the quality assessment model includes multiple evaluation subnetworks, each evaluation subnetwork is used to determine an initial score of the 3D ultrasound image in one dimension, and the quality assessment score is determined based on multiple initial scores.

[0105] In the above scheme, when performing quality assessment on ultrasound images, 2D ultrasound images, multi-frame 3D ultrasound images and patient clinical text data can be simultaneously input into the quality assessment network, and the accuracy of quality assessment can be improved by processing multimodal data; in addition, the quality assessment network can realize multi-dimensional image quality scoring, and perform quality assessment on each image by comprehensively evaluating clarity, anatomical integrity, noise level, contrast and other dimensions, thereby providing a more comprehensive image quality reference.

[0106] Further, based on the above embodiments, the quality assessment model includes: a feature extraction module, a feature fusion module and a quality assessment module; the input module 202 is specifically used to: use the feature extraction module to extract the first feature corresponding to the 2D ultrasound image, the second feature corresponding to the 3D ultrasound image and the third feature corresponding to the patient's clinical text data; use the feature fusion module to fuse the first feature, the second feature and the third feature to obtain a fused feature; use the quality assessment module to determine the quality assessment score corresponding to each frame of the 3D ultrasound image according to the fused feature; determine the 3D ultrasound image with the highest quality assessment score as the target ultrasound image, and output the quality assessment score corresponding to the target ultrasound image.

[0107] In the above scheme, the features of 2D ultrasound images, 3D ultrasound images and patient clinical text data can be extracted and fused to obtain fused features; based on the fused features, the image quality can be easily evaluated to obtain a quality evaluation result with a higher accuracy.

[0108] Further, based on the above embodiments, the feature extraction module includes: a convolutional neural network, a 3D convolutional neural network and a word embedding model; the input module 202 is also used to: input the 2D ultrasound image into the convolutional neural network to obtain the first feature output by the convolutional neural network; and, input the 3D ultrasound image into the 3D convolutional neural network to obtain the second feature output by the 3D convolutional neural network; and, input the patient's clinical text data into the word embedding model to obtain the third feature output by the word embedding model.

[0109] In the above scheme, a convolutional neural network can be used to extract the first feature corresponding to the 2D ultrasound image. The convolutional neural network can capture the shape, boundary and structural information of the organ through layer-by-layer convolution, and further provide support for subsequent organ classification and key anatomical structure segmentation; a 3D convolutional neural network can be used to extract the second feature corresponding to the 3D ultrasound image. By applying the 3D convolutional layer, it can not only identify the organs in a single frame image, but also learn the change pattern of the organs in continuous video frames, thereby improving the accuracy of quality assessment; the word embedding model can be used to extract the third feature corresponding to the patient's clinical text data, thereby extracting key information that is helpful for image quality assessment.

[0110] Further, based on the above embodiment, the input module 202 is also used to: fuse the first feature, the second feature and the third feature based on a cross-modal attention mechanism. In the above scheme, the cross-modal attention mechanism can effectively learn how to weight and select the features of each modality by modeling the relationship between different modal data, thereby improving the efficiency of feature fusion.

[0111] Further, based on the above embodiments, the quality assessment module includes: an organ classification model, an image segmentation model and the multiple evaluation sub-networks; the input module 202 is also used to: use the organ classification model to perform organ classification on the 3D ultrasound image according to the fusion features to obtain the organ category corresponding to each frame of the 3D ultrasound image; use the image segmentation model to perform key anatomical structure segmentation on the 3D ultrasound image according to the fusion features and the organ category to obtain the segmentation features corresponding to each frame of the 3D ultrasound image; use the evaluation sub-network to determine the initial score of each frame of the 3D ultrasound image according to the segmentation features; and determine the quality assessment score corresponding to each frame of the 3D ultrasound image based on the initial score.

[0112] In the above scheme, the quality assessment module can first classify the organs of the input ultrasound image and accurately identify the specific organs involved in the image, providing a basis for subsequent quality assessment and anatomical structure segmentation; based on the organ classification, the quality assessment module can further segment the key anatomical structures to improve the accuracy and reliability of quality assessment; finally, through multi-dimensional image quality scoring, a more comprehensive image quality reference is provided.

[0113] Furthermore, based on the above embodiments, the ultrasound image quality assessment device 200 also includes: a training module, which is used to train the neural network model using the following steps to obtain the quality assessment model: obtaining a sample data set, wherein the sample data set includes 2D sample images, 3D sample images, and sample text data; inputting the sample data set into the neural network model to obtain a loss value corresponding to the neural network model; optimizing the neural network model according to the loss value to obtain the quality assessment model.

[0114] In the above scheme, the sample data set can be used to train the neural network model to obtain a quality assessment model, and then the quality assessment model is used to assess the quality of the image.

[0115] Further, based on the above embodiment, the loss value includes at least one of the following: an organ classification loss value, an anatomical structure segmentation loss value, an image contrast loss value, a quality score loss value, and an organ position loss value.

[0116] In the above scheme, by calculating the above-mentioned organ classification loss value, anatomical structure segmentation loss value, image retention comparison loss value, quality score loss value and organ position loss value, the neural network model can be optimized to obtain a better quality assessment model.

[0117] Please refer to Figure 3 , Figure 3 The structural block diagram of an electronic device provided in an embodiment of the present application, the electronic device 300 includes: at least one processor 301, at least one communication interface 302, at least one memory 303 and at least one communication bus 304. Among them, the communication bus 304 is used to realize the direct connection and communication of these components, the communication interface 302 is used to communicate signaling or data with other node devices, and the memory 303 stores machine-readable instructions executable by the processor 301. When the electronic device 300 is running, the processor 301 communicates with the memory 303 through the communication bus 304, and the above-mentioned ultrasound image quality assessment method is executed when the machine-readable instructions are called by the processor 301.

[0118] For example, the processor 301 of the embodiment of the present application reads a computer program from the memory 303 through the communication bus 304 and executes the computer program to implement the following method: obtaining a frame of 2D ultrasound image, multiple frames of 3D ultrasound images, and patient clinical text data; inputting the 2D ultrasound image, the 3D ultrasound image, and the patient clinical text data into a quality assessment model to obtain a quality assessment score corresponding to the target ultrasound image output by the quality assessment model; wherein the quality assessment model includes multiple evaluation subnetworks, each evaluation subnetwork is used to determine an initial score of the 3D ultrasound image in one dimension, and the quality assessment score is determined based on multiple initial scores.

[0119] Among them, the processor 301 includes one or more, which can be an integrated circuit chip with signal processing capabilities. The above-mentioned processor 301 can be a general-purpose processor, including a central processing unit (CPU), a micro control unit (MCU), a network processor (NP) or other conventional processors; it can also be a dedicated processor, including a neural network processor (NPU), a graphics processor (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. In addition, when there are multiple processors 301, some of them can be general-purpose processors, and the other part can be dedicated processors.

[0120] The memory 303 includes one or more, which may be, but not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc.

[0121] Understandably, Figure 3 The structure shown is for illustration only. The electronic device 300 may also include Figure 3 More or fewer components as shown, or with Figure 3 Different configurations shown. Figure 3 Each component shown in can be implemented by hardware, software or a combination thereof. In the embodiment of the present application, the electronic device 300 can be, but is not limited to, a physical device such as a desktop computer, a laptop computer, a smart phone, a smart wearable device, a vehicle-mounted device, etc., and can also be a virtual device such as a virtual machine. In addition, the electronic device 300 is not necessarily a single device, but can also be a combination of multiple devices, such as a server cluster, etc.

[0122] The embodiment of the present application also provides a computer program product, including a computer program stored on a computer-readable storage medium, the computer program including computer program instructions, when the computer program instructions are executed by a computer, the computer can execute the steps of the ultrasound image quality assessment method in the above embodiment, for example including: Step S101: Acquire a frame of 2D ultrasound image, multiple frames of 3D ultrasound image and patient clinical text data. Step S102: Input the 2D ultrasound image, 3D ultrasound image and patient clinical text data into a quality assessment model to obtain a quality assessment score corresponding to the target ultrasound image output by the quality assessment model.

[0123] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer program instructions. When the computer program instructions are executed by a computer, the computer executes the ultrasound image quality assessment method described in the aforementioned method embodiment.

[0124] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0125] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0126] Furthermore, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0127] It should be noted that if the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0128] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0129] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for evaluating the quality of an ultrasound image, characterized in that: include: Acquire a frame of 2D ultrasound image, multiple frames of 3D ultrasound image, and patient clinical text data; Inputting the 2D ultrasound image, the 3D ultrasound image, and the patient clinical text data into a quality assessment model to obtain a quality assessment score corresponding to the target ultrasound image output by the quality assessment model; Wherein, the quality assessment model includes a plurality of assessment subnetworks, each assessment subnetwork is used to determine an initial score of the 3D ultrasound image in one dimension, and the quality assessment score is determined based on the plurality of initial scores; The quality assessment model includes: a feature extraction module, a feature fusion module and a quality assessment module; The step of inputting the 2D ultrasound image, the 3D ultrasound image, and the patient clinical text data into a quality assessment model to obtain a quality assessment score corresponding to a target ultrasound image output by the quality assessment model includes: Extracting, by using the feature extraction module, a first feature corresponding to the 2D ultrasound image, a second feature corresponding to the 3D ultrasound image, and a third feature corresponding to the patient's clinical text data; Using the feature fusion module to fuse the first feature, the second feature and the third feature to obtain a fused feature; Determine, by using the quality assessment module, a quality assessment score corresponding to each frame of the 3D ultrasound image according to the fusion feature; The 3D ultrasound image with the highest quality assessment score is determined as a target ultrasound image, and the quality assessment score corresponding to the target ultrasound image is output.

2. The method for evaluating the quality of an ultrasonic image according to claim 1, characterized in that: The feature extraction module includes: a convolutional neural network, a 3D convolutional neural network and a word embedding model; The extracting, by using the feature extraction module, the first feature corresponding to the 2D ultrasound image, the second feature corresponding to the 3D ultrasound image, and the third feature corresponding to the patient's clinical text data includes: Inputting the 2D ultrasound image into the convolutional neural network to obtain the first feature output by the convolutional neural network; and, Inputting the 3D ultrasound image into the 3D convolutional neural network to obtain the second feature output by the 3D convolutional neural network; and, The patient's clinical text data is input into the word embedding model to obtain the third feature output by the word embedding model.

3. The method for evaluating the quality of ultrasonic images according to claim 1, characterized in that: The fusing the first feature, the second feature and the third feature to obtain a fused feature includes: The first feature, the second feature and the third feature are fused based on a cross-modal attention mechanism.

4. The method for evaluating the quality of ultrasonic images according to claim 1, characterized in that: The quality assessment module includes: an organ classification model, an image segmentation model and the multiple assessment sub-networks; Determining the quality assessment score corresponding to each frame of the 3D ultrasound image according to the fusion feature includes: Using the organ classification model to classify organs of the 3D ultrasound image according to the fusion features, to obtain the organ category corresponding to each frame of the 3D ultrasound image; Using the image segmentation model to perform key anatomical structure segmentation on the 3D ultrasound image according to the fusion features and the organ category, and obtaining segmentation features corresponding to each frame of the 3D ultrasound image; Determine an initial score of each frame of the 3D ultrasound image according to the segmentation features using the evaluation subnetwork; A quality assessment score corresponding to each frame of the 3D ultrasound image is determined based on the initial score.

5. The method for evaluating the quality of an ultrasonic image according to any one of claims 1 to 4, characterized in that: The method further comprises: The neural network model is trained using the following steps to obtain the quality assessment model: Acquire a sample data set, wherein the sample data set includes 2D sample images, 3D sample images, and sample text data; Inputting the sample data set into the neural network model to obtain a loss value corresponding to the neural network model; The neural network model is optimized according to the loss value to obtain the quality assessment model.

6. The method for evaluating the quality of ultrasonic images according to claim 5, characterized in that: The loss value includes at least one of the following: an organ classification loss value, an anatomical structure segmentation loss value, an image retention comparison loss value, a quality score loss value, and an organ position loss value.

7. A computer program product, characterized in that The method comprises computer program instructions, and when the computer program instructions are read and executed by a processor, the method for evaluating the quality of an ultrasound image according to any one of claims 1 to 6 is executed.

8. An electronic device, characterized in that: include: processor, memory, and bus; The processor and the memory communicate with each other via the bus; The memory stores computer program instructions that can be executed by the processor, and the processor calls the computer program instructions to execute the ultrasound image quality assessment method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a computer, the computer is enabled to perform the ultrasound image quality assessment method according to any one of claims 1 to 6.

Images