Ultrasonic image quality evaluation method and device based on image-text matching

Abstract

The invention relates to the technical field of ultrasonic image quality evaluation, in particular to an ultrasonic image quality evaluation method and device based on image-text matching, and the method comprises the steps: extracting the image features of an ultrasonic image, and extracting the text features of a text representing an image quality evaluation standard; performing semantic fusion on the image text features, and calculating a matching score between the image-text pairs; and obtaining a final image quality score of the ultrasonic image according to the matching score, calculating a mean square error loss between the final image quality score and a preset standard score by using the labeled image sample, generating a final ultrasonic image quality evaluation model, and outputting a quality evaluation result of any ultrasonic image by using the ultrasonic image quality evaluation model. Therefore, the problem that the generalization ability of a model in an actual scene is insufficient due to the fact that an evaluation method in the related technology ignores a key structure that an image should have as embodied by a clinical standard and lacks the modeling ability for medical knowledge is solved.

Description

Technical Field

[0001] The present application relates to the technical field of ultrasound image quality assessment, and in particular to an ultrasound image quality assessment method based on image-text matching. Background Art

[0002] In recent years, medical ultrasound, as a real-time, non-invasive, and low-cost imaging method, has been widely used in clinical diagnosis, prenatal screening, disease monitoring, and other fields. As the scope of application of ultrasound images continues to expand, the quality of their images directly affects doctors' identification and judgment of lesion structures. Ultrasound images have the characteristics of strong noise, blurred structure, and acquisition methods that rely on operator experience, making image quality assessment particularly complex. Therefore, the automated assessment of ultrasound image quality has become a research hotspot. In actual applications, due to differences in operator skill level, patient posture, and equipment settings, the acquired images often have problems such as unclear resolution, incomplete structure, and low signal-to-noise ratio. These problems not only increase the reading burden on doctors, but also affect the diagnostic accuracy of artificial intelligence models.

[0003] In related technologies, researchers have attempted to introduce image quality assessment models to help determine whether an image is suitable for subsequent analysis. These methods often rely on the image's texture, edge, or depth features to predict image quality scores.

[0004] However, the evaluation methods in related technologies ignore the "key structures that images should have" reflected in clinical standards and lack the ability to model medical knowledge, which in turn leads to problems such as insufficient generalization ability of the model in actual scenarios, and urgently needs to be improved. Summary of the Invention

[0005] The present application provides an ultrasound image quality assessment method and device based on image-text matching to address the problems that the assessment methods in related technologies ignore the "key structures that images should have" reflected in clinical standards, lack the ability to model medical knowledge, and thus lead to insufficient generalization ability of the model in actual scenarios.

[0006] The first aspect of the present application provides an ultrasound image quality assessment method based on image-text matching, comprising the following steps: extracting image features of at least one ultrasound image, and extracting text features of at least one text representing an image quality assessment standard; semantically fusing the image features and the text features to calculate a matching score between the image and text pairs; obtaining a final image quality score of the at least one ultrasound image based on the matching score, calculating the mean square error loss between the final image quality score and a preset standard score using labeled image samples, and generating a final ultrasound image quality assessment model, so as to output a quality assessment result of any ultrasound image using the ultrasound image quality assessment model.

[0007] Through the above technical solution, the embodiment of the present application can extract text features representing image quality assessment standards and semantically fuse them with image features, and further use labeled pattern samples to calculate the mean square error loss to guide the self-supervised training of the model, thereby constructing an image-text matching network with image-text multimodal representation as the core, so that the model can understand the "medical semantic standards that the image should meet" and indirectly judge whether the image quality meets the standards.

[0008] Optionally, in one embodiment of the present application, the image features and the text features are semantically fused to calculate the matching score between the image and text pairs, including: using the image features as a query and the text features as a key-value pair to obtain the text fusion features from the image perspective; using the text features as a query and the image features as a key-value pair to obtain the image fusion features from the text perspective.

[0009] Through the above technical solution, the embodiment of the present application can use cross-attention to semantically fuse image features and text features, and align the image and text features after interaction, so as to better achieve image-text matching, and output the matching score, which is used as the performance of the image under multiple standards to effectively reflect the image's satisfaction with the current standard.

[0010] Optionally, in one embodiment of the present application, the calculation formula of the matching score is:

[0011]

[0012] Among them, s represents the degree of match between the image and the text description, z x represents the extracted image features, z t Represents the extracted text features.

[0013] In the embodiment of the present application, the matching score can be calculated by establishing the above-mentioned model to describe the degree of matching between the image and the text, so as to further derive the final image quality score for constructing the loss function for model training and optimization.

[0014] Optionally, in one embodiment of the present application, the calculation formula of the mean square error loss is:

[0015]

[0016] in, represents the mean square error loss, N represents the number of images, Represents the classification label, y i Represents the calculated quality score.

[0017] The application embodiment can construct a mean square error loss model based on the real image quality score and the model prediction score, and then guide the training and optimization of the quality assessment model with the goal of minimizing the error, thereby improving the accuracy of the model assessment.

[0018] Optionally, in one embodiment of the present application, it also includes: inputting the unlabeled image into the ultrasound image quality assessment model to output a predicted image-text matching score, and estimating the confidence of the predicted image-text matching score; using the confidence as a pseudo label to participate in training to optimize the ultrasound image quality assessment model.

[0019] The embodiments of the present application can build an image-text matching network with image-text multimodal representation as the core, so that the model can understand the "medical semantic standards that the image should meet", thereby indirectly judging whether the image quality meets the standards, and continuously optimize the model performance by using unlabeled samples through pseudo-label generation and confidence filtering mechanisms.

[0020] The second aspect of the present application provides an ultrasound image quality assessment method based on image-text matching, including: an extraction module for extracting image features of at least one ultrasound image and extracting text features of at least one text representing an image quality assessment standard; a fusion module for semantically fusing the image features and the text features to calculate a matching score between the image and text pairs; an assessment module for obtaining a final image quality score of the at least one ultrasound image based on the matching score, calculating the mean square error loss between the final image quality score and a preset standard score using labeled image samples, and generating a final ultrasound image quality assessment model, so as to output a quality assessment result of any ultrasound image using the ultrasound image quality assessment model.

[0021] Through the above technical solution, the embodiment of the present application can extract text features representing image quality assessment standards and semantically fuse them with image features, and further use labeled pattern samples to calculate the mean square error loss to guide the self-supervised training of the model, thereby constructing an image-text matching network with image-text multimodal representation as the core, so that the model can understand the "medical semantic standards that the image should meet" and indirectly judge whether the image quality meets the standards.

[0022] Through the above technical solution, the embodiment of the present application can use cross-attention to semantically fuse image features and text features, and align the image and text features after interaction, so as to better achieve image-text matching, and output the matching score, which is used as the performance of the image under multiple standards to effectively reflect the image's satisfaction with the current standard.

[0023] Optionally, in one embodiment of the present application, the fusion module includes: a first acquisition unit, used to use the image features as a query and the text features as a key-value pair to obtain text fusion features from an image perspective; a second acquisition unit, used to use the text features as a query and the image features as a key-value pair to obtain image fusion features from a text perspective.

[0024] Optionally, in one embodiment of the present application, the calculation formula of the matching score is:

[0025]

[0026] Among them, s represents the degree of match between the image and the text description, z x represents the extracted image features, z t Represents the extracted text features.

[0027] In the embodiment of the present application, the matching score can be calculated by establishing the above-mentioned model to describe the degree of matching between the image and the text, so as to further derive the final image quality score for constructing the loss function for model training and optimization.

[0028] Optionally, in one embodiment of the present application, the calculation formula of the mean square error loss is:

[0029]

[0030] in, represents the mean square error loss, N represents the number of images, Represents the classification label, y i Represents the calculated quality score.

[0031] The application embodiment can construct a mean square error loss model based on the real image quality score and the model prediction score, and then guide the training and optimization of the quality assessment model with the goal of minimizing the error, thereby improving the accuracy of the model assessment.

[0032] Optionally, in one embodiment of the present application, it also includes: a prediction module, used to input the unlabeled image into the ultrasound image quality assessment model to output a predicted image-text matching score and estimate the confidence of the predicted image-text matching score; an optimization module, used to use the confidence as a pseudo-label to participate in training to optimize the ultrasound image quality assessment model.

[0033] The embodiments of the present application can build an image-text matching network with image-text multimodal representation as the core, so that the model can understand the "medical semantic standards that the image should meet", thereby indirectly judging whether the image quality meets the standards, and continuously optimize the model performance by using unlabeled samples through pseudo-label generation and confidence filtering mechanisms.

[0034] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement the ultrasound image quality assessment method based on image-text matching as described in the above embodiment.

[0035] The fourth aspect of the present application provides a computer-readable storage medium, which stores a computer program. When the program is executed by a processor, it implements the above-mentioned ultrasound image quality assessment method based on image-text matching.

[0036] A fifth aspect of the present application provides a computer program product, which stores a computer program that, when executed by a processor, implements the above-mentioned ultrasound image quality assessment method based on image-text matching.

[0037] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0039] Figure 1 This is a flowchart of an ultrasound image quality assessment method based on image-text matching provided according to an embodiment of the present application;

[0040] Figure 2 Schematic diagram of an ultrasound image quality assessment device based on image-text matching according to an embodiment of the present application;

[0041] Figure 3 A schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0042] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0043] The following describes an ultrasound image quality assessment method and apparatus based on image-text matching according to an embodiment of the present application, with reference to the accompanying drawings. In response to the problem that the assessment methods in the related art mentioned in the background art center ignore the "key structures that images should have" reflected in clinical standards, lack the ability to model medical knowledge, and thus lead to insufficient generalization ability of the model in real-world scenarios, the present application provides an ultrasound image quality assessment method based on image-text matching. In this method, text features representing image quality assessment standards are extracted and semantically fused with image features. Furthermore, a mean squared error loss is calculated using labeled image samples to guide self-supervised training of the model, thereby constructing an image-text matching network with image-text multimodal representation as the core. This enables the model to understand the "medical semantic standards that images should meet" and thus indirectly determine whether the image quality meets the standards. Furthermore, through pseudo-label generation and confidence filtering mechanisms, unlabeled samples are used to continuously optimize model performance. Thus, the present application solves the problem that the assessment methods in the related art ignore the "key structures that images should have" reflected in clinical standards, lack the ability to model medical knowledge, and thus lead to insufficient generalization ability of the model in real-world scenarios.

[0044] Specifically, Figure 1 A flowchart of an ultrasound image quality assessment method based on image-text matching provided in an embodiment of the present application.

[0045] like Figure 1 As shown, the ultrasound image quality assessment method based on image-text matching includes the following steps:

[0046] In step S101 , image features of at least one ultrasound image are extracted, and text features of at least one text representing an image quality assessment criterion are extracted.

[0047] It is understandable that the text representing the image quality assessment standard includes but is not limited to "complete image structure", "blurred image edges" and the like.

[0048] In the actual implementation process, the embodiment of the present application can use the pre-trained image encoder to extract image features from the input ultrasound image x. Input the text t representing the image quality assessment standard (such as "complete image structure", "blurred image edges", etc.) into the language encoder to extract text features

[0049] In a specific embodiment, feature extraction can be achieved through the following process:

[0050] Data preparation:

[0051] An ultrasound image dataset was collected. The data source was fetal four-chamber heart views, which included quality score labels based on three ultrasound image guidelines: display of the left and right atrioventricular cavities and two sets of atrioventricular valves, and at least one complete rib; the ventricular septum was displayed as a complete linear structure throughout, without echo interruptions or blurred areas, forming a cross structure with the mitral and tricuspid valves; and the image was clear, with clear and complete boundaries that did not affect the identification of cardiac structures. Quality scores were provided by ultrasound imaging experts.

[0052] Image and text feature extraction:

[0053] The image encoder uses ResNet-18 as the visual feature extraction network to encode the input image features and output the image feature vector

[0054] The text encoder uses a pre-trained BERT model to embed medical evaluation text into semantic features.

[0055] To ensure the consistency of feature dimensions, the text features are reduced to the same dimension as the image features through a linear layer (512).

[0056] Through the above technical solution, the embodiment of the present application can extract text features by inputting text representing image quality assessment standards into a language encoder, and then fuse the extracted text features with image features to construct an image quality assessment model, so that the model takes into account the "key structures that the image should have" reflected in the clinical standards during the construction process, and has the ability to model medical knowledge.

[0057] In step S102 , the image features and the text features are semantically fused to calculate the matching score between the image and text pairs.

[0058] Among them, in the embodiments of the present application, the semantic fusion of image features and text features can be understood as associating, aligning and complementing information of two different modalities (visual and language) at the semantic level to achieve richer joint representation or more accurate downstream tasks (such as cross-modal retrieval, visual question answering, image generation, etc.).

[0059] During the actual implementation process, the embodiment of the present application can use a cross-attention module to semantically fuse image features and text features, input the image and text as query-key-value pairs into two rounds of cross-attention modules respectively, and input the above-mentioned interactive image and text features into a multi-layer cadre-to-text mapping network of the same structure, map them to a unified alignment space, and then calculate the matching score between the image and text.

[0060] Through the above technical solution, the embodiment of the present application can achieve two-way semantic fusion by performing two rounds of cross-attention processing on the image and text, thereby enhancing the matching relationship between the image and text.

[0061] In step S103, a final image quality score of at least one ultrasound image is obtained based on the matching score, and the mean square error loss between the final image quality score and the preset standard score is calculated using labeled image samples to generate a final ultrasound image quality assessment model, so as to output the quality assessment result of any ultrasound image using the ultrasound image quality assessment model.

[0062] In the embodiment of the present application, the final image quality score can be obtained by weighted fusion of the matching scores, and the formula is as follows:

[0063]

[0064] Where, α i Indicates the importance weight of each text standard, s i represents the matching score between the image and the i-th text, and n represents the number of text criteria.

[0065] The embodiment of the present application can use labeled image samples to calculate the mean square error loss between the final image quality score and the preset standard score to construct a self-supervised training strategy for the model, and optimize the training of the evaluation model based on this strategy, thereby improving the evaluation accuracy of the model.

[0066] Based on steps S101-S103, the ultrasound image quality assessment method based on image-text matching provided in the embodiment of the present application can achieve intelligent assessment of ultrasound image quality without the participation of a large number of scoring labels by introducing an image-text semantic alignment mechanism, a cross-attention interaction structure, and a self-supervised training strategy, thereby improving the interpretability and generalization ability of the model. Compared with traditional ultrasound image quality assessment methods, the quality assessment method of the embodiment of the present application has the following advantages:

[0067] Semantic Standard: A dual cross-attention mechanism is used to achieve deep semantic fusion of images and medical text, enabling the model to have a human-like understanding of image quality.

[0068] Strong interpretability: The image-text similarity score has clear semantic meaning, making it easier for clinicians to understand and use.

[0069] Strong adaptability: The method can be extended to other medical image quality assessment tasks or multimodal analysis scenarios.

[0070] Optionally, in one embodiment of the present application, image features and text features are semantically fused to calculate the matching score between image and text pairs, including: using image features as queries and text features as key-value pairs to obtain text fusion features from the image perspective; using text features as queries and image features as key-value pairs to obtain image fusion features from the text perspective.

[0071] Among them, the key-value pair can be understood as a structured data storage method, consisting of two core parts: key and value.

[0072] In one embodiment of the present application, image features and text features are semantically fused to calculate the matching score between the image and text pairs, which can be specifically achieved in the following manner:

[0073] First, the cross attention module is used to transform the image feature f x With text feature f t The query-key-value pairs are input into two rounds of cross-attention modules to perform bidirectional cross-attention interaction between images and text:

[0074] The first attention interaction: using image feature f x is the query, the text feature f t As key (Key) and value (Value), we can get the text fusion feature f from the image perspective tx ;

[0075] Second attention interaction: using text features as queries and image features as key-value pairs, we obtain the image fusion features f from the text perspective. xt ;

[0076] The above interactions are implemented using a standard cross-attention mechanism, where each interaction module includes residual connections and layer normalization (Layer Normalization, LayerNorm).

[0077] Secondly, perform MLP mapping and similarity calculation:

[0078] Use two weight-sharing multi-layer perceptrons (MLPs) to analyze the image features f after interaction. xt With text feature f tx Mapped to a unified alignment space:

[0079] z x =MLP(f xt ),z t =MLP(f tx )(2)

[0080] The MLP structure consists of two linear layers with a ReLU activation function in the middle.

[0081] Then calculate the cosine similarity between the image and text features as the matching score, that is, use cosine similarity to calculate the image and text matching score:

[0082]

[0083] in, Indicates the degree of matching between the image and the text description. Multiple matching scores {s1, s2, ..., s n}, as the performance of the image under multiple standards.

[0084] The embodiment of the present application can use cross-attention to semantically fuse image features and text features, and align the image and text features after interaction, so as to better achieve image-text matching, and output a matching score as the performance of the image under multiple standards to effectively reflect the image's satisfaction with the current standard.

[0085] Optionally, in one embodiment of the present application, the calculation formula for the matching score is:

[0086]

[0087] Among them, s represents the degree of match between the image and the text description, among them, s represents the degree of match between the image and the text description, z x represents the extracted image features, z t Represents the extracted text features.

[0088] In the actual implementation process, the embodiment of the present application can be based on z x 、z t Construct a matching score calculation model and calculate the cosine similarity between image and text features as the matching score.

[0089] The embodiment of the present application will be able to calculate the matching score by establishing the above-mentioned model, thereby describing the degree of matching between the image and the text, so as to further derive the final image quality score for constructing the loss function for model training and optimization.

[0090] Optionally, in one embodiment of the present application, the calculation formula of the mean square error loss is:

[0091]

[0092] in, represents the mean square error loss, N represents the number of images, Represents the classification label, y i Represents the calculated quality score.

[0093] In the embodiment of the present application, each image corresponds to multiple text standards, and the average of all matching scores is calculated as the final quality prediction score of the image.

[0094] In the supervised training phase, the error between the predicted score and the expert score is minimized, and labeled image samples can be used to predict the image quality score. And calculate the mean square error loss between the expert score y:

[0095]

[0096] The application embodiment can construct a mean square error loss model based on the real image quality score and the model prediction score, and then guide the training and optimization of the quality assessment model with the goal of minimizing the error, thereby improving the accuracy of the model assessment.

[0097] Optionally, in one embodiment of the present application, it also includes: inputting the unlabeled image into the ultrasound image quality assessment model to output the predicted image-text matching score, and estimating the confidence of the predicted image-text matching score; using the confidence as a pseudo-label to participate in training to optimize the ultrasound image quality assessment model.

[0098] During the actual implementation process, the embodiment of the present application can introduce a pseudo-label mechanism. For unlabeled images, the current model is used to predict the image-text matching score and estimate the confidence; only high-confidence prediction results are retained as pseudo-labels for training; and the pseudo-labels and model parameters are continuously updated to achieve iterative optimization.

[0099] Specifically, in the pseudo-label generation stage, MC Dropout can be used to perform multiple predictions on unlabeled images, and the variance of the prediction results can be calculated as a confidence estimate. Only image samples with prediction confidence higher than the threshold are retained, and their prediction scores are added to the training set as pseudo-labels. The pseudo-label samples are used in subsequent training to achieve model self-evolution optimization and generate the final ultrasound image quality assessment model.

[0100] It has been verified that the quality assessment method of the embodiment of the present application has higher correlation and lower error on the fetal four-chamber heart image dataset than other quality assessment methods (BRISQUE, CLIP-IQA), among which the experimental evaluation indicators used include: PLCC (Pearson correlation coefficient) and MSE (mean square error).

[0101] The embodiment of the present application can construct an image-text matching network with image-text multimodal representation as the core, so that the model can understand the "medical semantic standards that the image should meet", thereby indirectly judging whether the image quality meets the standards; on this basis, through pseudo-label generation and confidence filtering mechanism, the model performance is continuously optimized using unlabeled samples.

[0102] According to the ultrasound image quality assessment method based on image-text matching proposed in the embodiment of the present application, it is possible to extract text features representing image quality assessment standards and perform semantic fusion with image features, and further use labeled pattern samples to calculate the mean square error loss to guide the self-supervised training of the model, thereby constructing an image-text matching network with image-text multimodal representation as the core, so that the model can understand the "medical semantic standards that the image should meet" and thus indirectly judge whether the image quality meets the standards; and through pseudo-label generation and confidence filtering mechanisms, the unlabeled samples are used to continuously optimize the model performance. In this way, the evaluation methods in the related art are solved. Due to the lack of modeling ability of medical knowledge due to ignoring the "key structure that the image should have" reflected in the clinical standards, the model has insufficient generalization ability in actual scenarios.

[0103] Next, refer to the attached Figure 2 The present invention describes an ultrasound image quality assessment device based on image-text matching proposed in an embodiment of the present application.

[0104] Figure 2 4 is a block diagram of an ultrasound image quality assessment device based on image-text matching according to an embodiment of the present application.

[0105] like Figure 2 As shown, the ultrasound image quality assessment device 10 based on image-text matching includes: an extraction module 100 , a fusion module 200 , and an assessment module 300 .

[0106] The extraction module 100 is used to extract image features of at least one ultrasound image and extract text features of at least one text representing an image quality assessment standard.

[0107] The fusion module 200 is used to semantically fuse image features and text features to calculate the matching score between the image and text pairs.

[0108] The evaluation module 300 is used to obtain the final image quality score of at least one ultrasound image based on the matching score, calculate the mean square error loss between the final image quality score and the preset standard score using labeled image samples, and generate a final ultrasound image quality evaluation model to output the quality evaluation result of any ultrasound image using the ultrasound image quality evaluation model.

[0109] Optionally, in one embodiment of the present application, the fusion module 200 includes: a first acquisition unit and a second acquisition unit; wherein the first acquisition unit is used to use image features as queries and text features as key-value pairs to obtain text fusion features from an image perspective; the second acquisition unit is used to use text features as queries and image features as key-value pairs to obtain image fusion features from a text perspective.

[0110] Optionally, in one embodiment of the present application, the calculation formula for the matching score is:

[0111]

[0112] Among them, s represents the degree of match between the image and the text description, z x represents the extracted image features, z t Represents the extracted text features.

[0113] Optionally, in one embodiment of the present application, the calculation formula of the mean square error loss is:

[0114]

[0115] in, represents the mean square error loss, N represents the number of images, Represents the classification label, y i Represents the calculated quality score.

[0116] Optionally, in one embodiment of the present application, the ultrasound image quality assessment device 10 based on image-text matching further includes: a prediction module and an optimization module; wherein the prediction module is used to input the unlabeled image into the ultrasound image quality assessment model to output a predicted image-text matching score and estimate the confidence of the predicted image-text matching score; the optimization module is used to use the confidence as a pseudo-label to participate in training to optimize the ultrasound image quality assessment model.

[0117] It should be noted that the aforementioned explanation of the embodiment of the ultrasound image quality assessment method based on image-text matching is also applicable to the ultrasound image quality assessment device based on image-text matching in this embodiment, and will not be repeated here.

[0118] According to the ultrasound image quality assessment device based on image-text matching proposed in the embodiment of the present application, it is possible to extract text features representing image quality assessment standards and perform semantic fusion with image features, and further use labeled pattern samples to calculate the mean square error loss to guide the self-supervised training of the model, thereby constructing an image-text matching network with image-text multimodal representation as the core, so that the model can understand the "medical semantic standards that the image should meet" and thus indirectly judge whether the image quality meets the standards; and through pseudo-label generation and confidence filtering mechanisms, the unlabeled samples are used to continuously optimize the model performance. This solves the problem that the evaluation methods in the related art ignore the "key structures that the image should have" reflected in the clinical standards, lack the ability to model medical knowledge, and thus lead to insufficient generalization of the model in actual scenarios.

[0119] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0120] Memory 301 , processor 302 , and computer programs stored in the memory 301 and executable on the processor 302 .

[0121] When the processor 302 executes the program, the ultrasound image quality assessment method based on image-text matching provided in the above embodiment is implemented.

[0122] Furthermore, the electronic device further includes:

[0123] The communication interface 303 is used for communication between the memory 301 and the processor 302 .

[0124] The memory 301 is used to store computer programs that can be run on the processor 302 .

[0125] The memory 301 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0126] If the memory 301, processor 302, and communication interface 303 are implemented independently, the communication interface 303, memory 301, and processor 302 can be interconnected via a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0127] Optionally, in a specific implementation, if the memory 301 , the processor 302 and the communication interface 303 are integrated on a chip, the memory 301 , the processor 302 and the communication interface 303 can communicate with each other through an internal interface.

[0128] The processor 302 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0129] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned ultrasound image quality assessment method based on image-text matching.

[0130] An embodiment of the present application further provides a computer program product, which stores a computer program, and when the program is executed by a processor, implements the above-mentioned ultrasound image quality assessment method based on image-text matching.

[0131] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0132] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this application, "N" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0133] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing a custom logical function or process step, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed in a different order than shown or discussed, including performing functions in a substantially simultaneous manner or in a reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application pertain.

[0134] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or N wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program can be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing it in other suitable ways as necessary, and then storing it in a computer memory.

[0135] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiment, the N steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented using hardware, as in another embodiment, it can be implemented using any one or a combination of the following technologies known in the art: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0136] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0137] In addition, the functional units in the various embodiments of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into a module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0138] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A method for ultrasound image quality assessment based on image-text matching, characterized in that: The following steps are involved: extracting image features of at least one ultrasound image and extracting text features of at least one text representing an image quality assessment criterion; semantically fusing the image features and the text features to calculate a matching score between the image and text pairs; A final image quality score of the at least one ultrasound image is obtained based on the matching score, and a mean square error loss between the final image quality score and a preset standard score is calculated using labeled image samples to generate a final ultrasound image quality assessment model, so as to output a quality assessment result of any ultrasound image using the ultrasound image quality assessment model.

2. The method according to claim 1, characterized in that The semantically fusing the image features and the text features to calculate a matching score between the image and text pairs includes: Using the image feature as a query and the text feature as a key-value pair, obtaining text fusion features from the image perspective; The text features are used as a query, and the image features are used as a key-value pair to obtain image fusion features from a text perspective.

3. The method according to claim 1 or 2, characterized in that The calculation formula of the matching score is: Among them, s represents the degree of match between the image and the text description, z x represents the extracted image features, z t Represents the extracted text features.

4. The method according to claim 1, wherein The calculation formula of the mean square error loss is: in, represents the mean square error loss, N represents the number of images, Represents the classification label, y i Represents the calculated quality score.

5. The method according to claim 1, wherein Also includes: Inputting the unlabeled image into the ultrasound image quality assessment model to output a predicted image-text matching score, and estimating the confidence of the predicted image-text matching score; The confidence level is used as a pseudo label to participate in training to optimize the ultrasound image quality assessment model.

6. An ultrasound image quality assessment device based on image-text matching, characterized in that: include: an extraction module, configured to extract image features of at least one ultrasound image and extract text features of at least one text representing an image quality assessment criterion; A fusion module, configured to semantically fuse the image features and the text features to calculate a matching score between the image and text pairs; An evaluation module is configured to obtain a final image quality score of the at least one ultrasound image based on the matching score, calculate a mean square error loss between the final image quality score and a preset standard score using labeled image samples, and generate a final ultrasound image quality evaluation model, so as to output a quality evaluation result of any ultrasound image using the ultrasound image quality evaluation model.

7. The device according to claim 6, characterized in that The fusion module includes: A first acquisition unit is configured to use the image feature as a query and the text feature as a key-value pair to acquire text fusion features from an image perspective; The second acquisition unit is configured to use the text feature as a query and the image feature as a key-value pair to obtain the image fusion feature from a text perspective.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the ultrasound image quality assessment method based on image-text matching according to any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the ultrasound image quality assessment method based on image-text matching as described in any one of claims 1 to 5.

10. A computer program product comprising a computer program, characterized in that The computer program is executed to implement the ultrasound image quality assessment method based on image-text matching according to any one of claims 1 to 5.

Images