Image analysis method and system, electronic equipment and medium
Through deep learning models, the ultrasonic images are classified and quality tested, which solves the problem of uneven ultrasonic image quality in the prior art, and achieves efficient and accurate image analysis results.
Patent Information
- Application Number
- CN202411905414.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-23
- Publication Date
- 2025-03-28
AI Technical Summary
The lack of integrated ultrasound image classification and quality evaluation methods in the prior art leads to uneven ultrasound image quality, affecting the accuracy and consistency of diagnosis.
The deep learning model is used to classify and quality detection of ultrasound images, classify images through the first deep learning model, and use the second deep learning model to perform quality detection, and comprehensive analysis is carried out in combination with the classification results and quality detection results to obtain image analysis results.
It realizes efficient view classification and quality detection, and improves the accuracy and consistency of ultrasonic image classification and quality detection.
Smart Images
Figure CN119722651A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of medical image processing technology, and relates to an image analysis method, and in particular to an image analysis method, system, electronic equipment and medium. Background Art
[0002] Extended Focused Assessment with Sonography for Trauma (eFAST) is an ultrasound examination technique commonly used in emergency departments and surgical departments to quickly assess bleeding or fluid accumulation in the body cavities of trauma patients. The eFAST ultrasound scan includes images of multiple areas including the abdominal cavity, thoracic cavity, pericardium, and pelvic cavity, and uses non-invasive, real-time images to observe whether there is pathological fluid in the body. The eFAST specification emphasizes the accuracy and efficiency of ultrasound examinations, requiring ultrasound images to have sufficient clarity and integrity of key anatomical parts so that medical staff can quickly and accurately determine the patient's injuries. However, due to different operators' experience and technical levels, the quality of ultrasound images varies, affecting the accuracy and consistency of subsequent diagnosis. Summary of the Invention
[0003] In view of the above-mentioned shortcomings of the prior art, the purpose of this application is to provide an image analysis method, system, electronic device and medium to solve the problem of the lack of an integrated ultrasound image classification and quality assessment method in the prior art.
[0004] In a first aspect, the present application provides an image analysis method, which includes: obtaining an ultrasound image to be analyzed; using a first deep learning model to classify the ultrasound image to be analyzed to obtain a classification result, and using a second deep learning model to perform quality detection on the ultrasound image to be analyzed to obtain a quality detection result; performing a comprehensive analysis of the ultrasound image to be analyzed based on the classification result and the quality detection result to obtain an image analysis result; wherein, the first deep learning model and most of the second deep learning models are built using the same backbone network.
[0005] In this application, a first deep learning model is used to classify the ultrasound images to be analyzed, and a second deep learning model is used to perform quality inspection on the ultrasound images to be analyzed. The classification results and quality inspection results are integrated for comprehensive analysis to obtain image analysis results. This image analysis method integrates image classification and quality inspection into the same model, achieving efficient view classification and quality inspection, and improving the accuracy and consistency of ultrasound image classification and quality inspection.
[0006] In an implementation of the first aspect, training the first deep learning model and the second deep learning model includes: acquiring ultrasound image data; annotating and preprocessing the ultrasound image data to obtain training data; and using the training data to train an initial first deep learning model and an initial second deep learning model to obtain the first deep learning model and the second deep learning model, wherein the first deep learning model and the second deep learning model are built using the same backbone network.
[0007] In an implementation of the first aspect, labeling and preprocessing the ultrasound image data to obtain training data includes: labeling the ultrasound image data to obtain the category, organizational structure and quality characteristics of the ultrasound image data; performing quality classification on the ultrasound image data to obtain the image quality level of the ultrasound image data; and normalizing, denoising and / or image enhancement processing on the ultrasound image to obtain the training data.
[0008] In an implementation of the first aspect, using the training data to train the initial first deep learning model includes: using the initial first deep learning model to extract features of the training data labeled with image categories to obtain image features; mapping the image features to multiple categories to obtain multi-category output results; using an activation function to map the multi-category output results to a preset interval to obtain the classification probability of the image.
[0009] In an implementation of the first aspect, training the initial second deep learning model using the training data includes: using the initial second deep learning model to fuse the image features to obtain a fused feature map; upsampling the fused map to obtain a feature map with the same resolution; splicing the feature maps with the same resolution to obtain a fused feature matrix; and mapping the fused feature matrix using a multilayer perceptron to obtain an image quality detection result.
[0010] In an implementation of the first aspect, when training the first deep learning model and the second deep learning model, the first deep learning model and the second deep learning model are trained using cross entropy loss, mean square error loss, and contrastive learning loss as loss functions.
[0011] In an implementation of the first aspect, the cross entropy loss is used to train the first deep learning model, the mean square error loss is used to train the second deep learning model, and the contrastive learning loss is used to train the first deep learning model and the second learning model.
[0012] In second aspect, the present application provides an image analysis system, which includes: an image acquisition module for acquiring an ultrasonic image to be analyzed; a model processing module for using a first deep learning model to classify and process the ultrasonic image to be analyzed to obtain a classification result, and using a second deep learning model to perform quality detection on the ultrasonic image to be analyzed to obtain a quality detection result; a comprehensive analysis module for performing a comprehensive analysis of the ultrasonic image to be analyzed based on the classification result and the quality detection result to obtain an image analysis result; wherein, the first deep learning model and most of the second deep learning models are built using the same backbone network.
[0013] In a third aspect, the present application provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program stored in the memory, so that the electronic device performs the image analysis method as described in any one of the first aspects.
[0014] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image analysis method described in any one of the first aspects. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1A Shown is a schematic diagram of an application scenario of the image analysis method described in this application.
[0016] Figure 1B Shown are structural diagrams of the client-cloud interaction scenarios in these implementations.
[0017] Figure 2 Shown is a schematic flow chart of the image analysis method described in an embodiment of the present application.
[0018] Figure 3 Shown is a schematic flow chart of the image analysis method described in an embodiment of the present application.
[0019] Figure 4 Shown is a schematic flow chart of the image analysis method described in an embodiment of the present application.
[0020] Figure 5 Shown is a schematic flow chart of the image analysis method described in an embodiment of the present application.
[0021] Figure 6 Shown is a schematic flow chart of the image analysis method described in an embodiment of the present application.
[0022] Figure 7 Shown is a structural schematic diagram of the image analysis system described in an embodiment of the present application.
[0023] Figure 8Shown is a structural schematic diagram of an electronic device described in an embodiment of the present application.
[0024] Component number description
[0025] 1 Image analysis device
[0026] 11 Image acquisition equipment
[0027] 12 Local Processor
[0028] 13 Display Terminal
[0029] 2-end-cloud interactive system
[0030] 20 Terminal
[0031] 21 Cloud Servers
[0032] 100 Image Analysis System
[0033] 110 Image Acquisition Module
[0034] 120 Model processing module
[0035] 130 Comprehensive Analysis Module
[0036] 800 Electronic Equipment
[0037] 810 Memory
[0038] 820 processor
[0039] 830 Display
[0040] Steps S11 to S14
[0041] Steps S21 to S23
[0042] Steps S221 to S223
[0043] Steps S231 to S237 DETAILED DESCRIPTION
[0044] The following describes the embodiments of the present application through specific examples. Those skilled in the art can easily understand the other advantages and effects of the present application from the content disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.
[0045] It should be noted that in the embodiments of this application, words such as "optionally" or "for example" represent examples, illustrations, or descriptions. Any embodiment or design described in this application as "optionally" or "for example" should not be interpreted as being more preferred or advantageous than other embodiments or designs. Rather, the use of words such as "optionally" or "for example" is intended to present the relevant concepts in a concrete manner.
[0046] In the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can represent: a, b, c, ab, ac, bc or abc, where a, b, c can be single or multiple.
[0047] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application. Therefore, the illustrations only show components related to the present application and are not drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component can be changed at will, and the component layout type may also be more complicated.
[0048] The quality detection of ultrasound images under the eFAST specification has the following problems: High data annotation cost: The high accuracy of deep learning models depends on a large amount of labeled data, but the annotation of ultrasound images needs to be completed by professional physicians, which is time-consuming and expensive. Although the active learning mechanism can reduce the amount of labeled data, representative samples still need to be selected in practical applications, and this process poses great technical challenges. Insufficient real-time performance of image quality detection: Existing methods cannot provide real-time feedback on image quality during the inspection process, resulting in a large number of low-quality images being collected during the operation, increasing the workload of screening and analysis. Imperfect multi-view classification: eFAST examinations involve standard views of multiple anatomical regions, and each view requires specific image quality standards. Traditional methods generally lack support for classification and quality control of multiple views.
[0049] At least to address the above-mentioned problems, an embodiment of the present application provides an image analysis method, which includes: obtaining an ultrasound image to be analyzed; using a first deep learning model to classify the ultrasound image to be analyzed to obtain a classification result, and using a second deep learning model to perform quality detection on the ultrasound image to be analyzed to obtain a quality detection result; performing a comprehensive analysis on the ultrasound image to be analyzed based on the classification result and the quality detection result to obtain an image analysis result; wherein, the first deep learning model and most of the second deep learning models are built using the same backbone network.
[0050] In this embodiment of the present application, a first deep learning model is used to classify ultrasound images to be analyzed, and a second deep learning model is used to perform quality inspection on the ultrasound images to be analyzed. The classification results and quality inspection results are integrated for comprehensive analysis to obtain image analysis results. This image analysis method integrates image classification and quality inspection into the same model, achieving efficient view classification and quality inspection, and improving the accuracy and consistency of ultrasound image classification and quality inspection.
[0051] Figure 1A The image analysis device 1 can be used to implement the image analysis method provided in the embodiment of the present application, but the application scenario of the image analysis method provided in the embodiment of the present application is not limited to Figure 1A The image analysis device 1 shown. Figure 1A As shown, the image analysis apparatus 1 includes an image acquisition device 11, a local processor 12, and a display terminal 13. The image analysis method provided in the embodiment of the present application can be applied to the local processor 12.
[0052] in, Figure 1A The local processor 12 in the embodiment can be a local processor or a local processor cluster composed of multiple local processors or a cloud computing center, etc., which is not limited here. Figure 1A Only one image acquisition device 11, one local processor 12 and one display terminal 13 are shown in FIG, but it should be understood that Figure 1A The examples are only used to understand this solution, and the specific numbers of local processors 12 and display terminals 13 should be flexibly determined based on actual conditions.
[0053] In other implementations, the image analysis device 1 may not include the display terminal 13, but may only include a local processor 12 with a display function and an image acquisition device 11. The image analysis method provided in the embodiments of the present application can be applied to the local processor 12. The local processor 12 with a display function may include a tablet computer, a laptop computer, a PDA, a mobile phone, a personal computer, or a voice interaction device, without limitation herein.
[0054] In still other implementations, the image analysis method described in this application can be applied to end-cloud interaction scenarios. Figure 1B Shown is a schematic diagram of the structure of the end-cloud interaction scenario in these implementation methods. Figure 1B As shown, the terminal-cloud interaction system 2 includes a terminal 20 and a cloud server 21. The terminal 20 and the cloud server 21 can communicate with each other, and the communication method is not limited to wired or wireless.
[0055] Among them, the terminal 20 can be mobile or fixed. For example, the terminal 20 can be a wireless terminal or a wired terminal. The wireless terminal can refer to a device with wireless transceiver functions, which can be deployed indoors, outdoors, and in industrial workshops. The terminal 20 can be an ultrasonic analyzer, a mobile phone, a tablet computer, a laptop computer, etc., which is not limited here. The cloud server 21 can include one or more servers, or one or more processing nodes, or one or more virtual machines running on the server. The cloud server 21 can also be called a server cluster, a management platform, a data processing center, etc., which is not limited in the embodiments of the present application.
[0056] The technical solutions in the embodiments of the present application will be described in detail below with reference to the accompanying drawings in the embodiments of the present application.
[0057] The following examples of the present application provide an image analysis method, for example, Figure 1A The local processor 12 shown or Figure 1B It is implemented by the cloud server 21 shown. Figure 2 The flowchart of the image analysis method described in the embodiment of the present application is shown as follows: Figure 2 As shown, the image analysis method includes steps S11 to S14.
[0058] Step S11: Acquire the ultrasound image to be analyzed.
[0059] Step S12: Use the first deep learning model to classify the ultrasound image to be analyzed to obtain a classification result.
[0060] Step S13: Use the second deep learning model to perform quality detection on the ultrasound image to be analyzed to obtain a quality detection result.
[0061] Step S14: Comprehensively analyze the ultrasound image to be analyzed based on the classification result and the quality detection result to obtain an image analysis result. The first deep learning model and the majority of the second deep learning models are built using the same backbone network.
[0062] In some possible implementations, an ultrasound image to be analyzed is obtained, where the ultrasound image is an ultrasound image obtained during an eFAST examination and covers key areas such as the abdominal cavity, thoracic cavity, and heart. The ultrasound image to be analyzed is classified using a first deep learning model to obtain a classification result. The first deep learning model is a lightweight and efficient image classification model, which is used to classify the ultrasound image obtained during the eFAST examination and obtain a classification result. The ultrasound image to be analyzed is quality-checked using a second deep learning model to obtain a quality check result. The second deep learning model is a multi-task learning model based on a pyramid network structure, which is used to output a quality score as the quality check result. The first deep learning model and the second deep learning model share a MobileNet backbone network and use the same model parameters and features. The ultrasound image to be analyzed is comprehensively analyzed based on the classification result and the quality check result to obtain an analysis result. The analysis result integrates the quality score and classification of the ultrasound image to be analyzed, integrating the view classification and quality assessment tasks into the same backbone network framework.
[0063] In this embodiment of the present application, a first deep learning model is used to classify ultrasound images to be analyzed, and a second deep learning model is used to perform quality inspection on the ultrasound images to be analyzed. The classification results and quality inspection results are integrated for comprehensive analysis to obtain image analysis results. This image analysis method integrates image classification and quality inspection into the same model, achieving efficient view classification and quality inspection, and improving the accuracy and consistency of ultrasound image classification and quality inspection.
[0064] Figure 3 The flowchart of the image analysis method described in the embodiment of the present application is shown as follows: Figure 3 As shown, training the first deep learning model and the second deep learning model includes steps S21 to S23.
[0065] Step S21: Acquire ultrasound image data.
[0066] Step S22: labeling and preprocessing the ultrasound image data to obtain training data.
[0067] Step S23: Using the training data, train the initial first deep learning model and the initial second deep learning model to obtain the first deep learning model and the second deep learning model. The first deep learning model and the second deep learning model are built using the same backbone network.
[0068] In some possible implementations, ultrasound image data collected during eFAST examinations are compiled into an ultrasound image dataset, in which ultrasound images cover key areas such as the abdominal cavity, thoracic cavity, and heart, ensuring data diversity and representativeness. The ultrasound image data includes ultrasound images of different quality levels so that the model can effectively learn quality differentiation capabilities. The ultrasound image data is annotated, including the location and boundaries of tissue structures, as well as quality features. At the same time, the ultrasound images are preprocessed by grayscale conversion, normalization, etc. to obtain training data. The initial MobileNet V3 model is trained using the training data. The first deep learning model classifies images into various view categories of eFAST to ensure that the correct view category is input into the second deep learning model. The initial multi-scale feature fusion model based on the feature pyramid network is trained using the training data. The second deep learning model improves the differentiation of different quality levels by learning shared features through multi-task learning and combining supervised contrastive learning. During the model training process, the first deep learning model and the second deep learning model use shared backbone network features to achieve information sharing between tasks.
[0069] Figure 4 The flowchart of the image analysis method described in the embodiment of the present application is shown as follows: Figure 4 As shown, step S22 includes steps S221 to S223.
[0070] Step S221 : labeling the ultrasound image data to obtain the category, tissue structure, and quality characteristics of the ultrasound image data.
[0071] Step S222: performing quality classification on the ultrasound image data to obtain image quality levels of the ultrasound image data.
[0072] Step S223: performing normalization, denoising and / or image enhancement processing on the ultrasound image to obtain the training data.
[0073] In some possible implementations, the ultrasound image data is annotated to mark the boundaries and locations of tissue structures, such as the abdominal cavity, thoracic cavity, and heart. Furthermore, quality features of the ultrasound images, such as the clarity and contrast of tissue structures, are annotated to facilitate subsequent model training. Each ultrasound image is graded based on four criteria: overall outline, key tissue structures and clarity, standard view display, and appropriateness of parameter adjustments. The image quality rating is used as a label for supervised learning in the second deep learning model. Various preprocessing methods are performed to enhance image quality and unify the data format. These preprocessing methods include image normalization, which linearly transforms each pixel value in each ultrasound image to the range [0, 1] to improve image contrast. A Gaussian filter is applied to each ultrasound image to denoise it and reduce noise interference. Image enhancement is performed on each ultrasound image, using histogram equalization to enhance image contrast and make the thyroid structure in the ultrasound image clearer. These preprocessing methods ensure the diversity and representativeness of the training data, laying the foundation for subsequent feature extraction and model training. The above is only one possible implementation method of the embodiment of the present application. In actual application, the steps S221 to S223 can be randomly combined and the order can be adjusted. The present application is not limited to this.
[0074] Figure 5 The flowchart of the image analysis method described in the embodiment of the present application is shown as follows: Figure 5 As shown, step S23 includes steps S231 to S233.
[0075] Step S231: Use the initial first deep learning model to perform feature extraction on the training data of the labeled image category to obtain image features.
[0076] Step S232: Map the image features to multiple categories to obtain a multi-category output result.
[0077] Step S233: Mapping the multi-classification output results to a preset interval using an activation function to obtain the classification probability of the image.
[0078] In some possible implementations, the initial first deep learning model uses a MobileNet V3 model as a view classification branch, performs multi-class classification on eFAST images based on a lightweight network, and divides the images into different eFAST view categories (such as abdominal view, thoracic view, etc.). Specifically, the training data of the labeled image category is represented by a three-dimensional tensor with a shape of H×W×C, where H is the height of the ultrasound image in the training data, W is the width of the ultrasound image in the training data, and C is the number of channels of the ultrasound image in the training data. The annotation information includes the eFAST image category, for example, abdominal view, thoracic view. The input image has a training data shape of H×W×C, and the initial first deep learning model MobileNet V3 model is used to extract ultrasound image features, wherein the MobileNet V3 model includes a standard convolution layer, a depth-separable convolution layer, and an attention module. The image features are mapped to a multi-classification output to obtain a multi-classification output result. The output layer then uses a sigmoid activation function to map the multi-classification output result to the [0, 1] interval to obtain the classification probability of the image.
[0079] Figure 6 The flowchart of the image analysis method described in the embodiment of the present application is shown as follows: Figure 6 As shown, step S23 includes steps S234 to S237.
[0080] Step S234: Use the initial second deep learning model to perform fusion processing on the image features to obtain a fusion feature map.
[0081] Step S235 : performing upsampling processing on the fusion image to obtain a feature image with the same resolution.
[0082] Step S236: splicing the feature maps of the same resolution to obtain a fused feature matrix.
[0083] Step S237: Use a multi-layer perceptron to perform mapping processing on the fused feature matrix to obtain an image quality detection result.
[0084] In some possible implementations, the initial second deep learning model is based on a pyramid network structure, shares the features of the MobileNet V3 model backbone network through multi-task learning, and outputs image quality detection results. The branch of the initial second deep learning model outputs a continuous score value, representing the quality level of the image. The multi-scale image features obtained by the first deep learning model are represented by S1, S2, S3, and S4. The image features are fused using a pyramid network structure using the initial second deep learning model, starting from the top-level features and fusing low-resolution features layer by layer to retain high-level semantic information while gradually adding low-level details. The calculation formula is:
[0085] P4=S4
[0086] P3=f up (S3)+S3
[0087] P2=f up (S2)+S2
[0088] P1=f up (S1)+S1,
[0089] Among them, f up represents the upsampling operation, and P1, P2, P3, and P4 represent the fused feature maps.
[0090] The fusion image is upsampled to obtain a feature map of the same resolution. The calculation formula is:
[0091] P i '=f up (P i) ,i=2,3,4,
[0092] Among them, P i ′ is a feature map with the same resolution.
[0093] The feature maps of the same resolution are spliced together to obtain a fused feature matrix. The calculation formula for the fused feature matrix is:
[0094] F fusion =concat(P1',P2',P3',P4'),
[0095] Among them, F fusion is the fused feature matrix.
[0096] The fused feature matrix is mapped using a multi-layer perceptron to obtain the image quality detection result. First, the fused feature matrix is globally averaged pooled, and the calculation formula is:
[0097]
[0098] Among them, z fusion Represents the output result of global average pooling, H and W represent the height and width of the input training data, and F fusion Represents the fused feature matrix. Mapping is performed through several fully connected layers MLP to output the image quality detection result. The calculation formula for outputting the image quality detection result is:
[0099]
[0100] During the training process of the initial second deep learning model, the mean square error loss is used as the loss function. The calculation formula of the loss function is:
[0101]
[0102] Among them, L quality is the loss function, is the true image quality detection result of the i-th ultrasound image, Image quality detection results predicted by the model.
[0103] In one embodiment of the present application, when training the first deep learning model and the second deep learning model, cross entropy loss, mean square error loss and contrastive learning loss are used as loss functions to train the first deep learning model and the second learning model.
[0104] In one embodiment of the present application, the cross entropy loss is used to train the first deep learning model, the mean square error loss is used to train the second deep learning model, and the contrastive learning loss is used to train the first deep learning model and the second learning model.
[0105] In some possible implementations, the first deep learning model and the second deep learning model share the same backbone network structure. During the multi-task learning process of the first deep learning model and the second deep learning model, cross entropy loss is used as the loss function for training the first deep learning model, mean square error loss is used as the loss function for training the second deep learning model, and contrastive learning loss is used as the loss function for the entire image analysis model. The loss function is calculated as follows:
[0106]
[0107] L total =αL class +βL quality +γL contrastive ,
[0108] Among them, sim(zi ,z j ) represents the similarity between the features of two samples, τ is the temperature coefficient, L class is the cross entropy loss of the first deep learning model, L quality is the mean square error loss of the second deep learning model, L contrastive It is the contrast loss between the first deep learning model and the second deep learning model, and is used in the view classification task of the first deep learning model to enhance the feature differentiation of different view categories.
[0109] In some other possible implementations, the training process for the first and second deep learning models includes: initializing backbone network weights using a pretrained MobileNet V3 model; sharing backbone network features during training, and simultaneously optimizing parameters of the first and second deep learning models; updating parameters of the first and second deep learning models using gradient descent by adaptively balancing terms in the loss function; and implementing an early stopping strategy based on performance on a validation dataset to prevent model overfitting.
[0110] Figure 7 Shown is a structural diagram of the image analysis system described in an embodiment of the present application, as shown in FIG. Figure 7 As shown, the image analysis system 100 includes an image acquisition module 110 , a model processing module 120 and a comprehensive analysis module 130 .
[0111] The image acquisition module 110 is used to acquire ultrasound images to be analyzed.
[0112] The model processing module 120 is used to use the first deep learning model to classify the ultrasound image to be analyzed to obtain a classification result, and use the second deep learning model to perform quality detection on the ultrasound image to be analyzed to obtain a quality detection result.
[0113] The comprehensive analysis module 130 is used to perform a comprehensive analysis on the ultrasound image to be analyzed based on the classification result and the quality detection result to obtain an image analysis result. Wherein, the first deep learning model and the majority of the second deep learning models are built using the same backbone network.
[0114] In some possible implementations, the image acquisition module 110 is used to acquire ultrasound images to be analyzed, which are ultrasound images in eFAST examinations, and are ultrasound images covering key parts such as the abdominal cavity, thoracic cavity, and heart. The model processing module 120 is used to classify the ultrasound images to be analyzed using a first deep learning model to obtain classification results. The first deep learning model is a lightweight and efficient image classification model, which is used to classify the ultrasound images obtained by the eFAST examination and obtain classification results. The model processing module 120 is also used to perform quality inspection on the ultrasound images to be analyzed using a second deep learning model to obtain quality inspection results. The second deep learning model is a multi-task learning model based on a pyramid network structure, which is used to output a quality score as the quality inspection result. The first deep learning model and the second deep learning model share the MobileNet backbone network and use the same model parameters and features. The comprehensive analysis module 130 is used to perform a comprehensive analysis of the ultrasound images to be analyzed based on the classification results and the quality inspection results to obtain analysis results. The analysis results integrate the quality scoring and classification of the ultrasound images to be analyzed, and integrate the view classification and quality assessment tasks into the same backbone network framework.
[0115] In this embodiment of the present application, the model processing module 120 is used to classify the ultrasound images to be analyzed using a first deep learning model and to perform quality inspection on the ultrasound images to be analyzed using a second deep learning model. The comprehensive analysis module 130 is used to integrate the classification results and quality inspection results for comprehensive analysis to obtain image analysis results. This image analysis system 100 can integrate image classification and quality inspection into the same model, achieving efficient view classification and quality inspection, and improving the accuracy and consistency of ultrasound image classification and quality inspection.
[0116] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices or methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules / units is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules or units, which can be electrical, mechanical or other forms.
[0117] The modules / units described as separate components may or may not be physically separate, and the components displayed as modules / units may or may not be physical modules, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules / units may be selected according to actual needs to achieve the purpose of the embodiments of the present application. For example, the functional modules / units in the various embodiments of the present application may be integrated into a processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into a single module / unit.
[0118] Those skilled in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0119] An embodiment of the present application also provides an electronic device. Figure 8 Shown is a schematic diagram of the structure of the electronic device 800 according to an embodiment of the present application. Figure 8 As shown, in this embodiment, the electronic device 800 includes a memory 810 and a processor 820.
[0120] The memory 810 is used to store computer programs; preferably, the memory 810 includes: ROM, RAM, magnetic disk, USB flash drive, memory card or optical disk, etc., various media that can store program codes.
[0121] Specifically, the memory 810 may include a computer system readable medium in the form of a volatile memory, such as a random access memory (RAM) and / or a cache memory. The electronic device 800 may further include other removable / non-removable, volatile / non-volatile computer system storage media. The memory 810 may include at least one program product having a set (e.g., at least one) program modules that are configured to perform the functions of the various embodiments of the present application. It is understood that the memory 810 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM) or a programmable read-only memory (PROM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM) and synchronous static random access memory (SSRAM). The memory described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable categories of memory.
[0122] The processor 820 is connected to the memory 810 and is used to execute the computer program stored in the memory 810 so that the electronic device 800 performs the image analysis method described in any embodiment of the present application.
[0123] Optionally, the processor 820 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.
[0124] Optionally, the electronic device 800 in this embodiment may further include a display 830. The display 830 is communicatively connected to the memory 810 and the processor 820, and is used to display a graphical user interface (GUI) interactive interface related to the image analysis method described in the embodiment of the present application.
[0125] The present application also provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the image analysis method described in any embodiment of the present application is implemented.
[0126] As used in this specification, the terms "component," "module," "system," and the like are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. By way of illustration, both an application running on a computing device and a computing device can be a component. One or more components can reside in a process and / or an execution thread, and a component can be located on a computer and / or distributed between two or more computers. In addition, these components can be executed from various computer-readable media having various data structures stored thereon. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component on a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).
[0127] The descriptions of the processes or structures corresponding to the above figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.
[0128] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical concepts disclosed in this application shall be covered by the claims of this application.
Claims
1. An image analysis method, characterized in that: include: Acquiring an ultrasound image to be analyzed; Using the first deep learning model to classify the ultrasound image to be analyzed to obtain a classification result, and using the second deep learning model to perform quality detection on the ultrasound image to be analyzed to obtain a quality detection result; Performing a comprehensive analysis on the ultrasound image to be analyzed according to the classification result and the quality detection result to obtain an image analysis result; Among them, the first deep learning model and most of the second deep learning models are built using the same backbone network.
2. The image analysis method according to claim 1, characterized in that: Training the first deep learning model and the second deep learning model includes: Acquiring ultrasound image data; Annotating and preprocessing the ultrasound image data to obtain training data; The training data is used to train an initial first deep learning model and an initial second deep learning model to obtain the first deep learning model and the second deep learning model, wherein the first deep learning model and the second deep learning model are built using the same backbone network.
3. The image analysis method according to claim 2, characterized in that: Labeling and preprocessing the ultrasound image data to obtain training data includes: Performing labeling processing on the ultrasound image data to obtain the category, organizational structure and quality characteristics of the ultrasound image data; Performing quality classification on the ultrasound image data to obtain image quality levels of the ultrasound image data; and The ultrasonic image is normalized, denoised and / or image enhanced to obtain the training data.
4. The image analysis method according to claim 2, characterized in that: Training the initial first deep learning model using the training data includes: Using the initial first deep learning model to perform feature extraction on training data of labeled image categories to obtain image features; Mapping the image features to multiple categories to obtain a multi-category output result; The multi-classification output results are mapped to a preset interval using an activation function to obtain the classification probability of the image.
5. The image analysis method according to claim 4, characterized in that: Training the initial second deep learning model using the training data includes: Using the initial second deep learning model to perform fusion processing on the image features to obtain a fusion feature map; Upsampling the fusion image to obtain a feature image with the same resolution; The feature maps with the same resolution are spliced to obtain a fused feature matrix; The fused feature matrix is mapped using a multi-layer perceptron to obtain an image quality detection result.
6. The image analysis method according to claim 1, characterized in that: When training the first deep learning model and the second deep learning model, the first deep learning model and the second deep learning model are trained using cross entropy loss, mean square error loss and contrastive learning loss as loss functions.
7. The image analysis method according to claim 6, characterized in that: The first deep learning model is trained using the cross entropy loss, the second deep learning model is trained using the mean square error loss, and the first deep learning model and the second learning model are trained using contrastive learning loss.
8. An image analysis system, characterized in that: include: An image acquisition module, used to acquire an ultrasonic image to be analyzed; A model processing module, used to classify the ultrasound image to be analyzed using the first deep learning model to obtain a classification result, and to perform quality detection on the ultrasound image to be analyzed using the second deep learning model to obtain a quality detection result; A comprehensive analysis module, used to perform a comprehensive analysis on the ultrasound image to be analyzed according to the classification result and the quality detection result to obtain an image analysis result; Among them, the first deep learning model and most of the second deep learning models are built using the same backbone network.
9. An electronic device, characterized in that: The electronic device comprises: Memory for storing computer programs; A processor, wherein the processor is used to execute the computer program stored in the memory so that the electronic device performs the image analysis method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the image analysis method according to any one of claims 1 to 7 is implemented.