Image detection method and system based on fusion medical image learning

Abstract

The invention relates to the technical field of medical image processing, in particular to an image detection method and system based on fusion medical image learning, and the method comprises the following steps: obtaining a medical image; based on the medical image, training a medical image detection model for detecting a medical target in the medical image by using a deep neural network; and carrying out lightweight processing on the medical image detection model to obtain a medical image detection lightweight model for rapidly detecting the medical target in the medical image. According to the invention, the medical image detection model used for detecting the medical target in the medical image is trained by using the deep neural network, the high-precision medical image detection model can be obtained, and then the medical image detection model is subjected to lightweight processing to obtain the medical image detection lightweight model. The detection and recognition effect of the medical target is effectively guaranteed, meanwhile, the model structure is light, and the defect that the highlighting efficiency is insufficient is overcome for a large number of medical images.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to an image detection method and system based on fusion medical image learning. Background Art

[0002] The purpose of medical image detection is to detect medical targets in medical images, such as the location of lesions, blood vessels, etc. There are many methods currently used for medical image detection, most of which use deep learning methods to perform image recognition processing to locate the location information of medical targets. In order to pursue the accuracy of medical image detection, more complex model structures will be used in deep learning. Although this can effectively guarantee the detection and recognition effect of medical targets, the complex model structure will also cause a huge computational burden, resulting in a long computational time. Faced with a large number of medical images, it is easier to highlight the defect of insufficient efficiency.

[0003] Therefore, the high-precision medical image detection model in the existing technology will cause a huge computational burden due to its complex model structure, prolong the calculation time, and is more likely to show the defect of insufficient efficiency when faced with a large number of medical images. Summary of the invention

[0004] The purpose of the present invention is to provide an image detection method and system based on fusion medical image learning, so as to solve the technical problem that the high-precision medical image detection model in the prior art will cause a huge computational burden and prolong the calculation time due to its complex model structure, and it is more likely to highlight the insufficient efficiency when facing a large number of medical images.

[0005] In order to solve the above technical problems, the present invention specifically provides the following technical solutions: An image detection method based on fusion medical image learning includes the following steps: Access to medical images; Based on the medical image, a medical image detection model for detecting medical targets in the medical image is trained using a deep neural network; The medical image detection model is lightweighted to obtain a lightweight model for medical image detection for quickly detecting medical targets in medical images.

[0006] As a preferred embodiment of the present invention, the medical images include MRI images, CT images, and ultrasound images.

[0007] As a preferred solution of the present invention, the deep neural network includes a contour feature extraction layer, a contour feature fusion layer, a contour feature pooling layer and a contour detection output layer; Wherein, the contour feature extraction layer is used to extract the contour features of the medical target in the medical image; The contour feature fusion layer is used to enhance the contour features of the medical target in the medical image; The contour feature pooling layer is used to perform pooling processing on the contour features; The contour detection output layer is used to output the contour information of the medical target in the medical image according to the contour feature.

[0008] As a preferred solution of the present invention, the method for extracting the contour feature by the contour feature extraction layer includes: Use the DenseNet121 network to obtain multiple levels of features in medical images; The highest level features among multiple levels of features are upsampled and concatenated with the lowest level features, and then processed by the channel attention mechanism to obtain the contour features; The contour feature extraction layer obtains the contour feature extraction formula as follows: ; In the formula, is the contour feature, The DenseNet121 network obtains multiple levels of features in medical images. max is the maximum identifier and min is the minimum identifier. It is the highest level feature among multiple levels of features. It is the lowest level feature among multiple levels of features, UP is the upsampling identifier, CAM is the channel attention mechanism identifier, For medical imaging.

[0009] As a preferred solution of the present invention, the method for enhancing contour features by contour feature fusion layer includes: Use the Canny operator to perform edge detection on medical images and extract edge pixels of medical targets in medical images; Binarize edge pixels in the medical image and create an edge mask; The contour features are weighted by edge masks and then fused with the highest level features after upsampling to obtain enhanced contour features. The enhancement formula of the contour feature fusion layer on the contour feature is: ; In the formula, For the enhanced contour features, It is the highest level feature among multiple level features, UP is the upsampling identifier, and max is the maximum identifier. is the Canny operator identifier, Bin is the binary operation identifier, is the edge pixel of the medical target in the medical image, is the edge mask, is the contour feature, The DenseNet121 network obtains multiple levels of features in medical images. For medical imaging.

[0010] As a preferred solution of the present invention, the pooling process of the contour feature pooling layer is a maximum pooling operation, and the maximum pooling operation formula is: ; In the formula, is the contour feature after pooling. is the max pooling operation identifier, It is the enhanced contour feature.

[0011] As a preferred solution of the present invention, the contour detection output layer is a fully connected layer composed of a 1*1 convolutional layer and a ReLU activation function, and the analysis formula of the contour detection output layer for the medical target contour information is: ; In the formula, For medical target profile information, is the contour feature after pooling. is the 1*1 convolutional layer identifier, is the ReLU activation function identifier.

[0012] As a preferred solution of the present invention, a method for constructing a medical image detection model includes: Select multiple medical images and mark the contour information of the medical target in the multiple medical images; The data set consisting of multiple medical features and contour information of medical targets is divided into a training set and a test set; Using the training set, the mean square error (MSE) between the output of the deep neural network and the true value of the contour information is used as the loss function to train the deep neural network to obtain a medical image detection model. The test set is used to evaluate the performance of the medical image detection model.

[0013] As a preferred solution of the present invention, a method for constructing a lightweight model for medical image detection includes: Using the test set, the mean square error (MSE) between the output of the medical image detection model and the output of the lightweight medical image detection model, as well as the mean square error (MSE) between the output of the lightweight medical image detection model and the true value of the contour information are used as loss functions to train the MobileNet network to obtain the lightweight medical image detection model. Use the test set to evaluate the performance of the lightweight model for medical image detection; The lightweight model for medical image detection is: ; In the formula, The contour information output by the lightweight model for medical image detection, is the MobileNet network identifier, For medical imaging.

[0014] As a preferred solution of the present invention, the present invention provides an image detection system based on fusion medical images, which is applied to an image detection method based on fusion medical image learning. The system includes: A data acquisition unit for acquiring medical images; A model building unit, configured to train a medical image detection model for detecting medical targets in the medical image using a deep neural network based on the medical image; A lightweight unit, used for performing lightweight processing on the medical image detection model to obtain a lightweight medical image detection model for quickly detecting medical targets in medical images; The detection application unit is used to quickly detect medical targets in medical images using a lightweight model for medical image detection and obtain contour information of the medical targets.

[0015] Compared with the prior art, the present invention has the following beneficial effects: The present invention utilizes a deep neural network to train a medical image detection model for detecting medical targets in medical images, and is able to obtain a high-precision medical image detection model. The medical image detection model is then lightweighted to obtain a lightweight medical image detection model for quickly detecting medical targets in medical images. This effectively ensures the detection and recognition effects of medical targets while making the model structure lightweight, thus avoiding the defect of insufficient efficiency in the face of a large number of medical images. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0017] Figure 1 A flow chart of an image detection method based on fusion medical image learning provided by an embodiment of the present invention; Figure 2 A block diagram of an image detection system based on fusion medical image learning provided by an embodiment of the present invention; Figure 3 An edge mask provided by an embodiment of the present invention; Figure 4 The output result of the contour detection output layer provided by the embodiment of the present invention; Figure 5 This is an effect diagram of mapping the output result of the contour detection output layer provided in an embodiment of the present invention onto the original medical image. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] like Figure 1 As shown, the present invention provides an image detection method based on fusion medical image learning, comprising the following steps: Access to medical images; Based on medical images, a medical image detection model is trained using deep neural networks to detect medical targets in medical images; The medical image detection model is lightweighted to obtain a lightweight model for medical image detection for quickly detecting medical targets in medical images.

[0020] In order to ensure high-precision detection of medical targets in medical images, that is, to obtain accurate contour information of medical targets, the present invention uses a deep neural network with a complex model structure and multiple functions for model training, so as to realize a detection model (i.e., a medical image detection model) for accurately obtaining contour information of medical targets in medical images.

[0021] The deep neural network used in the medical image detection model contains four modules, namely, contour feature extraction layer, contour feature fusion layer, contour feature pooling layer and contour detection output layer; Among them, the first module is the contour feature extraction layer, which is used to extract the contour features of medical targets in medical images. First, multi-level feature extraction is performed in medical images to obtain comprehensive multi-level features. High-level features contain semantic information, mostly describing abstract concepts such as object categories and attributes, and low-level features contain detailed information such as contours and textures, mostly describing concrete information such as shape and position. Then, after obtaining multi-level features, the lowest-level features are spliced ​​with the highest-level features. Since the lowest-level features contain the most detailed information, the highest-level features contain the most semantic information, and there is isolation between the highest-level features and the lowest-level features, there is the least cross-information between the two. By splicing the two and using the attention mechanism, features containing the richest detailed information and the richest semantic information for identifying medical targets can be obtained, and they are independent of each other, that is, the features (contour features) that are most relevant to identifying medical targets and have the least redundancy are obtained, achieving the best feature extraction and laying the foundation for the subsequent model.

[0022] The second module is the contour feature fusion layer, which is used to enhance the contour features of medical targets in medical images. First, the contour lines of medical targets in medical images are estimated through edge detection, and they are made into masks as weighted weights of contour features, so that the contour information in the contour features weighted by the edge mask is enhanced, and the rest of the information remains unchanged, enhancing the difference between contour information and non-contour information. Then, since the contour features after contour information enhancement may lose detail information after enhancement, the contour features after contour information enhancement are fused with the lowest level features, so that the detail information lost by the contour features after contour information enhancement is filled in by the lowest level features again. Contour information enhancement ultimately achieves further enhancement of contour information in the features (contour features) that are most relevant to the identification of medical targets and have the least redundancy, making it easier to capture contour information in subsequent processing, thereby more accurately detecting the contour information of medical targets.

[0023] The third module is the contour feature pooling layer, which is used to perform pooling processing on the contour features. The present invention uses maximum pooling processing to reduce the spatial size of the feature map in the aforementioned enhanced contour features, reduce the computational complexity of the subsequent layers, and retain the most important feature information by selecting the maximum value in the local area. It is also insensitive to slight changes in the input, enhancing the generalization ability of the model.

[0024] The fourth module is the contour detection output layer, which is used to output the contour information of the medical target in the medical image based on the contour features. The fully connected layer is used to process the pooled features to obtain the contour information of the medical target.

[0025] The medical image detection model is constructed based on the deep neural network of the above four modules. The excellent functionality of each module ensures the detection accuracy of the medical target contour information, but the model structure is complex. The medical image detection model is lightweighted to obtain a lightweight model for medical image detection that is used to quickly detect medical targets in medical images. This effectively ensures the detection and recognition effect of medical targets while lightweighting the model structure.

[0026] In order to achieve lightweight model structure, the present invention uses distillation learning mode, takes the medical image detection model as the teacher model, takes the medical image detection lightweight model as the student model, and uses the medical image detection model to guide the training of the medical image detection lightweight model. The lightweight model structure selects the MobileNet structure, which uses depth separable convolution to replace the standard convolution, and decomposes the convolution operation into two parts: depth convolution and point-by-point convolution, thereby reducing the number of parameters. During the training process, the parameters of the teacher model (medical image detection model) will be frozen and remain unchanged, and the student model (medical image detection lightweight model) will complete the reverse parameter update based on the total loss function value. The total loss function value will consist of two parts: the first part is the difference between the output result of the teacher model (medical image detection model) and the output result of the student model (medical image detection lightweight model), and the second part is the difference between the output result of the student model (medical image detection lightweight model) and the true annotation.

[0027] Therefore, after lightweight processing, the present invention can effectively ensure the detection and recognition effect of medical targets while making the model structure lightweight, thus avoiding the defect of insufficient efficiency in the face of a large number of medical images.

[0028] Medical images include MRI images, CT images, and ultrasound images.

[0029] The deep neural network includes a contour feature extraction layer, a contour feature fusion layer, a contour feature pooling layer, and a contour detection output layer; Among them, the contour feature extraction layer is used to extract the contour features of medical targets in medical images; The contour feature fusion layer is used to enhance the contour features of medical targets in medical images; The contour feature pooling layer is used to perform pooling processing on the contour features; The contour detection output layer is used to output the contour information of medical targets in medical images according to the contour features (such as Figure 4 and Figure 5 As shown, Figure 4 is the output result of the contour detection output layer, Figure 5 This is the effect diagram of mapping the output result of the contour detection output layer onto the original medical image).

[0030] The method of extracting contour features in the contour feature extraction layer includes: Use the DenseNet121 network to obtain multiple levels of features in medical images; The highest level features among multiple levels of features are upsampled and concatenated with the lowest level features, and then processed by the channel attention mechanism to obtain the contour features; The contour feature extraction layer obtains the contour feature extraction formula as follows: ; In the formula, is the contour feature, The DenseNet121 network obtains multiple levels of features in medical images. max is the maximum identifier and min is the minimum identifier. It is the highest level feature among multiple levels of features. It is the lowest level feature among multiple levels of features, UP is the upsampling identifier, CAM is the channel attention mechanism identifier, For medical imaging.

[0031] First, multi-level feature extraction is performed in medical images to obtain comprehensive multi-level features. High-level features contain semantic information and mostly describe abstract concepts such as object categories and attributes. Low-level features contain detailed information such as contours and textures and mostly describe concrete information such as shapes and positions. Then, after obtaining multi-level features, the lowest-level features are spliced ​​with the highest-level features. Since the lowest-level features contain the most detailed information, the highest-level features contain the most semantic information, and there is isolation between the highest-level features and the lowest-level features, there is the least cross-information between the two. By splicing the two and using the attention mechanism, features containing the richest detailed information and the richest semantic information for identifying medical targets can be obtained, and they are independent of each other, that is, the features (contour features) that are most relevant to identifying medical targets and have the least redundancy are obtained, achieving the best feature extraction and laying the foundation for the subsequent model.

[0032] The method of enhancing contour features in the contour feature fusion layer includes: Use the Canny operator to perform edge detection on medical images and extract edge pixels of medical targets in medical images; The edge pixels in the medical image are binarized and created as edge masks (such as Figure 3 shown); The contour features are weighted by edge masks and then fused with the highest level features after upsampling to obtain enhanced contour features. The enhancement formula of the contour feature fusion layer on the contour feature is: ; In the formula, For the enhanced contour features, It is the highest level feature among multiple level features, UP is the upsampling identifier, and max is the maximum identifier. is the Canny operator identifier, Bin is the binary operation identifier, is the edge pixel of the medical target in the medical image, is the edge mask, is the contour feature, The DenseNet121 network obtains multiple levels of features in medical images. For medical imaging.

[0033] First, the contour lines of the medical target in the medical image are estimated through edge detection, and the contour lines are made into a mask as the weighted weight of the contour feature, so that the contour information in the contour feature weighted by the edge mask is enhanced, and the rest of the information remains unchanged, enhancing the difference between the contour information and the non-contour information. Then, since the contour feature may lose detail information after the contour information is enhanced, the contour feature after the contour information is enhanced is fused with the lowest level feature, so that the detail information lost by the contour feature after the contour information is enhanced is filled in by the lowest level feature again. Contour information enhancement ultimately achieves further enhancement of contour information in the features (contour features) that are most relevant to the identification of medical targets and have the least redundancy, making it easier to capture contour information in subsequent processing, thereby more accurately detecting the contour information of the medical target.

[0034] The pooling process of the contour feature pooling layer is the maximum pooling operation, and the maximum pooling operation formula is: ; In the formula, is the contour feature after pooling. is the max pooling operation identifier, It is the enhanced contour feature.

[0035] The present invention utilizes the maximum pooling process to reduce the spatial size of the feature map in the aforementioned enhanced contour features, reduce the computational complexity of the subsequent layers, and retain the most important feature information by selecting the maximum value in the local area. It is also insensitive to slight changes in the input, thus enhancing the generalization ability of the model.

[0036] The contour detection output layer is a fully connected layer composed of a 1*1 convolutional layer and a ReLU activation function. The analysis formula of the contour detection output layer for the contour information of the medical target is: ; In the formula, For medical target profile information, is the contour feature after pooling. is the 1*1 convolutional layer identifier, is the ReLU activation function identifier.

[0037] The construction method of medical image detection model includes: Select multiple medical images and mark the contour information of the medical target in the multiple medical images; The data set consisting of multiple medical features and contour information of medical targets is divided into a training set and a test set; Using the training set, the mean square error (MSE) between the output of the deep neural network and the true value of the contour information is used as the loss function to train the deep neural network to obtain a medical image detection model. The test set is used to evaluate the performance of the medical image detection model.

[0038] The lightweight model construction method for medical image detection includes: Using the test set, the mean square error (MSE) between the output of the medical image detection model and the output of the lightweight medical image detection model, as well as the mean square error (MSE) between the output of the lightweight medical image detection model and the true value of the contour information are used as loss functions to train the MobileNet network to obtain the lightweight medical image detection model. Use the test set to evaluate the performance of the lightweight model for medical image detection; The lightweight model for medical image detection is: ; In the formula, Detect contour information output by lightweight models for medical images. is the MobileNet network identifier, For medical imaging.

[0039] In order to achieve lightweight model structure, the present invention uses distillation learning mode, takes the medical image detection model as the teacher model, takes the medical image detection lightweight model as the student model, and uses the medical image detection model to guide the training of the medical image detection lightweight model. The lightweight model structure selects the MobileNet structure, which uses depth separable convolution to replace the standard convolution, and decomposes the convolution operation into two parts: depth convolution and point-by-point convolution, thereby reducing the number of parameters. During the training process, the parameters of the teacher model (medical image detection model) will be frozen and remain unchanged, and the student model (medical image detection lightweight model) will complete the reverse parameter update based on the total loss function value. The total loss function value will consist of two parts: the first part is the difference between the output result of the teacher model (medical image detection model) and the output result of the student model (medical image detection lightweight model), and the second part is the difference between the output result of the student model (medical image detection lightweight model) and the true annotation.

[0040] like Figure 2 As shown, the present invention provides an image detection system based on fusion medical images, which is applied to an image detection method based on fusion medical image learning. The system includes: A data acquisition unit for acquiring medical images; A model building unit, used for training a medical image detection model for detecting medical targets in medical images by using a deep neural network based on medical images; A lightweight unit, used for performing lightweight processing on the medical image detection model to obtain a lightweight medical image detection model for quickly detecting medical targets in medical images; The detection application unit is used to quickly detect medical targets in medical images using a lightweight model for medical image detection and obtain contour information of the medical targets.

[0041] The present invention utilizes a deep neural network to train a medical image detection model for detecting medical targets in medical images, and is able to obtain a high-precision medical image detection model. The medical image detection model is then lightweighted to obtain a lightweight medical image detection model for quickly detecting medical targets in medical images. This effectively ensures the detection and recognition effects of medical targets while making the model structure lightweight, thus avoiding the defect of insufficient efficiency in the face of a large number of medical images.

[0042] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.

Claims

1. An image detection method based on fusion medical image learning, characterized in that: The following steps are involved: Access to medical images; Based on the medical image, a medical image detection model for detecting medical targets in the medical image is trained using a deep neural network; The medical image detection model is lightweighted to obtain a lightweight model for medical image detection for quickly detecting medical targets in medical images.

2. The image detection method based on fusion medical image learning according to claim 1 is characterized in that: The medical images include MRI images, CT images, and ultrasound images.

3. The image detection method based on fusion medical image learning according to claim 1 is characterized in that: The deep neural network includes a contour feature extraction layer, a contour feature fusion layer, a contour feature pooling layer and a contour detection output layer; Wherein, the contour feature extraction layer is used to extract the contour features of the medical target in the medical image; The contour feature fusion layer is used to enhance the contour features of the medical target in the medical image; The contour feature pooling layer is used to perform pooling processing on the contour features; The contour detection output layer is used to output the contour information of the medical target in the medical image according to the contour feature.

4. The image detection method based on fusion medical image learning according to claim 3 is characterized in that: The method of extracting the contour feature in the contour feature extraction layer includes: Use the DenseNet121 network to obtain multiple levels of features in medical images; The highest level features among multiple levels of features are upsampled and concatenated with the lowest level features, and then processed by the channel attention mechanism to obtain the contour features; The contour feature extraction layer obtains the contour feature extraction formula as follows: ； In the formula, is the contour feature, The DenseNet121 network obtains multiple levels of features in medical images. max is the maximum identifier and min is the minimum identifier. It is the highest level feature among multiple levels of features. It is the lowest level feature among multiple levels of features, UP is the upsampling identifier, CAM is the channel attention mechanism identifier, For medical imaging.

5. The image detection method based on fusion medical image learning according to claim 4 is characterized in that: The method of enhancing contour features in the contour feature fusion layer includes: Use the Canny operator to perform edge detection on medical images and extract edge pixels of medical targets in medical images; Binarize edge pixels in the medical image and create an edge mask; The contour features are weighted by edge masks and then fused with the highest level features after upsampling to obtain enhanced contour features. The enhancement formula of the contour feature fusion layer on the contour feature is: ； In the formula, For the enhanced contour features, It is the highest level feature among multiple level features, UP is the upsampling identifier, and max is the maximum identifier. is the Canny operator identifier, Bin is the binary operation identifier, is the edge pixel of the medical target in the medical image, is the edge mask, is the contour feature, The DenseNet121 network obtains multiple levels of features in medical images. For medical imaging.

6. The image detection method based on fusion medical image learning according to claim 5 is characterized in that: The pooling process of the contour feature pooling layer is a maximum pooling operation, and the maximum pooling operation formula is: ； In the formula, is the contour feature after pooling. is the max pooling operation identifier, It is the enhanced contour feature.

7. The image detection method based on fusion medical image learning according to claim 6 is characterized in that: The contour detection output layer is a fully connected layer composed of a 1*1 convolutional layer and a ReLU activation function. The analysis formula of the contour detection output layer for the medical target contour information is: ； In the formula, For medical target profile information, is the contour feature after pooling. is the 1*1 convolutional layer identifier, is the ReLU activation function identifier.

8. The image detection method based on fusion medical image learning according to claim 7 is characterized in that: The construction method of medical image detection model includes: Select multiple medical images and mark the contour information of the medical target in the multiple medical images; The data set consisting of multiple medical features and contour information of medical targets is divided into a training set and a test set; Using the training set, the mean square error (MSE) between the output of the deep neural network and the true value of the contour information is used as the loss function to train the deep neural network to obtain a medical image detection model. The test set is used to evaluate the performance of the medical image detection model.

9. The image detection method based on fusion medical image learning according to claim 8 is characterized in that: The lightweight model construction method for medical image detection includes: Using the test set, the mean square error (MSE) between the output of the medical image detection model and the output of the lightweight medical image detection model, and the mean square error (MSE) between the output of the lightweight medical image detection model and the true value of the contour information are used as loss functions to train the MobileNet network to obtain the lightweight medical image detection model. Use the test set to evaluate the performance of the lightweight model for medical image detection; The lightweight model for medical image detection is: ； In the formula, The contour information output by the lightweight model for medical image detection, is the MobileNet network identifier, For medical imaging.

10. An image detection system based on fusion medical images, characterized in that: An image detection method based on fusion medical image learning applied to any one of claims 1 to 9, the system comprising: A data acquisition unit for acquiring medical images; A model building unit, configured to train a medical image detection model for detecting medical targets in the medical image using a deep neural network based on the medical image; A lightweight unit, used for performing lightweight processing on the medical image detection model to obtain a lightweight medical image detection model for quickly detecting medical targets in medical images; The detection application unit is used to quickly detect medical targets in medical images using a lightweight model for medical image detection and obtain contour information of the medical targets.

Images