Stone identification and positioning method based on deep separable convolution model and related equipment

Through the stone recognition method based on the depth separable convolution model, the problems of high calculation cost, few training data and low accuracy in kidney stone recognition are solved, and efficient and accurate diagnosis of kidney stones is achieved, which is suitable for medical treatment in remote areas. Facilities improve identification accuracy.

CN115100183BActive Publication Date: 2024-11-19CENT SOUTH UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210852780.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-07-11
Publication Date
2024-11-19
Estimated Expiration
2042-07-11

AI Technical Summary

Technical Problem

The prior art has problems such as high computational cost, few training data and low accuracy in the identification and classification of kidney stones, and the black box characteristics of deep learning models limit its application in medical clinical diagnosis.

Method used

The stone recognition and positioning method based on the depth separable convolution model is adopted. By obtaining image data for preprocessing and inputting the depth separable convolution model for identification, the model is trained using enhanced image data, and the model structure includes a separable convolution layer, Global pooling layer, dropout layer and normalization layer reduce computational complexity and improve accuracy.

Benefits of technology

It has achieved rapid and accurate diagnosis of kidney stones, significantly improved the recognition accuracy to 97.98%, reduced calculation costs, and was suitable for medical facilities that lack resources, and improved application capabilities in remote areas.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115100183B_ABST
    Figure CN115100183B_ABST
Patent Text Reader

Abstract

The present invention provides a stone identification and positioning method based on a deep separable convolution model and related equipment, the method comprising: step 1, obtaining image data to be identified; step 2, inputting the image data into the deep separable convolution model for stone identification, and obtaining an identification result, wherein the identification result includes the presence of stones in the image data or the absence of stones in the image data; step 3, when stones exist in the image data, visualizing the stone area on the image data, and obtaining a visualized positioning result; the method is not only used for the identification and positioning of kidney stones, but also can solve the problems of high computational cost, small amount of training data and low accuracy in the prior art.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of medical image recognition, and in particular to a stone recognition and positioning method based on a deep separable convolution model and related equipment. Background Art

[0002] For kidney stone identification and classification, large-scale clinical examinations usually generate a large number of medical images. In the past, doctors could only rely on manual identification of a large number of CT images. These routine procedures are very time-consuming and may lead to misdiagnosis when there is a shortage of nephrologists.

[0003] In recent years, deep learning (DL) models have recently been introduced into the scientific community, and deep learning models are also used in the field of urology to automatically identify and locate kidney stones. Convolutional neural network (CNN) is a powerful deep learning tool for various computer vision tasks such as object detection, classification, and medical image analysis. Deep learning-based methods have also shown excellent performance in detecting and classifying kidney stones. However, there are relatively few studies on the recognition and location of kidney stones, and the limitations of previous studies are: first, the training data used is small and cannot support reliable test results; second, deep learning models have many parameters, large storage space, high complexity, and high computational cost, which are not suitable for real-time kidney stone diagnosis; third. The existing research methods are not accurate enough for kidney stone recognition, and there is still a lot of room for improvement; fourth, the black box characteristics of deep learning neural networks seriously limit the application of deep learning in medical clinical diagnosis.

[0004] Therefore, the existing methods still have technical defects such as high computational cost, little training data and low accuracy. Summary of the invention

[0005] The present invention provides a stone identification and positioning method based on a deep separable convolutional model and related equipment, the purpose of which is to provide a fast and accurate diagnostic tool that is not only used for the identification and positioning of kidney stones, but also can solve the problems of high computational cost, small amount of training data and low accuracy in the prior art.

[0006] In order to achieve the above object, the present invention provides a stone identification and positioning method based on a deep separable convolution model, the method comprising:

[0007] Step 1, obtaining image data to be identified;

[0008] Step 2, inputting the image data into a depthwise separable convolutional model to perform stone recognition, and obtaining a recognition result, wherein the recognition result includes whether a stone exists in the image data or whether a stone does not exist in the image data;

[0009] Step 3: When stones exist in the image data, visualize the stone area on the image data to obtain a visualized positioning result.

[0010] Further, the image data is preprocessed;

[0011] The preprocessed image data is input into the deep separable convolutional model for stone recognition.

[0012] Furthermore, before the depthwise separable convolutional model is trained, the image data used to train the depthwise separable convolutional model is enhanced.

[0013] Specifically, the depthwise separable convolutional model includes a separable convolutional layer, a global pooling layer, a dropout layer 1, a dense layer 1, a normalization layer 1, a dropout layer 2, a dense layer 3, and a normalization layer 2, which are connected sequentially.

[0014] Among them, the kinetic energy of normalized layer 1 and normalized layer 2 is 0.99.

[0015] Furthermore, the depthwise separable convolutional model also includes a dense layer 3 for outputting recognition results, and the input end of the dense layer 3 is connected to the output end of the normalization layer 2.

[0016] The present invention also provides a stone identification and positioning device based on a deep separable convolution model, which is characterized by comprising:

[0017] An acquisition module, used for acquiring image data to be identified;

[0018] A processing module, used for preprocessing the image data to be identified;

[0019] A depthwise separable convolution module is used to perform stone recognition on the input pre-processed image data to obtain recognition results;

[0020] The visualization processing module is used to visualize the stone area on the image data when there is a stone in the image data, so as to obtain a visualization positioning result.

[0021] A computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed, the stone identification and positioning method based on the deep separable convolution model as described above is implemented.

[0022] The present invention also provides a stone identification and positioning device based on a deep separable convolution model, comprising a memory and a processor; the memory is used to store computer programs and intermediate data during program processing, and the processor is used to execute the computer program to implement the stone identification and positioning method based on a deep separable convolution model as described above.

[0023] The above technical solution of the present invention has the following beneficial effects:

[0024] The use of a deep separable convolutional model improves the trade-off between latency and accuracy, can effectively learn features from image data, and significantly reduce computational parameters and computational complexity; it is efficient and computationally inexpensive, and can be successfully applied to remote and rural areas that lack medical facilities for stone screening; it has a very high accuracy rate in identifying and locating stones, reaching 97.98%, greatly improving recognition accuracy.

[0025] Other beneficial effects of the present invention will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic diagram of a flow chart of an embodiment of the present invention;

[0027] Figure 2 2 is a structural diagram of a depthwise separable convolutional model in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0030] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a locking connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0031] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0032] In view of the existing problems, the present invention provides a stone identification and positioning method based on a deep separable convolution model and related equipment.

[0033] like Figure 1 As shown, a stone identification and positioning method based on a deep separable convolutional model, the method includes:

[0034] Step 1, obtaining image data to be identified;

[0035] Step 2, inputting the image data into a depthwise separable convolutional model to perform stone recognition, and obtaining a recognition result, wherein the recognition result includes whether a stone exists in the image data or whether a stone does not exist in the image data;

[0036] Step 3: When stones exist in the image data, visualize the stone area on the image data to obtain a visualized positioning result.

[0037] Specifically, before step 2, it also includes:

[0038] Preprocessing the image data;

[0039] The preprocessed image data is input into the deep separable convolutional model for stone recognition.

[0040] Specifically, before the depthwise separable convolutional model is trained, the image data used to train the depthwise separable convolutional model is enhanced.

[0041] Specifically, if Figure 2 As shown in the figure, the depth-wise separable convolution model includes a separable convolution layer, a global pooling layer, a dropout layer 1, a dense layer 1, a normalization layer 1, a dropout layer 2, a dense layer 2, and a normalization layer 2 connected in sequence.

[0042] Specifically, the kinetic energy of normalization layer 1 and normalization layer 2 are both 0.99 and the precision is 0.01.

[0043] The enhanced image data is passed through a separable convolutional layer to generate a multi-dimensional feature vector; the GAP global pooling layer flattens the multi-dimensional feature vector to obtain a one-dimensional feature map; the one-dimensional feature map is generated into a one-dimensional vector through dropout layer 1, dense layer 1, dropout layer 2, dense layer 2, and normalization layer 1 and normalization layer 2 are used after dense layer 1 and dense layer 2 respectively; the recognition result is output through dense layer 3 to determine whether there are stones in the image data or not.

[0044] The embodiment of the present invention introduces a publicly available CT image dataset produced by Yildrim et al., which consists of 1799 CT images divided into two categories, including 790 kidney stone CT images and 1009 healthy people CT images. These CT images are from 433 subjects, including 268 stone-positive patients and 165 normal subjects. The kidney stone CT images in the image data were collected from patients with urinary stones admitted to a city hospital in Turkey. The age of stone-positive patients was between 18 and 80 years old, and the coronal CT slices were evaluated and marked.

[0045] Each image data in the collected dataset must be fully prepared through preprocessing before being input into the depthwise separable convolutional model. All images in the dataset are resized to 224×224×3 pixels.

[0046] After obtaining the preprocessed images, the dataset was divided into training and testing. 80% of the images were reserved for training purposes. The training set contained 1453 CT images (625 kidney stone images, 828 healthy images), while 20% of the images were reserved for testing purposes. The testing folder contained 346 CT images (165 kidney stone images, 181 healthy images).

[0047] The embodiment of the present invention uses depth-separable convolution similar to decomposed convolution, in which the standard convolution is decomposed into depth convolution and point convolution. This decomposition minimizes the size of the neural network and reduces the computational complexity. The embodiment of the present invention uses depth-separable convolution to replace the standard convolution in the prior art to improve the trade-off between latency and accuracy. The standard convolution is decomposed into depth convolution and 1×1 convolution, i.e., point convolution. Standard convolution performs filtering and combines inputs in one step, while depth convolution divides it into two layers. The depth convolution applies a filter on each input channel, and then a point convolution applies a 1×1 convolution to combine the outputs. This decomposition minimizes the size of the neural network and reduces the computational complexity.

[0048] CNN models require a large number of images to be effectively trained and show better performance on large data images. Due to the difficulty in collecting medical image data, deep learning models have overfitting problems due to the small number of images. The embodiments of the present invention use different enhancements of kidney CT images such as shearing, scaling, rotation, shifting and flipping during model training to improve generalization ability, increase data sets and reduce overfitting problems.

[0049] Specifically, the enhanced image data is input into the separable convolution layer to generate a multi-dimensional feature vector of size 7×7×1024; the multi-dimensional feature vector is flattened through the global pooling GAP layer to generate a one-dimensional feature map of size 1024; the one-dimensional vector has 64 features, dense layer 1 has 128 neurons, dense layer 2 has 64 neurons and Relu activation function, the kinetic energy of normalization layer 1 and normalization layer 2 are both 0.99, and the epsilon is 0.01; dense layer 3 has two neurons and a softmax activation function.

[0050] In order to better explain the stone identification and positioning method based on the deep separable convolution model provided by the embodiment of the present invention, the embodiment of the present invention visualizes the positioning information of the stone area on the CT image through Grad-CAM, which is helpful for the medical diagnosis process. This technology actually highlights the key areas of the CT image, which refers to the areas where the deep separable convolution model pays close attention when performing detection work. Once the predicted label is calculated using the deep separable convolution model, Grad-CAM is applied to any convolution layer, and finally the image data with stones is visualized using the deep separable convolution model through Grad-CAM to obtain the visualization result.

[0051] First get the neuron importance weights:

[0052]

[0053] Among them, Y c is the probability of classification c, and Z is the total number of pixels in the activation map.

[0054] The embodiment of the present invention performs weighted combination on the activation maps of the features, and then obtains a discriminant saliency map of class c of the target image through a ReLU filter.

[0055] Lc: in Represents the activation map of the k-th filter.

[0056] The StoneNet model based on the deep separable convolutional model used in the embodiment of the present invention is implemented using python and keras framework. After multiple configurations, the optimal model for detecting kidney stones was obtained. All experiments were performed using Google Colab Notebook equipped with a 12GB TeslaK80 GPU.

[0057] The embodiment of the present invention was trained and tested on different optimizers and batch sizes, trained for 20 epochs, and the initial learning rate was 0.001.

[0058] In the embodiment of the present invention, the kidney stone CT image dataset is detected and located by using the StoneNet model based on the deep separable convolution model, and the kidney stone recognition accuracy is 97.98%, and the model parameter size is 3.3M.

[0059]

[0060] From the analysis of the results in the above table, it can be seen that the method for identifying and locating kidney stones based on the deep separable convolutional model provided by the embodiment of the present invention provides significant capabilities for the identification of kidney stones, which is superior to the model method of Yildrim et al. for identifying and locating kidney stones using CT images.

[0061] The embodiment of the present invention further provides a stone identification and positioning device based on a deep separable convolution model, comprising:

[0062] An acquisition module, used for acquiring image data to be identified;

[0063] A processing module, used for preprocessing the image data to be identified;

[0064] A depthwise separable convolution module is used to perform stone recognition on the input pre-processed image data to obtain recognition results;

[0065] The visualization processing module is used to visualize the stone area on the image data when there is a stone in the image data, so as to obtain a visualization positioning result.

[0066] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed, the stone identification and positioning method based on the deep separable convolution model as described above is implemented.

[0067] An embodiment of the present invention also provides a stone positioning device based on a deep separable convolution model, comprising a memory and a processor; the memory is used to store computer programs and intermediate data during program processing, and the processor is used to execute the computer program to implement the stone identification and positioning method based on the deep separable convolution model as described above.

[0068] The embodiment of the present invention uses deep separable convolution to improve the trade-off between latency and accuracy, can effectively learn features from kidney stone CT images, and significantly reduce calculation parameters and computational complexity. It is highly efficient and computationally inexpensive, and can be successfully applied to remote and rural areas that lack medical facilities for kidney stone screening; it has a very high accuracy rate of 97.98% in identifying and locating kidney stones, greatly improving recognition accuracy.

[0069] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A stone identification and positioning method based on a deep separable convolutional model, characterized in that: The method comprises: Step 1, obtaining image data to be identified; Step 2, inputting the image data into a depthwise separable convolutional model to perform stone recognition, and obtaining a recognition result, wherein the recognition result includes whether a stone exists in the image data or whether a stone does not exist in the image data; Step 3, when there is a stone in the image data, visualizing the stone area on the image data to obtain a visual positioning result; The depth-wise separable convolution model includes a depth-wise separable convolution layer, a global pooling layer, a dropout layer 1, a dense layer 1, a normalization layer 1, a dropout layer 2, a dense layer 2, and a normalization layer 2 connected in sequence; The depthwise separable convolutional model also includes a dense layer 3 for outputting recognition results, and the input end of the dense layer 3 is connected to the output end of the normalization layer 2.

2. The stone identification and positioning method based on the deep separable convolution model according to claim 1 is characterized in that: The step 2 also includes: Preprocessing the image data; The preprocessed image data is input into the deep separable convolutional model for stone recognition.

3. The stone identification and positioning method based on the deep separable convolution model according to claim 1 is characterized in that: Before the depthwise separable convolutional model is trained, the image data used to train the depthwise separable convolutional model is enhanced.

4. The stone identification and positioning method based on the deep separable convolution model according to claim 1 is characterized in that: The kinetic energy of the normalized layer 1 and the normalized layer 2 are both 0.

99.

5. A stone identification and positioning device based on a deep separable convolutional model, characterized in that: include: An acquisition module, used for acquiring image data to be identified; A processing module, used for preprocessing the image data to be identified; A depthwise separable convolution module is used to perform stone recognition on the input pre-processed image data to obtain recognition results; A visualization processing module, used for visualizing the stone area on the image data to obtain a visualization positioning result when there is a stone in the image data; The depth-wise separable convolution model includes a depth-wise separable convolution layer, a global pooling layer, a dropout layer 1, a dense layer 1, a normalization layer 1, a dropout layer 2, a dense layer 2, and a normalization layer 2 connected in sequence; The depthwise separable convolutional model also includes a dense layer 3 for outputting recognition results, and the input end of the dense layer 3 is connected to the output end of the normalization layer 2.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, the stone identification and positioning method based on the depthwise separable convolutional model as described in any one of claims 1 to 4 is implemented.

7. A stone identification and positioning device based on a deep separable convolution model, characterized in that: It includes a memory and a processor; the memory is used to store computer programs and intermediate data during program processing, and the processor is used to execute the computer program to implement the stone identification and positioning method based on the deep separable convolution model as described in any one of claims 1 to 4.