AI identification method and system for intra-airway stent type

Abstract

According to the AI identification method and system for the type of the stent in the airway, the AI model is constructed, T-type silicone, silicone and metal stents are distinguished through a deep learning method, Y-type or non-Y-type stents are identified through the internal images of the stents, and therefore the purpose of identifying five different types of stents is finally achieved, and the AI identification method and system have the advantages of being independent of manual observation, high in identification accuracy and the like. The method can timely, rapidly and accurately identify the type of the support, and can overcome the problems that direct observation, iconography technology and fluorescence imaging technology depend on experience, and changes of the support cannot be dynamically observed in real time.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence-assisted medical technology, and particularly to an AI model for identifying the type of airway stent. Background Art

[0002] As a therapeutic device, stents are commonly used in the treatment of patients with airway stenosis. Currently, the highly innovative "robotic bronchoscope" requires the operator to operate the handle for examination and cannot yet achieve fully automated bronchoscope examination operations. To achieve full automation, in addition to the need for AI recognition of normal airway anatomical sites, it is also necessary to identify special airway sites. Therefore, it is necessary to identify the type of airway stent. Common stent types include metal stents, silicone stents, etc. From the perspective of shape, they are further divided into Y-shaped stents and non-Y-shaped stents. The techniques for identifying stent types mainly include the following:

[0003] 1. Direct bronchoscopic observation: By directly observing the morphology, position, and relationship with surrounding tissues of the stent through a bronchoscope to determine the stent type (such as metal stents, silicone stents, etc.). This technique has strong intuitiveness, can directly observe the morphology and position of the stent, and can evaluate in real time whether the stent has shifted or developed complications. However, this method has limited vision, is difficult to comprehensively observe the subtle changes between the stent and surrounding tissues, and depends on the operator's experience, which may lead to misjudgment or missed diagnosis.

[0004] 2. Imaging techniques: Obtain images of the stent through X-ray or CT scans, analyze its morphology, position, and relationship with surrounding tissues to determine the stent type. It can comprehensively evaluate the position and morphology of the stent, especially for complex airway lesions. However, this method has a relatively high risk of radiation exposure, especially when multiple scans are required, and it cannot dynamically observe the changes of the stent in real time.

[0005] 3. Fluorescence imaging techniques: By fluorescently labeling the stent or surrounding tissues to enhance the visibility of the stent and assist in identifying the stent type. This method can improve the contrast between the stent and surrounding tissues and facilitate identification. However, the selection and labeling process of the fluorescent marker are complex, and the equipment cost is high, so it is difficult to popularize.

[0006] With the development of artificial intelligence-assisted recognition technology, using deep learning algorithms to analyze bronchoscopic or imaging images to automatically identify the stent type and its position has gradually come into view. AI recognition based on images of different types of stents has become the basis of future intelligent medicine. Therefore, this application proposes a method using deep learning to construct an AI model to automatically identify the material and shape of airway stents to assist clinicians in precise diagnosis and treatment. Summary of the Invention

[0007] In view of the above problems, the present application proposes a method and system for constructing an AI model to identify the type of airway stent using deep learning methods, which can perform AI identification on common airway stents starting from the material, shape, and real-world bronchoscope images.

[0008] In a first aspect, a method for constructing an AI model to identify the type of airway stent using deep learning methods is characterized by including the following steps:

[0009] S1. Collect training data and construct an identification model;

[0010] S2. Train the identification model using the training data;

[0011] S3. Through the identification model, identify the stent material based on the material and color features through deep learning methods;

[0012] S4. Through the identification model, identify the stent shape based on the internal image of the stent;

[0013] Among them, the model is trained and constructed using the EfficientNet model.

[0014] Optionally, before the step S1, it further includes determining the principles for selecting training data, including the screening conditions for subjects and the principles for selecting images.

[0015] Optionally, before the step S1, it further includes: determining the identification strategy.

[0016] Optionally, before the step S2, it further includes an image preprocessing process.

[0017] Optionally, the preprocessing process includes: for each piece of data included in the training set, first process it using Gaussian filtering, then remove the black edges in the acquired image, retain the effective information part, then adjust the image to a unified appropriate size for subsequent model training, and finally perform standardization and normalization operations on the image.

[0018] Optionally, the EfficientNet model is derived by stacking and scaling preset convolutional blocks from a benchmark model according to a certain strategy.

[0019] Optionally, the mathematical representation of the strategy is shown in the following formula:

[0020] depth: d = α φ

[0021] width: w = β φ

[0022] resolution: r = γ φ

[0023] such that α·β 2 ·γ 2 ≈2

[0024] α≥1, β≥1, γ≥1

[0025] Wherein, α, β, and γ respectively represent the scaling bases of depth, width, and resolution.

[0026] On the other hand, the present application also provides a system for constructing an AI model to identify the type of airway stent using a deep learning method, which is characterized in that it includes:

[0027] A collection and modeling device for collecting training data and constructing an identification model;

[0028] A training device for training the identification model using the training data;

[0029] A material identification device for identifying the stent material through the identification model based on material and color features by means of a deep learning method

[0030] A stent shape identification device for identifying the stent shape based on the internal image of the stent through the identification model;

[0031] Wherein, the identification model is trained and constructed using the EfficientNet model.

[0032] On the other hand, the present application provides an electronic device, including: one or more processors; a storage device on which a computer program or instruction is stored, and when the computer program or instruction is executed by the processor, the method for constructing an AI model to identify the type of airway stent using a deep learning method as described above is implemented.

[0033] On the other hand, the present application provides a computer-readable storage medium on which a computer program or instruction is stored and coupled to a processor, wherein when the computer program or instruction is executed by the processor, the method for constructing an AI model to identify the type of airway stent using a deep learning method as described above is implemented.

[0034] Through the above solutions, the identification of the type of airway stent based on the AI model is achieved. By constructing an AI model, three types of T-shaped silicone, silicone, and metal stents are distinguished through a deep learning method (mainly based on material and color features), and then the internal image of the stent (mainly through internal shape features) is used to identify Y-shaped or non-Y-shaped stents, so as to finally achieve the purpose of identifying 5 different types of stents. It has the effects of not relying on manual observation, being able to identify the stent type in a timely, fast, and accurate manner, and being able to overcome problems such as relying on experience in direct observation, imaging technology, and fluorescence imaging technology and being unable to observe the changes of the stent in real-time and dynamically. Brief Description of the Drawings

[0035] Figure 1 For the upper end of the stent;

[0036] Figure 2 For the Y-shaped silicone;

[0037] Figure 3 For the Y-shaped metal;

[0038] Figure 4 For the stent type recognition strategy;

[0039] Figure 5 For the stent type recognition process;

[0040] Figure 6 For the confusion matrix of recognizing silicone stents, T-shaped silicone stents, and metal stents through the upper end image of the stent;

[0041] Figure 7 For the confusion matrix of recognizing non-Y-shaped silicone stents and Y-shaped silicone stents through the internal image of the stent;

[0042] Figure 8 For the confusion matrix of recognizing non-Y-shaped metal stents and Y-shaped metal stents through the internal image of the stent. Detailed implementation method

[0043] The following describes the specific implementation process of this solution.

[0044] First: Data overview

[0045] To establish a classification model for tracheal stent material type recognition, we select data according to the following principles:

[0046] The selection principle for subjects is: 1) Patient age > 14 years old, 2) Undergoing airway stent treatment, etc.; 3) Containing complete upper end and internal images of the stent.

[0047] The selection principle for images is: 1) Containing the upper end of the stent for differentiating stent materials ( Figure 1 For the example of the upper end of the stent), 2) Y-shaped stents must contain a specific shape for differentiating Y-shaped and non-Y-shaped stents ( Figure 2 , Figure 3 For the example of the inside of the stent), 3) Without secretions, 4) Only select one of the multiple bronchoscopic examination images of the same part of the same patient.

[0048] Based on the above image selection principle, 1256 upper end images of the stent are collected.

[0049] Second: Data preprocessing

[0050] Before using the airway anatomical part image to train the relevant model, the image needs to be preprocessed. The present invention uses a strategy based on image enhancement to preprocess the image. The specific steps are: for each data included in the training set, first use Gaussian filtering to process it, so as to eliminate the Gaussian noise of the image, wherein the Gaussian filter is a linear filter that can effectively suppress noise and smooth the image. To perform Gaussian filtering, it is necessary to first determine the Gaussian kernel, and then perform a convolution operation based on the Gaussian kernel, that is, to perform a weighted summation of the pixel values of the pixel points in the rectangular window. For a window of (2n+1)×(2n+1), the weight calculation formula is as follows:

[0051]

[0052] Correspondingly, the weight of the entire convolution kernel can be expressed as:

[0053]

[0054] Then the final Gaussian kernel can be expressed as:

[0055]

[0056] Subsequently, the original image is convolved using the Gaussian kernel as above to obtain a Gaussian filtered image.

[0057] After Gaussian filtering, the black edges in the captured image are removed, the effective information is retained, and the image is then adjusted to a uniform and appropriate size for subsequent model training. Finally, the image is standardized and normalized.

[0058] Third: Model selection

[0059] Next, model selection is carried out. The present invention selects EfficientNet as the benchmark model for training. EfficientNet is a convolutional neural network designed based on the standardized convolutional network expansion method of compound model scaling. It balances the three dimensions of input resolution, depth and width to achieve network optimization in efficiency and accuracy, thereby achieving a higher accuracy and fully saving computing resources in the process of training the model.

[0060] Among them, the composite model scaling method based on the EfficientNet we chose is a strategy that derives multiple different models from a baseline model according to a certain strategy. Among them, there are three common model scaling strategies: scaling network depth, scaling model width, and scaling input size. Among them, changing the model depth is the most common model scaling method, because as the network deepens, the model's representation ability will also become stronger, but it will also cause a certain degree of gradient disappearance and model degradation problems; changing the model width is another common means, by adding more convolution kernels, so that the model can extract different types of features; and scaling the input size can optimize the performance of the model according to actual conditions. The composite model scaling method is a model adjustment strategy that combines the above three methods. Because the network input size is used as the benchmark, as the model input becomes larger, the model needs a deeper depth to obtain a larger receptive field. At the same time, a larger size means that the image has more information, which means that the network needs a larger width to carry more information.

[0061] The above strategy can be expressed in mathematical form as follows:

[0062] depth:d=α φ

[0063] width:w=β φ

[0064] resolution:r=γ φ

[0065] stα·β 2 γ 2 ≈2

[0066] α≥1,β≥1,γ≥1

[0067] Among them, α, β, and γ represent the scaling bases of depth, width, and resolution respectively. We take φ=1 for parameter search on the benchmark model. Finally, for EfficientNet-B0, the scaling parameters are as follows: α=1.2, β=1.1, γ=1.15. According to the above strategy, the preset convolution blocks are stacked and scaled to form the final EfficientNet.

[0068] Fourth: Model Training

[0069] The overall identification strategy is as follows Figure 4 As shown, (1) the general type of the stent is first identified based on the material of the upper end of the stent, and (2) whether the stent is Y-shaped is distinguished based on the shape inside the stent, thereby achieving the identification of a total of 5 main stent types. Figure 5 For the specific identification process.

[0070] In the model training stage, two models need to be trained, which are used to identify the stent material and shape respectively. Both models are trained based on the EfficientNet model. For the stent material classification model at the upper end of the stent, the input of the model is the image of the upper end of the stent without preliminary classification, and the output is the preliminary judgment of the stent material (metal, silicone, T-shaped silicone stent); for the model to distinguish whether the stent is a Y-shaped stent, the input of the model is the remaining stent pictures except the T-shaped silicone stent, and the output is the judgment result of whether the stent is a Y-shaped stent.

[0071] Based on the above ideas, 1256 images of the upper end of the stent were collected, and three types of T-shaped silicone, silicone, and metal stents were distinguished by deep learning methods (mainly based on material and color characteristics); then the internal image of the stent (mainly through internal shape characteristics) was used to identify Y-shaped or non-Y-shaped stents, so as to finally achieve the purpose of identifying 5 different types of stents.

[0072] According to the above principles for model training, using the above training set as training data, input it into the EfficientNet model. The model is trained by extracting features, and according to the extracted features, the input image data is classified. The final stent material type is obtained by combining the results obtained from the two models. After being tested by the test set, the accuracy index of the trained EfficientNet model can reach 98.5%.

[0073] Training results:

[0074] After being tested by the test set, the trained EfficientNet model has the following various indicators:

[0075] Accuracy: 95.5% (identifying Y silicone or non-Y silicone);

[0076] Accuracy: 100% (identifying Y metal or non-Y metal).

[0077] It can be seen that through our method, the overall accuracy rate of the AI model constructed by deep learning using the image of the upper end of the stent is more than 95% ( Figure 6 ); through deep learning on the internal image of the stent, the accuracy rate of the AI model in identifying Y-shaped silicone stents is more than 95% ( Figure 7 ); and the accuracy rate of identifying Y-shaped metal-covered stents is more than 95% ( Figure 8 ).

[0078] This embodiment proposes an identification model of a tracheal stent based on EfficientNet, and based on this model, an implementation scheme for airway stent identification is proposed, providing a further theoretical basis for future fully automated bronchoscopes. Moreover, the scheme we proposed balances the three dimensions of input resolution, depth, and width to a certain extent, reduces the computing power requirements for deploying related systems, thereby reducing costs and improving the corresponding work efficiency. At the same time, the present invention also proposes an electronic device, including: one or more processors; a storage device on which computer programs or instructions are stored, and when the computer programs or instructions are executed by the processor, the method for constructing an AI model to identify the type of airway stent by using the deep learning method is implemented. On the other hand, the present invention also provides a computer-readable storage medium on which computer programs or instructions are stored and coupled to the processor, wherein when the computer programs or instructions are executed by the processor, the method for constructing an AI model to identify the type of airway stent by using the deep learning method is implemented.

[0079] The technical features of the above embodiments can be combined arbitrarily. For the sake of simplicity of description, this specification does not list them exhaustively. However, as long as the combination of these technical features does not conflict, it should be considered as within the scope described in this specification.

[0080] The protection scope of the present invention is determined by the appended claims. Any improvement, variation, or equivalent replacement based on the principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for constructing an AI model to identify airway stent types using deep learning methods, characterized in that, Including the following steps: S1. Collect training data and construct an identification model. S2. Use the training data to train the identification model. S3. Through the identification model, identify the stent material based on material and color features by means of deep learning. S4. Through the identification model, identify the stent shape based on the internal image of the stent; wherein, the identification model is trained and constructed using the EfficientNet model.

2. The method for identifying the type of airway stent according to claim 1, characterized in that, Before the step S1, there is also a step of determining the principle for selecting training data, including the screening conditions for subjects and the principle for selecting images.

3. The method for identifying the type of airway stent according to claim 1, wherein Before the step S1, there is also: determining an identification strategy.

4. The method for identifying the type of airway stent according to claim 1, wherein Before the step S2, there is also an image preprocessing process.

5. The method for identifying the type of airway stent according to claim 5, wherein The image preprocessing process includes: for each piece of data included in the training set, first process it using Gaussian filtering, then remove the black edges in the acquired image, retain the effective information part, then adjust the image to a unified appropriate size for subsequent model training, and finally perform standardization and normalization operations on the image.

6. The method for identifying the type of airway stent according to claim 1, wherein The EfficientNet model is derived by stacking and scaling preset convolutional blocks from a benchmark model according to a certain strategy.

7. The method for identifying the type of airway stent according to claim 5, wherein The mathematical representation of the strategy is as shown in the following formula: depth:d=α φ width: w = β φ resolution:r=γ φ s.t.α·β 2 ·γ 2 ≈2 α≥1, β≥1, γ≥1 Wherein, α, β, and γ respectively represent the scaling bases for depth, width, and resolution.

8. A system for constructing an AI model to identify airway stent types using deep learning methods, characterized in that, Including: A collection and modeling device for collecting training data and constructing an identification model; A training device for training the identification model using the training data; A material identification device for identifying the stent material based on material and color features by means of deep learning through the identification model; A stent shape identification device for identifying the stent shape based on the internal image of the stent through the identification model; Wherein, the identification model is trained and constructed using the EfficientNet model.

9. An electronic device, comprising: One or more processors; A storage device storing a computer program or instruction, which when executed by the processor implements the above method for constructing an AI model to identify airway stent types using deep learning.

10. A computer-readable storage medium storing a computer program or instruction and coupled to a processor, wherein the computer program or instruction when executed by the processor implements the above method for constructing an AI model to identify airway stent types using deep learning.

Images