Deep learning-based bronchoscopic image analysis methods, systems, and storage media

Abstract

This application relates to the field of image recognition technology, and in particular to a method, system, and storage medium for bronchoscopic image analysis based on deep learning. The method includes: S1, examining and processing a bronchoscopic image to determine several feature image elements and recording the coordinates of these feature image elements; S2, obtaining a tracheal portion image from the bronchoscopic image based on the coordinates of the feature image elements; S3, using a trained deep learning model to analyze and process the tracheal portion image, obtaining analysis result data for each image element in the tracheal portion image, finding several reference images related to the tracheal portion image from all reference images recorded by the recording module, and further determining several final reference images from all the found reference images. This application can provide final reference images related to the tracheal portion image.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and in particular to a method, system and storage medium for bronchoscopic image analysis based on deep learning. Background Technology

[0002] Currently, the use of deep learning technology for automatic analysis of bronchoscopic images has become increasingly common, offering advantages such as high processing efficiency and good objectivity.

[0003] Similar prior art includes Chinese patent application CN118154602A, which discloses an image analysis method and system based on a colorectal polyp CT image dataset. This includes: obtaining the inner wall length of polyps in colorectal CT images containing polyps through image stripping; optimizing the inner wall length of the polyps to obtain a standard polyp length; using the standard polyp length to identify polyps in the colorectal polyp CT images; and obtaining the analysis results of the colorectal polyp CT images based on the maximum polyp length. Furthermore, similar prior art includes Chinese patent application CN119963493A, which discloses a deep learning-based CT image analysis and tumor prediction method and system. The method includes: collecting CT image data and annotation information from tumor patients to obtain a high-quality CT image dataset; combining neural structure search technology and composite model expansion methods to establish a second-generation efficient convolutional neural network tumor classification model; and adding a coordinate attention module and a hard smoothing activation function to the model to obtain an improved second-generation efficient convolutional neural network tumor classification model. However, neither of the above two patent applications can provide images similar to the collected images to assist in the analysis and processing. Summary of the Invention

[0004] This application extracts images of the trachea portion from bronchoscopic images, identifies several reference images related to the trachea portion from all recorded reference images, and further determines several final reference images from all identified reference images. This application aims to provide final reference images similar to the trachea portion images to assist in analysis and processing.

[0005] This application provides a deep learning-based method for bronchoscopic image analysis, including the following steps:

[0006] S1. The preparation module acquires a bronchoscopic image, and the preparation module examines and processes the bronchoscopic image to determine several feature image elements. The preparation module also records the coordinates of several feature image elements.

[0007] S2. The preparation module obtains a tracheal portion image from the bronchoscopy image based on the coordinates of several feature image elements, and sends the tracheal portion image to the analysis module.

[0008] S3. The analysis module uses a trained deep learning model to analyze and process the tracheal part image, obtains the analysis result data of each image element in the tracheal part image, and the analysis module finds several reference images related to the tracheal part image from all the reference images recorded by the recording module. The analysis module also determines several final reference images from all the reference images found.

[0009] As a preferred technical solution of this application, the preparation module examines and processes the bronchoscopic image to determine several feature image elements, including the following steps:

[0010] S11. The preparation module calculates the feature value of each image element in the bronchoscopy image;

[0011] S12. The preparation module sets an image element as a target image element, determines whether the feature value of the target image element is less than the first reference value, and if so, determines that the target image element belongs to the first category; otherwise, continues to the next step.

[0012] S13. The preparation module determines whether the feature value of the target image element is greater than the second reference value. If yes, the target image element is determined to belong to the second category; otherwise, the target image element is determined to belong to the third category.

[0013] S14. The preparation module calculates the feature value threshold corresponding to the target image element according to the category to which the target image element belongs, calculates the feature value difference between the target image element and each surrounding image element, calculates the average of all feature value differences, and determines whether the average is greater than the feature value threshold corresponding to the target image element. If so, the target image element is treated as a feature image element.

[0014] S15. The preparation module determines whether there are any image elements that have not been set as target image elements. If yes, it jumps to S12; otherwise, it ends all steps.

[0015] As a preferred technical solution of this application, the preparation module calculates the feature value threshold corresponding to the target image element according to the category to which the target image element belongs, including: when the target image element belongs to the first category, calculating the sum of the product of the preset first feature value threshold and the preset value and the feature value of the target image element; when the target image element belongs to the second category, first calculating the difference between the feature value of the target image element and the second reference value, and then calculating the difference between the product of the preset second feature value threshold and the preset value and the difference.

[0016] As a preferred technical solution of this application, the analysis module uses a trained deep learning model to analyze and process the trachea portion image, and obtains the analysis result data of each image element in the trachea portion image, including the following steps:

[0017] S311. For each image element, the analysis module obtains an image patch of a preset size with the image element as the core from the trachea part image, and inputs the obtained image patch into the trained deep learning model.

[0018] S312. Regarding the input image patch, the trained deep learning model outputs analysis result data, which consists of the probability values ​​of the core image elements in the image patch corresponding to each category.

[0019] As a preferred technical solution of this application, after the analysis module obtains the analysis result data of each image element in the tracheal part image, it further includes: for each image element, the analysis module takes the category corresponding to the maximum probability value in the corresponding analysis result data as the final category to which the image element belongs, and the analysis module uses several image elements with the same final category to form a feature image range in the tracheal part image.

[0020] As a preferred technical solution of this application, for each comparison image, the recording module also records the analysis result data of each image element in the comparison image, the feature image range in the comparison image, the coordinates of the feature image range, and the interpretation information of the feature image range.

[0021] As a preferred technical solution of this application, the analysis module finds several reference images related to the trachea portion image from all the reference images recorded by the recording module, including the following steps:

[0022] S321. The analysis module divides the tracheal image into several image ranges and calculates the average data of the analysis results of all image elements in each image range.

[0023] S322. For each control image, the analysis module divides the control image into several image ranges in the same way and calculates the average data of the analysis results of all image elements in each image range.

[0024] S323. For each control image, the analysis module determines several image ranges in the control image that correspond to each image range in the tracheal part image, calculates the similarity between two corresponding image ranges, and sums up all the similarity to obtain the overall similarity.

[0025] S324. The analysis module takes the corresponding overall similarity image that is greater than the preset overall similarity threshold as the reference image related to the tracheal part image.

[0026] As a preferred technical solution of this application, the analysis module further determines several final comparison images from all the comparison images found, including the following steps:

[0027] S331. The analysis module determines an image range of interest within the entire feature image range of the tracheal portion image, and also determines the coordinates of the image range of interest;

[0028] S332. For each found reference image, the analysis module determines a feature image range in the reference image whose coordinates are close to the coordinates of the range of the image of interest, and calculates the difference between the determined feature image range and the range of the image of interest.

[0029] S333, The analysis module uses the control image corresponding to the smallest difference as the final control image.

[0030] This application also provides a deep learning-based bronchoscopy image analysis system, including the following modules:

[0031] The recording module is used to record several comparison images;

[0032] The preparation module is used to acquire bronchoscopic images, examine and process the bronchoscopic images, determine several feature image elements, record the coordinates of several feature image elements, and use the coordinates of several feature image elements to acquire tracheal part images from the bronchoscopic images and send the tracheal part images to the analysis module.

[0033] The analysis module is used to analyze and process the tracheal images using a trained deep learning model, obtain the analysis results data of each image element in the tracheal images, find several reference images related to the tracheal images from all reference images recorded by the recording module, and further determine several final reference images from all the reference images found.

[0034] This application also provides a storage medium storing program instructions, wherein the program instructions, when executed, control the device where the storage medium is located to perform any of the methods described above.

[0035] Compared with the prior art, the beneficial effects of this application are at least as follows:

[0036] In the technical solution provided in this application, firstly, the preparation module acquires a bronchoscopic image and performs examination processing on the bronchoscopic image to determine several feature image elements. The preparation module also records the coordinates of these feature image elements. Secondly, based on the coordinates of the feature image elements, the preparation module extracts a tracheal portion image from the bronchoscopic image and sends the tracheal portion image to the analysis module. Finally, the analysis module uses a trained deep learning model to analyze the tracheal portion image, obtaining the analysis result data for each image element in the tracheal portion image. Furthermore, the analysis module searches for several reference images related to the tracheal portion image from all reference images recorded by the recording module, and further determines several final reference images from these selected reference images. Through this application, not only can the tracheal portion image be accurately extracted from the bronchoscopic image, but also final reference images similar to the tracheal portion image can be determined for the analysis of lesions, improving the reliability of the analysis results. Attached Figure Description

[0037] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a flowchart of the deep learning-based bronchoscopic image analysis method in the embodiments of this application;

[0039] Figure 2 This is a flowchart of a method for finding several comparison images in an embodiment of this application;

[0040] Figure 3 This is a flowchart illustrating the method for determining several final comparison images in an embodiment of this application;

[0041] Figure 4 This is a schematic diagram of a deep learning-based bronchoscopy image analysis system in an embodiment of this application. Detailed Implementation

[0042] This application provides a deep learning-based method, system, and storage medium for bronchoscopic image analysis. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0043] For ease of understanding, the specific process of the embodiments of this application is described below. Please refer to [link / reference]. Figure 1 The deep learning-based bronchoscopic image analysis method in this application includes the following main steps:

[0044] S1. The preparation module acquires bronchoscopic images and performs inspection and processing on the bronchoscopic images to determine several feature image elements. The preparation module also records the coordinates of several feature image elements.

[0045] S2. The preparation module obtains a tracheal portion image from the bronchoscopy image based on the coordinates of several feature image elements, and sends the tracheal portion image to the analysis module.

[0046] S3. The analysis module uses a trained deep learning model to analyze and process the tracheal image, obtains the analysis results data of each image element in the tracheal image, and finds several reference images related to the tracheal image from all the reference images recorded by the recording module. The analysis module also determines several final reference images from all the reference images found.

[0047] Specifically, when analyzing lesions solely based on acquired bronchoscopic images, the analysis results may lack reliability. To address this issue, in step S1, the preparation module acquires a bronchoscopic image, which includes not only the trachea but also the background. The preparation module then examines and processes the bronchoscopic image, identifying and recording the coordinates of several characteristic image elements. It should be noted that these characteristic image elements refer to those image elements on the edges of the trachea. In step S2, based on the coordinates of these characteristic image elements, the preparation module extracts a tracheal image from the original bronchoscopic image and sends this color image to the analysis module. In S3, the analysis module uses a trained deep learning model to analyze and process images of the trachea region, obtaining analysis results for each image element. The training process of the deep learning model can be implemented using existing technology and will only be briefly described here. The process includes: acquiring a large number of training images and their corresponding correct labels (the correct labels being the correct categories to which the core image elements in the training images belong); sequentially inputting the training images into the deep learning model to obtain the probability of each category belonging to the core image elements in the training images; taking the category with the highest probability as the final category; and adjusting the internal weights by comparing the difference between the final category and the actual label to minimize the error. The analysis module then finds several control images related to the trachea region from all the control images recorded by the recording module. It further determines several final control images from these found control images, which can then be used to assist in the analysis and processing, specifically the analysis and processing of lesions.

[0048] Furthermore, the preparation module examines and processes the bronchoscopic images to identify several characteristic image elements, including the following steps:

[0049] S11. The preparation module calculates the feature value of each image element in the bronchoscopy image;

[0050] S12. The preparation module sets an image element as the target image element, determines whether the feature value of the target image element is less than the first reference value. If yes, it determines that the target image element belongs to the first category. If no, it continues to the next step.

[0051] S13. The preparation module determines whether the feature value of the target image element is greater than the second reference value. If yes, the target image element is determined to belong to the second category; otherwise, the target image element is determined to belong to the third category.

[0052] S14. The preparation module calculates the feature value threshold corresponding to the target image element according to the category to which the target image element belongs, calculates the feature value difference between the target image element and each surrounding image element, calculates the average of all feature value differences, and determines whether the average is greater than the feature value threshold corresponding to the target image element. If so, the target image element is treated as a feature image element.

[0053] S15. The preparation module determines whether there are any image elements that have not been set as target image elements. If yes, it jumps to S12; otherwise, it ends all steps.

[0054] Specifically, the process of determining several feature image elements is described in S11. The preparation module calculates the feature value of each image element in the bronchoscopy image, specifically using the grayscale formula Gray = (R + G + B) / 3, where Gray is the feature value, R is the R value of the image element, G is the G value of the image element, and B is the B value of the image element. At this point, the color bronchoscopy image has been converted into a grayscale image to facilitate the determination of feature image elements. In S12, the preparation module sets one image element as the target image element. Specifically, it sets one image element in the grayscale image as the target image element and determines whether the feature value of the target image element is less than a first reference value. If it is less, the target image element is determined to belong to the first category; otherwise, the process continues to S13. In S13, the preparation module determines whether the feature value of the target image element is greater than the second reference value. If it is, the target image element is classified into the second category; otherwise, it is classified into the third category. It's important to note that the second reference value is greater than the first reference value. This classification of the target image element takes into account that within the range of high and low brightness in a grayscale image, the difference between the feature value of the characteristic image element and the feature values ​​of surrounding image elements is relatively small. In S14, the preparation module calculates the feature value threshold corresponding to the target image element based on its category, calculates the feature value difference between the target image element and each surrounding image element (the feature value difference can be greater than 0), calculates the average of all feature value differences, and determines whether the average is greater than the feature value threshold corresponding to the target image element. If it is, the target image element is treated as a characteristic image element; otherwise, it is not. In S15, the preparation module determines whether there are image elements in the grayscale image that have not been set as target image elements. If they exist, it jumps to S12; otherwise, it ends all steps. The coordinates of the feature image elements mentioned above actually refer to the coordinates of the feature image elements in the coordinate system of the grayscale image.

[0055] Furthermore, the preparation module calculates the feature value threshold corresponding to the target image element according to the category to which the target image element belongs, including: when the target image element belongs to the first category, calculating the sum of the product of the preset first feature value threshold and the preset value and the feature value of the target image element; when the target image element belongs to the second category, first calculating the difference between the feature value of the target image element and the second reference value, and then calculating the difference between the product of the preset second feature value threshold and the preset value and the difference.

[0056] Specifically, this section describes how to calculate the feature value threshold corresponding to a target image element based on its category. Before proceeding, it's important to note that in this embodiment, the feature value threshold changes with the feature value of the target image element. The process of this change is explained below. As mentioned above, the minimum feature value is 0, and the maximum is 255. When the feature value is 0, the corresponding feature value threshold is a preset first feature value threshold. When the feature value is greater than 0 but less than a first reference value (i.e., the target image element belongs to the first category), the corresponding feature value threshold increases linearly until the feature value reaches the first reference value, at which point the feature value threshold becomes a preset second feature value threshold. When the feature value is greater than or equal to the first reference value but less than or equal to the second reference value (i.e., the target image element belongs to the third category), the corresponding feature value threshold remains unchanged at the preset second feature value threshold. When the feature value is greater than the second reference value (i.e., the target image element belongs to the second category), the corresponding feature value threshold begins to decrease linearly. This is done to accurately determine the feature image element even in grayscale images with low or large brightness ranges. The first and second feature value thresholds are set according to the actual application scenario.

[0057] Therefore, when the target image element belongs to the first category, the sum of the product of the preset first feature value threshold and the preset value and the feature value of the target image element is calculated. The preset value is the result of dividing the difference between the second feature value threshold and the first feature value threshold by the first reference value. When the target image element belongs to the second category, the difference between the feature value of the target image element and the second reference value is calculated first, and then the difference between the product of the preset second feature value threshold and the preset value and the difference is calculated. When the target image element belongs to the third category, the second feature value threshold is used directly.

[0058] Furthermore, the analysis module uses a trained deep learning model to analyze and process the trachea portion of the image, obtaining the analysis results data for each image element in the trachea portion image, including the following steps:

[0059] S311. For each image element, the analysis module obtains an image patch of a preset size with the image element as the core from the trachea part image, and inputs the obtained image patch into the trained deep learning model.

[0060] S312. Regarding the input image patch, the trained deep learning model outputs analysis result data, which consists of the probability values ​​of the core image elements in the image patch corresponding to each category.

[0061] Specifically, the document describes how to obtain the analysis results data for each image element in the tracheal portion of the image. In S311, for each image element, the analysis module obtains an image patch of a preset size centered on the image element from the tracheal portion of the image; that is, the image element is located at the center of the image patch. The obtained image patch is then input into the trained deep learning model. In S312, for the input image patch, the trained deep learning model outputs analysis results data. The analysis results data consists of the probability values ​​of the core image elements in the input image patch corresponding to each category, and each category can be a different lesion type.

[0062] Furthermore, after the analysis module obtains the analysis result data of each image element in the tracheal part image, it also includes: for each image element, the analysis module takes the category corresponding to the maximum probability value in the corresponding analysis result data as the final category to which the image element belongs, and the analysis module uses several image elements with the same final category to form a feature image range in the tracheal part image.

[0063] Specifically, the analysis module performs processing after acquiring the analysis result data for each image element in the trachea image. For each image element, the analysis module takes the category corresponding to the highest probability value in the corresponding analysis result data as the final category to which the image element belongs. The corresponding analysis result data refers to the analysis result data obtained by inputting an image patch of a preset size with the image element as the core into a trained deep learning model. In the trachea image, several image elements with the same final category are used to form a feature image range. The number of feature image ranges may be more than one. For ease of understanding, for example, there are multiple image elements with the same category in the trachea image, and the coordinates of these image elements are close to each other in the image coordinate system. If the image range formed by these image elements is large enough, the feature image range can be obtained.

[0064] Furthermore, for each control image, the recording module also records the analysis results data for each image element in the control image, the feature image range in the control image, the coordinates of the feature image range, and the interpretation information of the feature image range.

[0065] Specifically, the recording module records the following: the control image and the tracheal image are the same size and shape, and the shape can both be circular. In addition to recording the control image, the recording module also records the analysis result data of each image element in the control image. The analysis result data here has the same meaning as the analysis result data corresponding to the tracheal image. It also records the feature image range in the control image, the coordinates of the feature image range, and the interpretation information of the feature image range. The feature image range here has the same meaning as the feature image range in the tracheal image. The coordinates of the feature image range refer to the coordinates of the image element located at the center of the feature image range in the image coordinate system. The interpretation information of the feature image range can be about the lesion range corresponding to the feature image range, such as "the mucosal surface shows changes such as unevenness, erosion, and ulceration". The interpretation information can be manually set to help analyze and process the lesion conditions mentioned above.

[0066] Furthermore, the analysis module identifies several control images related to the tracheal portion from all the control images recorded by the recording module, including the following steps:

[0067] S321. The analysis module divides the tracheal image into several image ranges and calculates the average data of the analysis results of all image elements in each image range.

[0068] S322. For each control image, the analysis module divides the control image into several image ranges using the same method, and calculates the average data of the analysis results of all image elements in each image range;

[0069] S323. For each control image, the analysis module determines several image ranges in the control image that correspond to each image range in the tracheal part image, calculates the similarity between two corresponding image ranges, and sums up all the similarity to obtain the overall similarity.

[0070] S324. The analysis module uses the corresponding overall similarity images that are greater than the preset overall similarity threshold as the reference images related to the tracheal part images.

[0071] Specifically, please refer to Figure 2This document describes how to identify several reference images related to the tracheal portion image from all reference images recorded by the recording module. In S321, the analysis module divides the tracheal portion image into several image ranges and calculates the average data of all image elements within each range. The analysis results can be in vector form. In S322, for each reference image, the analysis module divides the image into several image ranges using the same method and calculates the average data of all image elements within each range. In S323, for each reference image, the analysis module identifies several image ranges corresponding to each image range in the tracheal portion image and calculates the similarity between two corresponding ranges. "Corresponding" means that the coordinates of the two ranges are the same in their respective image coordinate systems. Specifically, this can be achieved by calculating the similarity between the average data of two corresponding ranges and then summing all the similarity scores to obtain the overall similarity score. In S324, the analysis module takes the corresponding overall similarity image that is greater than the preset overall similarity threshold as the reference image related to the tracheal part image. The reference image related to the tracheal part image is the reference image that is more similar to the tracheal part image.

[0072] Furthermore, the analysis module continues to identify several final control images from all the identified control images, including the following steps:

[0073] S331. The analysis module determines an image range of interest within the entire feature image range of the tracheal portion image, and also determines the coordinates of the image range of interest.

[0074] S332. For each found reference image, the analysis module determines a feature image range in the reference image whose coordinates are close to the coordinates of the range of the image of interest, and calculates the difference between the determined feature image range and the range of the image of interest.

[0075] S333, The analysis module uses the control image with the smallest difference as the final control image.

[0076] Specifically, please refer to Figure 3This document describes how to further determine several final control images from all the retrieved control images. In step S331, the analysis module identifies a region of interest from all feature image ranges in the tracheal portion image. Specifically, this can be done manually by the analysis module, selecting a feature image range as the region of interest. The lesions within the region of interest are typically more severe. The coordinates of the region of interest are then determined; the coordinates of the image element located at the center of the region of interest can be used as its coordinates in the image coordinate system. In step S332, for each retrieved control image, the analysis module identifies a feature image range whose coordinates are close to those of the region of interest. This is done by calculating the distance between the coordinates of the region of interest and the coordinates of each feature image range in the control image, determining the region of interest with the smallest distance. Then, the difference between the determined feature image range and the region of interest is calculated. This is done by calculating the average R-value, average G-value, and average B-value of all image elements within the determined feature image range, and calculating the R-value of all image elements within the region of interest. The average values ​​of the R, G, and B values ​​are calculated. The difference between the average R value of the determined feature image range and the average R value of the image range of interest is calculated, with the difference being greater than 0. Similarly, the difference between the average G value of the determined feature image range and the average G value of the image range of interest is calculated, with the difference being greater than 0. The difference between the average B value of the determined feature image range and the average B value of the image range of interest is calculated, with the difference being greater than 0. Weights are assigned to each difference, with the sum of different weights being 1. A weighted method is used to calculate the degree of difference between the determined feature image range and the image range of interest. In S333, the analysis module uses the control image corresponding to the smallest degree of difference as the final control image. Given that an image range of interest has already been determined in the tracheal portion image, the analysis of the lesion conditions mentioned above can be performed by referring to the interpretation information of the corresponding feature image range in the final control image, improving the reliability of the analysis results.

[0077] According to another aspect of the embodiments of this application, reference is made to... Figure 4 As shown, this application also provides a deep learning-based bronchoscopy image analysis system, including a recording module, a preparation module, and an analysis module, to implement the deep learning-based bronchoscopy image analysis method described above. The functions of each module are as follows:

[0078] The recording module is used to record several comparison images;

[0079] The preparation module is used to acquire bronchoscopic images, examine and process the bronchoscopic images, determine several feature image elements, record the coordinates of several feature image elements, and use the coordinates of several feature image elements to acquire tracheal part images from the bronchoscopic images and send the tracheal part images to the analysis module.

[0080] The analysis module is used to analyze and process the tracheal images using a trained deep learning model, obtain the analysis results data of each image element in the tracheal images, find several reference images related to the tracheal images from all reference images recorded by the recording module, and further determine several final reference images from all the reference images found.

[0081] According to another aspect of the embodiments of this application, a storage medium is also provided, which stores program instructions, wherein the program instructions, when executed, control the device where the storage medium is located to perform any of the methods described above.

[0082] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0083] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0084] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A bronchoscope image analysis method based on deep learning, characterized by, The method comprises the following steps: S1, a preparation module acquires a bronchoscope image, and the preparation module performs inspection processing on the bronchoscope image to determine a plurality of feature image elements, and the preparation module further records coordinates of the plurality of feature image elements; The preparation module performs inspection processing on the bronchoscope image to determine a plurality of feature image elements, and the preparation module further records coordinates of the plurality of feature image elements; S11, the preparation module calculates a feature value of each image element in the bronchoscope image; S12, the preparation module sets an image element as a target image element, judges whether the feature value of the target image element is less than a first reference value, if yes, determines that the target image element belongs to a first category, and if no, continues to the next step; S13, the preparation module judges whether the feature value of the target image element is greater than a second reference value, if yes, determines that the target image element belongs to a second category, and if no, determines that the target image element belongs to a third category; S14, the preparation module calculates a feature value threshold corresponding to the target image element according to the category to which the target image element belongs, calculates a feature value difference between the target image element and each image element around the target image element, calculates a mean value of all feature value differences, and judges whether the mean value is greater than the feature value threshold corresponding to the target image element, if yes, takes the target image element as a feature image element; S15, the preparation module judges whether there is an image element that is not set as the target image element, if yes, jumps to S12, and if no, ends all steps; S2, the preparation module acquires a bronchus part image from the bronchoscope image based on the coordinates of the plurality of feature image elements, and sends the bronchus part image to an analysis module; S3, the analysis module performs analysis processing on the bronchus part image using a trained deep learning model, acquires analysis result data of each image element in the bronchus part image, and the analysis module finds a plurality of control images related to the bronchus part image from all control images recorded by a recording module, and the analysis module further determines a plurality of final control images in the plurality of found control images, and uses the final control images to perform analysis processing on a lesion condition.

2. The method of claim 1, wherein, The preparation module calculates a feature value threshold corresponding to the target image element according to the category to which the target image element belongs, and comprises: in the case that the target image element belongs to the first category, calculating a sum of a first preset feature value threshold and a product of a preset value and the feature value of the target image element; in the case that the target image element belongs to the second category, first calculating a difference between the feature value of the target image element and the second reference value, and then calculating a difference between a second preset feature value threshold and a product of a preset value and the difference.

3. The method of claim 1, wherein, The analysis module performs analysis processing on the bronchus part image using a trained deep learning model to acquire analysis result data of each image element in the bronchus part image, and comprises the following steps: S311, for each image element, the analysis module acquires a preset size image block with the image element as the core from the bronchus part image, and inputs the acquired image block into the trained deep learning model; S312, the trained deep learning model outputs analysis result data for the input image block, and the analysis result data is a probability value of each category corresponding to a core image element in the image block.

4. The method of claim 3, wherein, After the analysis module obtains the analysis result data of each image element in the trachea part image, the analysis module takes the category corresponding to the maximum probability value in the corresponding analysis result data as the final category to which the image element belongs, and the analysis module uses a plurality of image elements with the same final category to form a feature image range in the trachea part image.

5. The method of claim 4, wherein, For each reference image, the recording module also records the analysis result data of each image element in the reference image, the feature image range in the reference image, the coordinates of the feature image range, and the interpretation information of the feature image range.

6. The method of claim 5, wherein, The analysis module finds a plurality of reference images related to the trachea part image from all the reference images recorded by the recording module, including the following steps: S321, the analysis module divides the trachea part image into a plurality of image ranges, and calculates the average data of the analysis result data of all image elements in each image range; S322, for each reference image, the analysis module divides the reference image into a plurality of image ranges in the same way, and calculates the average data of the analysis result data of all image elements in each image range; S323, for each reference image, the analysis module respectively determines a plurality of image ranges corresponding to each image range in the trachea part image in the reference image, respectively calculates the same degree between the two corresponding image ranges, and accumulates all the same degrees to obtain the overall same degree; S324, the analysis module takes the reference image with a corresponding overall same degree greater than a preset overall same degree threshold as a reference image related to the trachea part image.

7. The method of claim 6, wherein, The analysis module further determines a plurality of final reference images from the found plurality of reference images, including the following steps: S331, the analysis module determines a focus image range in all feature image ranges in the trachea part image, and also determines the coordinates of the focus image range; S332, for each found reference image, the analysis module determines a feature image range in the reference image with coordinates close to the coordinates of the focus image range, and calculates the difference degree between the determined feature image range and the focus image range; S333, the analysis module takes the reference image corresponding to the minimum difference degree as the final reference image.

8. A bronchoscopy image analysis system based on deep learning for implementing the method according to any one of claims 1 to 7, characterized in that, The analysis module further includes the following modules: a recording module for recording a plurality of reference images; a preparation module for obtaining a bronchoscope image, performing inspection processing on the bronchoscope image, determining a plurality of feature image elements, recording the coordinates of the plurality of feature image elements, and obtaining a trachea part image from the bronchoscope image based on the coordinates of the plurality of feature image elements, and sending the trachea part image to the analysis module; An analysis module is configured to analyze the trachea part images using the trained deep learning model, to obtain analysis result data of each image element in the trachea part images, to find out a plurality of control images related to the trachea part images from all the control images recorded by the recording module, and to further determine a plurality of final control images from the plurality of control images, and to analyze and process the lesion conditions using the final control images.

9. A storage medium, characterized by The storage medium stores program instructions, wherein the program instructions, when executed, control a device in which the storage medium is located to perform the method of any one of claims 1 to 7.

Images