Bronchoscope image analysis method and system based on deep learning and storage medium

Abstract

The invention relates to the technical field of image recognition, in particular to a bronchoscope image analysis method and system based on deep learning and a storage medium, and the method comprises the steps: S1, carrying out the inspection processing of a bronchoscope image, determining a plurality of feature image elements, and recording the coordinates of the plurality of feature image elements; s2, based on the coordinates of the plurality of feature image elements, acquiring a trachea part image from the bronchoscope image; s3, performing analysis processing on the trachea part image by using a trained deep learning model, obtaining analysis result data of each image element in the trachea part image, finding out a plurality of contrast images related to the trachea part image from all contrast images recorded by a recording module, and continuously determining a plurality of final contrast images in all the searched contrast images. According to the invention, the final contrast image related to the trachea part image can be provided.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a bronchoscopic image analysis method, system, and storage medium based on deep learning. Background Art

[0002] Currently, the method of using deep learning technology to automatically analyze bronchoscopic images has become increasingly common, with the advantages of high processing efficiency and good objectivity.

[0003] Similar prior art includes Chinese patent application publication number CN118154602A, which discloses an image analysis method and system based on a colorectal polyp CT image dataset. The method comprises: obtaining the inner wall length of polyps in colorectal CT images containing polyps through an image stripping method; optimizing the inner wall length of polyps to obtain a standard polyp length; using the standard polyp length to identify polyps in colorectal polyp CT images; and obtaining analysis results of colorectal polyp CT images based on the maximum polyp length. Furthermore, similar prior art includes Chinese patent application publication number CN119963493A, which discloses a deep learning-based CT image analysis and tumor prediction method and system. The method comprises: collecting CT image data and annotation information of tumor patients to obtain a high-quality CT image dataset; combining neural structure search technology and a composite model expansion method to establish a second-generation efficient convolutional neural network tumor classification model; and adding a coordinate attention module and a hard smoothing activation function to the model to obtain an improved second-generation efficient convolutional neural network tumor classification model. However, neither of the aforementioned patent applications provides images similar to the acquired images to assist in analysis and processing. Summary of the Invention

[0004] The present application extracts a partial image of the trachea from a bronchoscopic image, searches out several control images related to the partial image of the trachea from all the recorded control images, and further determines several final control images from all the searched control images. The present application aims to provide a final control image similar to the partial image of the trachea to assist in analysis and processing.

[0005] This application provides a bronchoscopic image analysis method based on deep learning, comprising the following steps:

[0006] S1. A preparation module acquires a bronchoscopic image, and the preparation module inspects and processes the bronchoscopic image to determine a plurality of characteristic image elements, and the preparation module further records the coordinates of the plurality of characteristic image elements;

[0007] S2. The preparation module obtains a partial image of the trachea from the bronchoscopic image based on the coordinates of the plurality of characteristic image elements, and sends the partial image of the trachea to the analysis module;

[0008] S3. The analysis module uses the trained deep learning model to analyze and process the tracheal partial image to obtain analysis result data for each image element in the tracheal partial image, and the analysis module searches out several control images related to the tracheal partial image from all control images recorded by the recording module. The analysis module also continues to determine several final control images from all the searched control images.

[0009] As a preferred technical solution of the present application, the preparation module inspects and processes the bronchoscopic image to determine a number of characteristic image elements, including the following steps:

[0010] S11, the preparation module calculates the characteristic value of each image element in the bronchoscopic image;

[0011] S12, the preparation module sets an image element as a target image element, and determines whether the characteristic value of the target image element is less than a first reference value. If yes, the target image element is determined to belong to the first category; if not, proceed to the next step;

[0012] S13, the preparation module determines whether the characteristic value of the target image element is greater than a second reference value, and if so, determines that the target image element belongs to the second category; if not, determines that the target image element belongs to the third category;

[0013] S14, the preparation module calculates a eigenvalue threshold corresponding to the target image element according to the category to which the target image element belongs, calculates the eigenvalue difference between the target image element and each surrounding image element, calculates the mean of all eigenvalue differences, and determines whether the mean is greater than the eigenvalue threshold corresponding to the target image element. If so, the target image element is regarded as a feature image element;

[0014] S15. The preparation module determines whether there is an image element that has not been set as a target image element. If yes, jump to S12; if not, end all steps.

[0015] As a preferred technical solution of the present application, the preparation module calculates the eigenvalue threshold corresponding to the target image element according to the category to which the target image element belongs, including: when the target image element belongs to the first category, calculating the sum of a preset first eigenvalue threshold and the product of a preset numerical value and the eigenvalue of the target image element; when the target image element belongs to the second category, first calculating the difference between the eigenvalue of the target image element and the second reference value, and then calculating the difference between the preset second eigenvalue threshold and the product of the preset numerical value and the difference.

[0016] As a preferred technical solution of the present application, the analysis module uses a trained deep learning model to analyze and process the tracheal image to obtain analysis result data of each image element in the tracheal image, including the following steps:

[0017] S311. For each image element, the analysis module obtains an image block of a preset size with the image element as the core from the image of the trachea portion, and inputs the obtained image block into the trained deep learning model;

[0018] S312. Regarding the input image block, the trained deep learning model outputs analysis result data, which is the probability value of the core image elements in the image block corresponding to each category.

[0019] As a preferred technical solution of the present application, after the analysis module obtains the analysis result data of each image element in the tracheal partial image, it also includes: for each image element, the analysis module regards the category corresponding to the maximum probability value in the corresponding analysis result data as the final category to which the image element belongs, and the analysis module uses several image elements with the same final category in the tracheal partial image to form a feature image range.

[0020] As a preferred technical solution of the present application, with respect to each control image, the recording module also records the analysis result data of each image element in the control image, the characteristic image range in the control image, the coordinates of the characteristic image range, and the interpretation information of the characteristic image range.

[0021] As a preferred technical solution of the present application, the analysis module searches for a plurality of reference images related to the tracheal portion image from all reference images recorded by the recording module, comprising the following steps:

[0022] S321, the analysis module divides the tracheal portion image into a plurality of image ranges, and calculates the average data of the analysis result data of all image elements in each image range;

[0023] S322: For each control image, the analysis module divides the control image into a plurality of image ranges using the same method, and calculates the average data of the analysis result data of all image elements in each image range;

[0024] S323. For each control image, the analysis module determines, in the control image, a plurality of image ranges corresponding to respective image ranges in the tracheal portion image, respectively, calculates the degree of identity between two corresponding image ranges, and accumulates all the degrees of identity to obtain an overall degree of identity.

[0025] S324. The analysis module regards the corresponding reference image whose overall similarity is greater than a preset overall similarity threshold as a reference image related to the trachea portion image.

[0026] As a preferred technical solution of the present application, the analysis module further determines a number of final comparison images from all the comparison images found, including the following steps:

[0027] S331, the analysis module determines a focus image range from all feature image ranges in the trachea portion image, and further determines the coordinates of the focus image range;

[0028] S332. For each found comparison image, the analysis module determines a feature image range in the comparison image whose coordinates are close to the coordinates of the focus image range, and calculates the difference between the determined feature image range and the focus image range;

[0029] S333: The analysis module uses the control image corresponding to the minimum difference as the final control image.

[0030] This application also provides a bronchoscopic image analysis system based on deep learning, including the following modules:

[0031] A recording module, used for recording a plurality of comparison images;

[0032] a preparation module, configured to acquire a bronchoscopic image, inspect and process the bronchoscopic image, determine a plurality of characteristic image elements, record the coordinates of the plurality of characteristic image elements, acquire a partial image of the trachea from the bronchoscopic image based on the coordinates of the plurality of characteristic image elements, and send the partial image of the trachea to the analysis module;

[0033] The analysis module is used to use the trained deep learning model to analyze and process the tracheal partial image, obtain the analysis result data of each image element in the tracheal partial image, find out several control images related to the tracheal partial image from all the control images recorded by the recording module, and continue to determine several final control images from all the control images found.

[0034] The present application also provides a storage medium storing program instructions, wherein when the program instructions are executed, the device where the storage medium is located is controlled to execute any one of the above methods.

[0035] Compared with the prior art, the present invention has at least the following advantages:

[0036] In the technical solution provided by the present application, first, the preparation module obtains a bronchoscopic image, and the preparation module inspects and processes the bronchoscopic image to determine a number of characteristic image elements, and the preparation module also records the coordinates of the several characteristic image elements. Secondly, the preparation module obtains a partial image of the trachea from the bronchoscopic image based on the coordinates of the several characteristic image elements, and sends the partial image of the trachea to the analysis module. Finally, the analysis module uses the trained deep learning model to analyze and process the partial image of the trachea, obtains analysis result data for each image element in the partial image of the trachea, and the analysis module finds a number of control images related to the partial image of the trachea from all the control images recorded by the recording module. The analysis module also continues to determine a number of final control images from all the control images found. Through the present application, not only can the partial image of the trachea be accurately extracted from the bronchoscopic image, but also a final control image similar to the partial image of the trachea can be determined for analysis and processing of the pathological condition, thereby improving the reliability of the analysis results. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0038] Figure 1 This is a flowchart of a bronchoscopic image analysis method based on deep learning in an embodiment of the present application;

[0039] Figure 2 A flowchart of a method for searching for multiple reference images in an embodiment of the present application;

[0040] Figure 3 Flowchart of a method for determining a plurality of final comparison images in an embodiment of the present application;

[0041] Figure 4 Schematic diagram of a bronchoscopic image analysis system based on deep learning in an embodiment of the present application. DETAILED DESCRIPTION

[0042] Embodiments of the present application provide a bronchoscopic image analysis method, system and storage medium based on deep learning. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or that are inherent to these processes, methods, products or devices.

[0043] For ease of understanding, the specific process of the embodiment of the present application is described below. Figure 1 The bronchoscopic image analysis method based on deep learning in the embodiment of the present application includes the following main steps:

[0044] S1. The preparation module obtains a bronchoscopic image, and the preparation module inspects and processes the bronchoscopic image to determine a number of characteristic image elements, and the preparation module also records the coordinates of the several characteristic image elements;

[0045] S2. The preparation module obtains a partial image of the trachea from the bronchoscopic image based on the coordinates of the plurality of characteristic image elements, and sends the partial image of the trachea to the analysis module;

[0046] S3. The analysis module uses the trained deep learning model to analyze and process the tracheal partial image, obtains the analysis result data of each image element in the tracheal partial image, and the analysis module searches out several control images related to the tracheal partial image from all the control images recorded by the recording module. The analysis module also continues to determine several final control images from all the control images found.

[0047] Specifically, when analyzing and processing the pathological condition based solely on the acquired bronchoscopic image, there may be a problem in which the analysis results are unreliable. To address this technical problem, in S1, the preparation module acquires a bronchoscopic image, which includes not only the tracheal portion but also the background portion. The preparation module then inspects and processes the bronchoscopic image, determines a number of characteristic image elements, and records the coordinates of the several characteristic image elements. It should be noted that the several characteristic image elements refer to those image elements on the edge of the tracheal portion. In S2, the preparation module acquires a tracheal portion image from the original tracheoscopic image based on the coordinates of the several characteristic image elements, and sends the tracheal portion image to the analysis module. The tracheal portion image is in color. In S3, the analysis module uses a trained deep learning model to analyze and process the tracheal image, obtaining analysis result data for each image element in the tracheal image. The process of training the deep learning model can be implemented using existing technologies and is briefly described here. The process of training the deep learning model includes: obtaining a large number of training images and the correct label corresponding to each training image, where the correct label is the correct category to which the core image element in the corresponding training image belongs; inputting the training images into the deep learning model in sequence; obtaining the probability of the core image element in the training image belonging to each category; taking the category corresponding to the highest probability as the final category; and adjusting the internal weight by comparing the difference between the final category and the actual label to minimize the error. The analysis module searches for several control images related to the tracheal image from all the control images recorded by the recording module, and further determines several final control images from all the control images found, so that the final control images can be used to assist in analysis and processing. The analysis and processing here refers to the analysis and processing of the lesion condition.

[0048] Furthermore, the preparation module examines and processes the bronchoscopic image to determine a number of characteristic image elements, including the following steps:

[0049] S11, a preparation module calculates the feature value of each image element in the bronchoscopic image;

[0050] S12, the preparation module sets an image element as a target image element, and determines whether the characteristic value of the target image element is less than a first reference value. If yes, the target image element is determined to belong to the first category; if not, the process proceeds to the next step;

[0051] S13, the preparation module determines whether the characteristic value of the target image element is greater than the second reference value. If yes, the target image element is determined to belong to the second category; if not, the target image element is determined to belong to the third category.

[0052] S14. The preparation module calculates a eigenvalue threshold corresponding to the target image element according to the category to which the target image element belongs, calculates the eigenvalue difference between the target image element and each surrounding image element, calculates the mean of all eigenvalue differences, and determines whether the mean is greater than the eigenvalue threshold corresponding to the target image element. If so, the target image element is regarded as a feature image element.

[0053] S15. The preparation module determines whether there is an image element that has not been set as the target image element. If yes, jump to S12; if not, end all steps.

[0054] Specifically, how to determine a number of characteristic image elements is described. In S11, the preparation module calculates the characteristic value of each image element in the bronchoscopic image. Specifically, the grayscale formula Gray = (R + G + B) / 3 can be used for calculation, where Gray is the characteristic value, R is the R value of the image element, G is the G value of the image element, and B is the B value of the image element. At this time, the color bronchoscopic image has been converted into a grayscale image to facilitate the determination of characteristic image elements. In S12, the preparation module sets an image element as a target image element. Specifically, an image element in the grayscale image is set as the target image element, and whether the characteristic value of the target image element is less than a first reference value is determined. If so, the target image element is determined to belong to the first category. If not, the process continues to S13. In S13, the preparation module determines whether the characteristic value of the target image element is greater than the second reference value. If so, the target image element is determined to belong to the second category. If not, the target image element is determined to belong to the third category. It should be noted that the second reference value is greater than the first reference value. The classification of the target image element is based on the consideration that in the image range with larger and smaller brightness of the grayscale image, the difference between the characteristic value of the characteristic image element and the characteristic value of the surrounding image elements is small. In S14, the preparation module calculates the characteristic value threshold corresponding to the target image element according to the category to which the target image element belongs, calculates the characteristic value difference between the target image element and each surrounding image element, and the characteristic value difference can take a value greater than 0. The mean of all characteristic value differences is calculated to determine whether the mean is greater than the characteristic value threshold corresponding to the target image element. If so, the target image element is regarded as a characteristic image element. If not, the target image element is not regarded as a characteristic image element. In S15, the preparation module determines whether there is an image element in the grayscale image that has not been set as a target image element. If so, jump to S12. If not, end all steps. The coordinates of the characteristic image elements mentioned above actually refer to the coordinates of the characteristic image elements in the coordinate system of the grayscale image.

[0055] Furthermore, the preparation module calculates the eigenvalue threshold corresponding to the target image element according to the category to which the target image element belongs, including: when the target image element belongs to the first category, calculating the sum of a preset first eigenvalue threshold and the product of a preset numerical value and the eigenvalue of the target image element; when the target image element belongs to the second category, first calculating the difference between the eigenvalue of the target image element and the second reference value, and then calculating the difference between the preset second eigenvalue threshold and the product of the preset numerical value and the difference.

[0056] Specifically, how to calculate the eigenvalue threshold corresponding to the target image element based on the category to which the target image element belongs is described. Prior to this description, it should be noted that in this embodiment, the eigenvalue threshold is set to change as the eigenvalue of the target image element changes. The change process is described here. As can be seen from the above, the minimum eigenvalue is 0 and the maximum is 255. When the eigenvalue is equal to 0, the corresponding eigenvalue threshold is the preset first eigenvalue threshold. When the eigenvalue is greater than 0 and less than a first reference value, that is, when the target image element belongs to the first category, the corresponding eigenvalue threshold increases linearly until the eigenvalue is the first reference value, at which point the eigenvalue threshold becomes the preset second eigenvalue threshold. When the eigenvalue is greater than or equal to the first reference value and less than or equal to the second reference value, that is, when the target image element belongs to the third category, the corresponding eigenvalue threshold remains unchanged at the preset second eigenvalue threshold. When the eigenvalue is greater than the second reference value, that is, when the target image element belongs to the second category, the corresponding eigenvalue threshold begins to decrease linearly. This is done to accurately determine the characteristic image element in both low and high brightness ranges of the grayscale image. The first and second eigenvalue thresholds are set based on the actual application scenario.

[0057] Thus, if the target image element belongs to the first category, the sum of the preset first eigenvalue threshold and the product of the preset value and the eigenvalue of the target image element is calculated, where the preset value is the difference between the second eigenvalue threshold and the first eigenvalue threshold divided by the first reference value. If the target image element belongs to the second category, the difference between the eigenvalue of the target image element and the second reference value is first calculated, and then the difference between the preset second eigenvalue threshold and the product of the preset value and the difference is calculated. If the target image element belongs to the third category, the second eigenvalue threshold is directly used.

[0058] Furthermore, the analysis module uses the trained deep learning model to analyze and process the tracheal image to obtain analysis result data of each image element in the tracheal image, including the following steps:

[0059] S311. For each image element, the analysis module obtains an image block of a preset size with the image element as the core from the image of the trachea portion, and inputs the obtained image block into the trained deep learning model;

[0060] S312. Regarding the input image block, the trained deep learning model outputs analysis result data, which is the probability value of the core image elements in the image block corresponding to each category.

[0061] Specifically, the method for obtaining analysis result data for each image element in a partial image of the trachea is described. In step S311, for each image element, the analysis module obtains an image block of a preset size with the image element as the core, i.e., the image element is located at the center of the image block, from the partial image of the trachea. The obtained image block is then input into a trained deep learning model. In step S312, the trained deep learning model outputs analysis result data for the input image block. The analysis result data is the probability value of each category corresponding to the core image element in the input image block. Each category can be a different lesion category.

[0062] Furthermore, after the analysis module obtains the analysis result data of each image element in the tracheal partial image, it also includes: for each image element, the analysis module regards the category corresponding to the maximum probability value in the corresponding analysis result data as the final category to which the image element belongs, and the analysis module uses several image elements with the same final category in the tracheal partial image to form a feature image range.

[0063] Specifically, the processing performed by the analysis module after obtaining the analysis result data of each image element in the tracheal partial image is introduced. For each image element, the analysis module regards the category corresponding to the maximum probability value in the corresponding analysis result data as the final category to which the image element belongs. The corresponding analysis result data refers to the analysis result data obtained by inputting an image block of a preset size with the image element as the core into a trained deep learning model. In the tracheal partial image, several image elements with the same final category are used to form a feature image range. The number of feature image ranges may be more than one. For ease of understanding, for example, there are multiple image elements with the same category in the tracheal partial image, and the coordinates of these image elements in the image coordinate system are close to each other. If the image range formed by these image elements is large enough, the feature image range can be obtained.

[0064] Furthermore, with respect to each comparison image, the recording module also records analysis result data of each image element in the comparison image, a characteristic image range in the comparison image, coordinates of the characteristic image range, and interpretation information of the characteristic image range.

[0065] Specifically, the recording content of the recording module is introduced. The control image and the tracheal part image have the same size and shape, and the shapes can both be circular. In addition to recording the control image, the recording module also records the analysis result data of each image element in the control image. The analysis result data here have the same meaning as the analysis result data corresponding to the tracheal part image, as well as the characteristic image range in the control image, the coordinates of the characteristic image range, and the interpretation information of the characteristic image range. The characteristic image range here has the same meaning as the characteristic image range in the tracheal part image. The coordinates of the characteristic image range refer to the coordinates of the image element located at the center of the characteristic image range in the image coordinate system. The interpretation information of the characteristic image range can be interpretation information about the lesion range corresponding to the characteristic image range, such as "changes such as unevenness, erosion, and ulceration on the mucosal surface". The interpretation information can be set manually to help analyze and process the lesion conditions mentioned above.

[0066] Furthermore, the analysis module searches for a plurality of reference images related to the tracheal portion image from all reference images recorded by the recording module, including the following steps:

[0067] S321, the analysis module divides the tracheal portion image into a plurality of image ranges, and calculates the average data of the analysis result data of all image elements in each image range;

[0068] S322: For each control image, the analysis module divides the control image into a plurality of image ranges using the same method, and calculates the average data of the analysis result data of all image elements in each image range;

[0069] S323. For each control image, the analysis module determines, in the control image, a plurality of image ranges corresponding to respective image ranges in the tracheal portion image, calculates the degree of identity between two corresponding image ranges, and accumulates all the degrees of identity to obtain an overall degree of identity.

[0070] S324: The analysis module regards the corresponding reference image whose overall similarity is greater than a preset overall similarity threshold as a reference image related to the trachea portion image.

[0071] Specifically, see Figure 2The present invention describes how to find several reference images related to the tracheal portion image from all reference images recorded by the recording module. In step S321, the analysis module divides the tracheal portion image into several image ranges and calculates the average of the analysis result data of all image elements in each image range. The analysis result data may be in the form of a vector. In step S322, for each reference image, the analysis module divides the reference image into several image ranges using the same method and calculates the average of the analysis result data of all image elements in each image range. In step S323, for each reference image, the analysis module identifies several image ranges in the reference image that correspond to respective image ranges in the tracheal portion image and calculates the degree of identity between two corresponding image ranges. Correspondence means that the coordinates of the two image ranges in their respective corresponding image coordinate systems are the same. Specifically, the similarity between the average data corresponding to the two corresponding image ranges may be calculated, and then all the degrees of identity are accumulated to obtain the overall degree of identity. In S324 , the analysis module regards the corresponding reference image whose overall similarity is greater than a preset overall similarity threshold as a reference image related to the tracheal image. The reference image related to the tracheal image is a reference image that is relatively similar to the tracheal image.

[0072] Furthermore, the analysis module further determines a number of final control images from all the control images found, including the following steps:

[0073] S331, the analysis module determines a focus image range from all feature image ranges in the trachea portion image, and also determines the coordinates of the focus image range;

[0074] S332: For each found comparison image, the analysis module determines a feature image range in the comparison image whose coordinates are close to the coordinates of the focus image range, and calculates the difference between the determined feature image range and the focus image range;

[0075] S333: The analysis module uses the control image corresponding to the minimum difference as the final control image.

[0076] Specifically, see Figure 3, introduces how to continue to determine several final control images from all the control images found. In S331, the analysis module determines a focus image range from all the feature image ranges in the tracheal part image. The specific method can be to manually select a feature image range through the analysis module as the focus image range. The lesion condition of the focus image range is relatively serious. Then, the coordinates of the focus image range are determined. The coordinates of the image element located at the center of the focus image range in the image coordinate system can be used as the coordinates of the focus image range. In S332, for each control image found, the analysis module determines a feature image range in the control image whose coordinates are close to the coordinates of the focus image range. The specific method is to calculate the distance between the coordinates of the focus image range and the coordinates of each feature image range in the control image, determine the feature image range corresponding to the minimum distance as the similar feature image range, and then calculate the difference between the determined feature image range and the focus image range. The specific method is to calculate the average R value, the average G value, and the average B value of all image elements in the determined feature image range, and calculate the R value of all image elements in the focus image range. The average value of the G value, the average value of the B value, the difference between the average value of the R value of the determined characteristic image range and the average value of the R value of the focus image range is calculated, and the difference is greater than 0. The difference between the average value of the G value of the determined characteristic image range and the average value of the G value of the focus image range is calculated, and the difference is greater than 0. The difference between the average value of the B value of the determined characteristic image range and the average value of the B value of the focus image range is calculated, and the difference is greater than 0. A weight is set for each difference, and the sum of different weights is 1. The weighted method is used to calculate the difference between the determined characteristic image range and the focus image range. In S333, the analysis module uses the control image corresponding to the smallest difference as the final control image. When a focus image range has been determined in the tracheal part image, the above-mentioned pathological condition analysis and processing can be performed with reference to the interpretation information of the corresponding characteristic image range in the final control image to improve the reliability of the analysis result.

[0077] According to another aspect of the embodiment of the present application, Figure 4 As shown, the present application also provides a bronchoscopic image analysis system based on deep learning, including a recording module, a preparation module, and an analysis module to implement the bronchoscopic image analysis method based on deep learning described above. The functions of each module are as follows:

[0078] A recording module, used for recording a plurality of comparison images;

[0079] a preparation module, configured to acquire a bronchoscopic image, inspect and process the bronchoscopic image, determine a plurality of characteristic image elements, record the coordinates of the plurality of characteristic image elements, acquire a partial image of the trachea from the bronchoscopic image based on the coordinates of the plurality of characteristic image elements, and send the partial image of the trachea to the analysis module;

[0080] The analysis module is used to use the trained deep learning model to analyze and process the tracheal partial image, obtain the analysis result data of each image element in the tracheal partial image, find out several control images related to the tracheal partial image from all the control images recorded by the recording module, and continue to determine several final control images from all the control images found.

[0081] According to another aspect of an embodiment of the present application, a storage medium is further provided, wherein the storage medium stores program instructions, wherein when the program instructions are executed, the device where the storage medium is located is controlled to execute any one of the above methods.

[0082] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0083] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0084] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A bronchoscopic image analysis method based on deep learning, characterized in that: The method comprises the following steps: S1. A preparation module acquires a bronchoscopic image, and the preparation module inspects and processes the bronchoscopic image to determine a plurality of characteristic image elements, and the preparation module further records the coordinates of the plurality of characteristic image elements; S2. The preparation module obtains a partial image of the trachea from the bronchoscopic image based on the coordinates of the plurality of characteristic image elements, and sends the partial image of the trachea to the analysis module; S3. The analysis module uses the trained deep learning model to analyze and process the tracheal partial image to obtain analysis result data for each image element in the tracheal partial image, and the analysis module searches out several control images related to the tracheal partial image from all control images recorded by the recording module. The analysis module also continues to determine several final control images from all the searched control images.

2. The method according to claim 1, characterized in that The preparation module examines and processes the bronchoscopic image to determine a number of characteristic image elements, including the following steps: S11, the preparation module calculates the characteristic value of each image element in the bronchoscopic image; S12, the preparation module sets an image element as a target image element, and determines whether the characteristic value of the target image element is less than a first reference value. If yes, the target image element is determined to belong to the first category; if not, proceed to the next step; S13, the preparation module determines whether the characteristic value of the target image element is greater than a second reference value, and if so, determines that the target image element belongs to the second category; if not, determines that the target image element belongs to the third category; S14, the preparation module calculates a eigenvalue threshold corresponding to the target image element according to the category to which the target image element belongs, calculates the eigenvalue difference between the target image element and each surrounding image element, calculates the mean of all eigenvalue differences, and determines whether the mean is greater than the eigenvalue threshold corresponding to the target image element. If so, the target image element is regarded as a feature image element; S15. The preparation module determines whether there is an image element that has not been set as a target image element. If yes, jump to S12; if not, end all steps.

3. The method according to claim 2, characterized in that The preparation module calculates the eigenvalue threshold corresponding to the target image element according to the category to which the target image element belongs, including: when the target image element belongs to the first category, calculating the sum of a preset first eigenvalue threshold and the product of a preset numerical value and the eigenvalue of the target image element; when the target image element belongs to the second category, first calculating the difference between the eigenvalue of the target image element and the second reference value, and then calculating the difference between the preset second eigenvalue threshold and the product of the preset numerical value and the difference.

4. The method according to claim 1, wherein The analysis module uses the trained deep learning model to analyze and process the tracheal image to obtain analysis result data of each image element in the tracheal image, including the following steps: S311. For each image element, the analysis module obtains an image block of a preset size with the image element as the core from the image of the trachea portion, and inputs the obtained image block into the trained deep learning model; S312. Regarding the input image block, the trained deep learning model outputs analysis result data, which is the probability value of the core image elements in the image block corresponding to each category.

5. The method according to claim 4, characterized in that After the analysis module obtains the analysis result data of each image element in the tracheal portion image, it also includes: for each image element, the analysis module regards the category corresponding to the maximum probability value in the corresponding analysis result data as the final category to which the image element belongs, and the analysis module uses several image elements with the same final category in the tracheal portion image to form a feature image range.

6. The method according to claim 5, characterized in that Regarding each control image, the recording module further records analysis result data of each image element in the control image, a characteristic image range in the control image, coordinates of the characteristic image range, and interpretation information of the characteristic image range.

7. The method according to claim 6, characterized in that The analyzing module searches for a plurality of reference images related to the trachea portion image from all reference images recorded by the recording module, including the following steps: S321, the analysis module divides the tracheal portion image into a plurality of image ranges, and calculates the average data of the analysis result data of all image elements in each image range; S322: For each control image, the analysis module divides the control image into a plurality of image ranges using the same method, and calculates the average data of the analysis result data of all image elements in each image range; S323. For each control image, the analysis module determines, in the control image, a plurality of image ranges corresponding to respective image ranges in the tracheal portion image, respectively, calculates the degree of identity between two corresponding image ranges, and accumulates all the degrees of identity to obtain an overall degree of identity. S324. The analysis module regards the corresponding reference image whose overall similarity is greater than a preset overall similarity threshold as a reference image related to the trachea portion image.

8. The method according to claim 7, characterized in that The analysis module further determines a number of final control images from all the searched control images, including the following steps: S331, the analysis module determines a focus image range from all feature image ranges in the trachea portion image, and further determines the coordinates of the focus image range; S332. For each found comparison image, the analysis module determines a feature image range in the comparison image whose coordinates are close to the coordinates of the focus image range, and calculates the difference between the determined feature image range and the focus image range; S333: The analysis module uses the control image corresponding to the minimum difference as the final control image.

9. A bronchoscopic image analysis system based on deep learning, for implementing the method according to any one of claims 1 to 8, characterized in that: Includes the following modules: A recording module, used for recording a plurality of comparison images; a preparation module, configured to acquire a bronchoscopic image, inspect and process the bronchoscopic image, determine a plurality of characteristic image elements, record the coordinates of the plurality of characteristic image elements, acquire a partial image of the trachea from the bronchoscopic image based on the coordinates of the plurality of characteristic image elements, and send the partial image of the trachea to the analysis module; The analysis module is used to use the trained deep learning model to analyze and process the tracheal partial image, obtain the analysis result data of each image element in the tracheal partial image, find out several control images related to the tracheal partial image from all the control images recorded by the recording module, and continue to determine several final control images from all the control images found.

10. A storage medium, characterized in that: The storage medium stores program instructions, wherein when the program instructions are executed, the device where the storage medium is located is controlled to execute the method according to any one of claims 1 to 8.

Images