A bronchoscope-assisted navigation method for respiratory infectious diseases
By clustering and arc fitting of bronchial grayscale images, the lesion area is screened, the initial swing angle is adjusted, and linear fitting correction is combined, the accuracy problem caused by abnormal protrusions and tangential distortions in bronchoscopic navigation is solved, and the accuracy problem is achieved. More accurate navigation.
Patent Information
- Application Number
- CN202411260658.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-10
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-09-10
AI Technical Summary
The prior art fails to effectively consider abnormal protrusions in the bronchial and tangential distortion in the image, resulting in insufficient adjustment of the bronchoscope angle and reducing the accuracy of the bronchoscope navigation.
By obtaining the grayscale image of the bronchial, clustering and segmenting areas, arc fitting the tracheal cartilage ring area, screening out the real lesion area, adjusting the initial swing angle of the bronchoscope, combining linear fitting and angle correction, the accuracy of the bronchoscope is achieved. navigation.
It improves the accuracy of bronchoscopic navigation, avoids secondary damage to the inner wall of the bronchial, and enhances the accuracy of navigation.
Smart Images

Figure CN118787301B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of bronchoscope navigation, and in particular to a bronchoscope-assisted navigation method for respiratory infectious diseases. Background Art
[0002] Bronchoscopes are mainly used to observe and treat various respiratory tract lesions. When patients suffer from infectious diseases that lead to respiratory infections, bronchoscopes are needed to check the respiratory infections. Therefore, the movement of the bronchoscope in the bronchi needs to be precisely controlled and navigated to prevent scratches on the inner wall of the bronchus during the examination with the bronchoscope, thereby causing secondary damage to the patient.
[0003] In the related art, the distance between the bronchoscope and the inner wall of the bronchus is usually analyzed based on the image collected by the bronchoscope, and the angle of the bronchoscope is adjusted to achieve navigation of the bronchoscope. However, since the bronchi of patients with respiratory tract infections usually have abnormal protrusions caused by the disease, and the images collected by the bronchoscope have tangential distortion, the adjustment of the bronchoscope angle is not accurate enough, thereby reducing the navigation accuracy of the bronchoscope. Summary of the invention
[0004] In order to solve the technical problem that the existing method does not take into account the abnormal protrusion phenomenon in the bronchus and the tangential distortion phenomenon in the image, resulting in inaccurate adjustment of the bronchoscope angle and reduced navigation accuracy of the bronchoscope, the purpose of the present invention is to provide a bronchoscope-assisted navigation method for respiratory infectious diseases, and the technical scheme adopted is as follows:
[0005] The present invention proposes a bronchoscope-assisted navigation method for respiratory infectious diseases, the method comprising:
[0006] Obtaining a grayscale image of the bronchus before the bronchoscope is moved, clustering the pixels in the grayscale image, and obtaining a plurality of segmented regions;
[0007] Perform arc fitting on the pixel points in each segmented area to obtain a fitting arc, a fitting center of the fitting arc, and a fitting error; filter out the tracheal cartilage ring area from all segmented areas according to the change in the grayscale value of the pixel points on both sides of the fitting arc in each segmented area and the fitting error; take the segmented areas other than the tracheal cartilage ring area and adjacent to the tracheal cartilage ring area as suspected lesion areas, and filter out the real lesion area from all suspected lesion areas according to the distribution of the grayscale values of the pixel points in each suspected lesion area and the distance between the fitting centers of the fitting arcs of the two tracheal cartilage ring areas adjacent to each suspected lesion area;
[0008] The nearest lesion area is selected from all the real lesion areas; the midpoint of the line between the fitting centers of the fitting arcs of two tracheal cartilage ring areas adjacent to the nearest lesion area is used as the common fitting center; according to the distance between the edge pixel point of the nearest lesion area and the common fitting center, and the number of pixel points in the nearest lesion area, the first straight line of the nearest lesion area and the initial swing angle of the bronchoscope are obtained; the fitting centers of the fitting arcs of all tracheal cartilage ring areas are fitted with a straight line to obtain a second straight line; according to the angle between the first straight line and the second straight line, the initial swing angle is adjusted to obtain the real swing angle of the bronchoscope;
[0009] The bronchoscope is navigated according to the actual swing angle and the number of actual lesion areas.
[0010] Furthermore, obtaining the fitting arc and the fitting center and fitting error of the fitting arc includes:
[0011] For any segmented area, the least square method is used to perform arc fitting on the pixel points in the segmented area to obtain the fitting arc and the fitting error of the fitting arc;
[0012] The intersection points of the normal lines at each pixel point on the fitting arc are obtained, and the intersection point with the largest number of normal lines passing through is used as the fitting center of the fitting arc.
[0013] Further, screening out the tracheal cartilage ring region from all segmented regions includes:
[0014] Taking any segmentation area as a target segmentation area, taking any pixel point on the fitting arc of the target segmentation area as a target pixel point, and taking the line connecting the two intersection points of the normal line of the fitting arc of the target segmentation area at the target pixel point and the edge of the target segmentation area as a reference line segment of the target pixel point; taking the target pixel point as the segmentation point, dividing the reference line segment into a first line segment and a second line segment;
[0015] Taking any one of the first line segment and the second line segment as the target line segment, starting from the end point of the target line segment at the edge of the target segmentation area, along the target line segment, calculating the difference between the grayscale value of each pixel point on the target line segment and the adjacent next pixel point, and obtaining the grayscale change of each pixel point on the target line segment; taking the accumulated value of the grayscale change of all pixels on the target line segment as the grayscale gradient value of the target line segment; taking the sum of the grayscale gradient values of the first line segment and the second line segment as the grayscale change characteristic value of the target pixel point;
[0016] The accumulated value of the grayscale change characteristic values of all the pixels on the fitting arc of the target segmentation region is used as the comprehensive change characteristic value of the target segmentation region;
[0017] Perform negative correlation mapping on the fitting error of the fitting arc of the target segmentation area to obtain the arc similarity of the target segmentation area;
[0018] The comprehensive change characteristic value and the arc similarity are integrated and normalized to obtain the possibility of the tracheal cartilage ring in the target segmentation area;
[0019] The segmented region whose tracheal cartilage ring possibility is greater than a preset first threshold is used as the tracheal cartilage ring region.
[0020] Furthermore, screening out the real lesion area from all the suspected lesion areas includes:
[0021] Taking any suspected lesion area as the target suspected lesion area, taking any pixel point in the target suspected lesion area as the pixel point to be tested, analyzing the discrete degree of the grayscale value of each pixel point in a preset neighborhood centered on the pixel point to be tested, and obtaining the local grayscale disorder degree of the pixel point to be tested;
[0022] The local grayscale disorder of all pixels in the target suspected lesion area is integrated to obtain the integrated grayscale disorder of the target suspected lesion area;
[0023] Negative correlation mapping is performed on the distance between the centers of the fitted arcs of two tracheal cartilage ring regions adjacent to the target suspected lesion region to obtain the distance characteristic value of the target suspected lesion region;
[0024] The comprehensive grayscale chaos degree and the distance characteristic value are integrated and normalized to obtain the lesion possibility of the target suspected lesion area;
[0025] The suspected lesion area whose lesion possibility is greater than a preset second threshold is regarded as a real lesion area.
[0026] Furthermore, selecting the nearest lesion region from all real lesion regions includes:
[0027] The tracheal cartilage ring area with the largest number of pixels is taken as the nearest cartilage ring area;
[0028] The nearest lesion region is selected from all real lesion regions, and the distance between the center of mass of the nearest lesion region and the nearest cartilage ring region is the smallest.
[0029] Furthermore, obtaining the first straight line of the nearest lesion area and the initial swing angle of the bronchoscope includes:
[0030] The edge pixel point corresponding to the minimum value of the distance between the most adjacent lesion area and the center of the common fitting circle is used as the outermost pixel point of the most adjacent lesion area;
[0031] The straight line where the outermost pixel point and the common fitting circle center are located is taken as the first straight line;
[0032] Performing negative correlation mapping on the distance between the outermost pixel point and the common fitting circle center to obtain the proximity degree of the nearest lesion area;
[0033] The number of pixels in the most adjacent lesion area and the proximity are integrated and normalized to obtain the adjustment degree of the bronchoscope;
[0034] The preset maximum angle is weighted using the adjustment degree to obtain the initial swing angle of the bronchoscope.
[0035] Furthermore, obtaining the real swing angle of the bronchoscope includes:
[0036] Normalizing the angle between the first straight line and the second straight line to obtain a correction coefficient of the bronchoscope;
[0037] Using the correction coefficient, weighting the initial swing angle to obtain an angle adjustment amount of the bronchoscope;
[0038] The sum of the initial swing angle and the angle adjustment amount is taken as the actual swing angle of the bronchoscope.
[0039] Furthermore, navigating the bronchoscope according to the real swing angle and the number of real lesion areas includes:
[0040] The actual swing angle and the number of actual lesion areas are integrated and normalized to obtain the possibility of bronchoscope withdrawal;
[0041] If the withdrawal possibility is not greater than a preset withdrawal threshold, the bronchoscope is adjusted along the direction of the outermost pixel point pointing to the common fitting circle center at the real swing angle, and the bronchoscope is pushed forward at a preset moving distance;
[0042] If the withdrawal possibility is greater than a preset withdrawal threshold, the bronchoscope is withdrawn from the bronchus.
[0043] Furthermore, the screening out of the real lesion area from all the suspected lesion areas based on the lesion possibility of each suspected lesion area includes:
[0044] The suspected lesion area whose lesion possibility is greater than a preset second threshold is regarded as a real lesion area.
[0045] Furthermore, the preset withdrawal threshold has a value range of .
[0046] The present invention has the following beneficial effects:
[0047] Since the existing method does not take into account the abnormal protrusions in the bronchi and the tangential distortion in the image, the adjustment of the bronchoscope angle is not accurate enough, which reduces the navigation accuracy of the bronchoscope. Therefore, the grayscale image of the bronchus is first clustered to obtain multiple segmented areas. Considering that the cartilage ring of the inner wall of the bronchus appears arc-shaped in the image, and the grayscale value of the pixels in the cartilage ring area in the image gradually increases from the inside to the sides, the tracheal cartilage ring area in the image can be extracted. Since the abnormally protruding lesion area on the inner wall of the bronchus will damage the integrity of the tracheal cartilage ring area, and the distribution of the grayscale value of the pixels in the lesion area is relatively chaotic, the real lesion area can be further extracted, and the area closest to the bronchoscope can be selected. The nearest lesion area, and the common fitting center represents the position of the bronchoscope. Since the lesion area has a convex feature, the bronchoscope is easy to touch the lesion area during the advancement of the bronchus. Therefore, the initial swing angle is used to preliminarily represent the angle that the bronchoscope needs to be adjusted before advancement to avoid secondary damage to the patient by the bronchoscope during navigation. Considering that the tangential distortion of the grayscale image will affect the accuracy of the bronchoscope angle adjustment, the more serious the tangential distortion, the larger the angle between the first straight line and the second straight line, making the initial swing angle smaller, so the initial swing angle can be adjusted to obtain the real swing angle of the bronchoscope, thereby improving the accuracy of the bronchoscope angle adjustment and the navigation accuracy of the bronchoscope. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0049] Figure 1 A flow chart of a bronchoscope-assisted navigation method for respiratory infectious diseases provided by one embodiment of the present invention;
[0050] Figure 2 A schematic diagram of a grayscale image of a bronchus provided by an embodiment of the present invention;
[0051] Figure 3 A schematic diagram of the structure of a suspected lesion area and two adjacent tracheal cartilage ring areas provided by an embodiment of the present invention;
[0052] Figure 4A flow chart of a method for obtaining the first straight line of the nearest lesion area and the initial swing angle of the bronchoscope provided in one embodiment of the present invention. DETAILED DESCRIPTION
[0053] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the specific implementation method, structure, features and effects of a bronchoscope-assisted navigation method for respiratory infectious diseases proposed by the present invention in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.
[0054] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
[0055] The specific scheme of the bronchoscope-assisted navigation method for respiratory infectious diseases provided by the present invention is described in detail below with reference to the accompanying drawings.
[0056] See also Figure 1 , which shows a flow chart of a bronchoscope-assisted navigation method for respiratory infectious diseases provided by an embodiment of the present invention, the method comprising:
[0057] Step S1: obtaining a grayscale image of the bronchus before each movement of the bronchoscope, clustering the pixels in the grayscale image, and obtaining a plurality of segmented regions.
[0058] In the process of using a bronchoscope to examine the patient's bronchi, it is necessary to accurately control the direction and distance of the bronchoscope to avoid damaging the inner wall of the bronchus. Therefore, before the bronchoscope moves, the embodiment of the present invention uses the bronchoscope to collect the original image of the inside of the bronchus. At the same time, in order to reduce the calculation amount of subsequent image processing and improve the processing speed, in one embodiment of the present invention, the collected original image is grayed and converted into a single-channel grayscale image. Please refer to Figure 2 , which shows a schematic diagram of a grayscale image of a bronchus provided by an embodiment of the present invention. It should be noted that grayscale processing is a technical means well known to those skilled in the art, and will not be described in detail here.
[0059] Since there are raised cricoid cartilage rings on the inner wall of the bronchus, they have obvious features in grayscale images. In addition, for patients with respiratory infectious diseases, due to the infection of the bronchi, abnormally raised lesion areas will appear on the inner wall of the bronchus, which also have obvious features in grayscale images. Therefore, the embodiment of the present invention first uses a density peak clustering algorithm to cluster the pixels in the grayscale image to segment various areas in the grayscale image and obtain multiple segmented areas, wherein the segmented areas include areas representing tracheal cartilage rings, lesion areas, and background areas of the inner wall of the bronchus other than these two areas. Subsequently, based on the overall morphology and grayscale value distribution of the segmented areas, the tracheal cartilage ring areas and lesion areas can be extracted to achieve precise navigation of the bronchoscope. The density peak clustering algorithm is a technical means well known to those skilled in the art and will not be elaborated here. In other embodiments of the present invention, clustering can also be achieved using existing clustering algorithms such as the DBSCAN clustering algorithm, which is not limited here.
[0060] Step S2: Perform arc fitting on the pixel points in each segmented area to obtain the fitting arc, the fitting center and the fitting error of the fitting arc; filter out the tracheal cartilage ring area from all segmented areas according to the change in the grayscale value of the pixels on both sides of the fitting arc and the fitting error in each segmented area; take the segmented areas other than the tracheal cartilage ring area and adjacent to the tracheal cartilage ring area as suspected lesion areas, and filter out the real lesion area from all suspected lesion areas according to the distribution of the grayscale values of the pixels in each suspected lesion area and the distance between the fitting centers of the fitting arcs of the two tracheal cartilage ring areas adjacent to each suspected lesion area.
[0061] In order to subsequently control the bronchoscope navigation, the embodiment of the present invention first needs to extract the area representing the tracheal cartilage ring in the grayscale image. Since the multiple cartilage rings on the inner wall of the bronchus are distributed in parallel with each other, and due to the shooting angle of the bronchoscope and the limitations of uneven lighting, the cartilage ring area in the grayscale image is not a complete annular area, but an arc-shaped area. Therefore, the present invention first performs arc fitting on the pixel points in each segmented area to obtain the fitting arc and the fitting center and fitting error of the fitting arc. Subsequently, the overall morphology of the segmented area can be analyzed based on the fitting error of the fitting arc.
[0062] Preferably, in one embodiment of the present invention, the method for obtaining the fitting arc and the fitting center and fitting error of the fitting arc specifically includes:
[0063] First, for any segmented area, the least squares method is used to perform arc fitting on the pixel points in the segmented area to obtain the fitting arc and the fitting error of the fitting arc. Among them, the least squares method is a technical means well known to technical personnel in this field and will not be described in detail here.
[0064] Then, the intersection points of the normals at each pixel point on the fitting arc are obtained, and the intersection point with the largest number of normals passing through is used as the fitting center of the fitting arc. The acquisition of normals is a technical means well known to technicians in this field and will not be elaborated here.
[0065] Since the annular cartilage ring on the inner wall of the bronchus has a convex feature, in the grayscale image, the grayscale value of the pixels in the area representing the cartilage ring shows a feature of gradually increasing from the inside to the sides. At the same time, the area representing the cartilage ring shows an arc feature in overall morphology. Therefore, the changes in the grayscale values of the pixels on both sides of the fitting arc in each segmented area can be analyzed. At the same time, combined with the fitting error of the fitting arc of each segmented area, multiple tracheal cartilage ring areas can be screened out from all segmented areas. The tracheal cartilage ring area can be further analyzed later, the lesion area can be extracted, and the advancement angle of the bronchoscope can be precisely adjusted.
[0066] Preferably, in one embodiment of the present invention, the method for obtaining the tracheal cartilage ring area specifically includes:
[0067] Take any segmentation area as the target segmentation area, and after fitting the segmentation area with an arc through the above process, the obtained fitting arc is generally distributed in the center of the segmentation area, and the segmentation area is divided into two areas with basically the same area. Therefore, any pixel point on the fitting arc of the target segmentation area can be taken as the target pixel point, and the line connecting the two intersection points where the normal of the fitting arc of the target segmentation area at the target pixel point intersects with the edge of the target segmentation area is taken as the reference line segment of the target pixel point; take the target pixel point as the segmentation point, divide the reference line segment into a first line segment and a second line segment, wherein the pixels on the first line segment and the second line segment are respectively distributed on both sides of the target pixel point, and subsequently analyze the possibility that the target segmentation area is the tracheal cartilage ring area based on the change of the grayscale value of the pixels on the first line segment and the second line segment.
[0068] It should be noted that, in order to smoothly perform subsequent calculations, it is stipulated that the target pixel point belongs neither to the first line segment nor to the second line segment.
[0069] Since the subsequent analysis process of the first line segment and the second line segment is the same, any one of the first line segment and the second line segment can be used as the target line segment. Starting from the end point of the target line segment at the edge of the target segmentation area, the difference between the grayscale values of each pixel point on the target line segment and the next adjacent pixel point is calculated along the target line segment to obtain the grayscale change of each pixel point on the target line segment. During the calculation process, for the last pixel point on the target line segment, the grayscale change of the pixel point can be obtained by calculating the difference between the grayscale values of the pixel point and the target pixel point; the accumulated value of the grayscale change of all pixels on the target line segment is used as the grayscale gradient value of the target line segment; the sum of the grayscale gradient values of the first line segment and the second line segment is used as the grayscale change characteristic value of the target pixel point. The larger the grayscale change characteristic value, the more obvious the increasing feature of the grayscale values of the pixels on both sides of the target pixel point, which provides a data basis for the subsequent analysis of the grayscale change in the target segmentation area.
[0070] As an example, in one embodiment of the present invention, the expression of the grayscale change characteristic value of the target pixel point can be specifically, for example, as follows:
[0071]
[0072]
[0073] in, Represents the grayscale change characteristic value of the target pixel; Represents the grayscale gradient value of the first line segment; Represents the grayscale gradient value of the second line segment; Indicates the grayscale gradient value of the target line segment; Indicates the first The grayscale change of each pixel; Indicates the number of pixels on the target line segment.
[0074] Then, the accumulated value of the grayscale change characteristic values of all pixels on the fitting arc of the target segmentation area can be used as the comprehensive change characteristic value of the target segmentation area. The larger the comprehensive change characteristic value, the more obvious the characteristic that the grayscale values of the pixels in the target segmentation area gradually increase from the inside to the sides, and thus the greater the possibility that the target segmentation area is the tracheal cartilage ring area. At the same time, the smaller the fitting error of the fitting arc of the target segmentation area, the more the overall shape of the target segmentation area conforms to the arc feature. Therefore, the fitting error of the fitting arc of the target segmentation area can be negatively correlated to obtain the arc similarity of the target segmentation area.
[0075] Then, the comprehensive change feature value and arc similarity can be integrated and normalized to obtain the possibility of tracheal cartilage rings in the target segmentation area.
[0076] As an example, in one embodiment of the present invention, the expression of the possibility of the tracheal cartilage ring in the target segmentation area can be specifically, for example, as follows:
[0077]
[0078] in, Indicates the possibility of tracheal cartilage ring in the target segmentation area; Represents the fitting error of the fitting arc of the target segmentation area; The first Grayscale change characteristic value of each pixel; Represents the number of pixels on the fitting arc of the target segmentation area; Indicates the arc similarity of the target segmentation area; Represents the comprehensive change feature value of the target segmentation area; Represents the activation function, which is used for normalization; Indicates the preset first adjustment parameter, which is used to prevent the denominator from being 0. The value range is In one embodiment of the present invention, Set to 0.01, The specific value of can also be set by the implementer according to the specific implementation scenario and is not limited here.
[0079] It should be noted that in other embodiments of the present invention, the positive and negative correlations may be represented by other basic mathematical operations, which will not be elaborated herein.
[0080] The possibility of tracheal cartilage rings in each segmented area can be obtained by the same method as above. The greater the possibility of tracheal cartilage rings, the more likely the segmented area is to be a region that represents the tracheal cartilage rings. Therefore, the segmented area with a possibility of tracheal cartilage rings greater than the preset first threshold is taken as the tracheal cartilage ring area, wherein the preset first threshold is set to 0.6. The specific value of the preset first threshold can also be set by the implementer according to the specific implementation scenario, and is not limited here.
[0081] When the patient's bronchi are infected, abnormally raised lesion areas will appear on the inner wall of the bronchus. Therefore, the bronchoscope is likely to touch the lesion areas during the advancement of the bronchus, thereby causing secondary damage to the patient. Therefore, the embodiment of the present invention also needs to extract the lesion areas in the grayscale image to accurately navigate the bronchoscope.
[0082] Considering that the segmented area also includes the lesion area and the bronchial inner wall background area in addition to the tracheal cartilage ring area, and the lesion area generally damages the structure of the tracheal cartilage ring area, that is, the lesion area divides a larger tracheal cartilage ring area into two relatively smaller areas, the embodiment of the present invention firstly regards the segmented area other than the tracheal cartilage ring area and adjacent to the tracheal cartilage ring area as the suspected lesion area, please refer to Figure 3 , which shows a structural schematic diagram of a suspected lesion area and two adjacent tracheal cartilage ring areas provided by an embodiment of the present invention. The smaller the distance between the fitting centers of the fitting arcs of the two tracheal cartilage ring areas adjacent to the suspected lesion area, the more likely it is that the two tracheal cartilage ring areas are the same area before being destroyed by the suspected lesion area, and thus the greater the possibility that the suspected lesion area is the real lesion area. At the same time, the surface of the abnormally raised lesion area is uneven, and the distribution of grayscale values in the lesion area in the grayscale image is relatively chaotic. Therefore, based on the above features, multiple real lesion areas can be screened out from all suspected lesion areas, and the bronchoscope can be accurately navigated in combination with the real lesion areas and tracheal cartilage ring areas to avoid secondary damage to the patient.
[0083] Preferably, in one embodiment of the present invention, the method for acquiring the real lesion area specifically includes:
[0084] Take any suspected lesion area as the target suspected lesion area, take any pixel in the target suspected lesion area as the pixel to be tested, analyze the discrete degree of the grayscale values of each pixel in the preset neighborhood centered on the pixel to be tested, and obtain the local grayscale chaos of the pixel to be tested. The larger the local chaos, the more chaotic the distribution of the local grayscale value of the pixel to be tested. Therefore, the local grayscale chaos of each pixel in the target suspected lesion area can be obtained by the same method, and then the local grayscale chaos of all pixels in the target suspected lesion area are integrated to obtain the comprehensive grayscale chaos of the target suspected lesion area. The larger the comprehensive grayscale chaos, the more chaotic the grayscale value distribution in the target suspected lesion area, and then the target suspected lesion area is more likely to be a real lesion area.
[0085] In one embodiment of the present invention, the size of the preset neighborhood is set to The specific size of the preset neighborhood can also be set by the implementer according to the specific implementation scenario and is not limited here.
[0086] In an embodiment of the present invention, the standard deviation or variance of the grayscale values of each pixel in a preset neighborhood centered on the pixel to be tested can be used as the local grayscale chaos of the pixel to be tested to analyze the degree of discreteness of the grayscale values of each pixel in the preset neighborhood, which is not limited here.
[0087] In an embodiment of the present invention, the accumulated value or multiplied value of the local grayscale chaos of all pixels in the target suspected lesion area can be used as the comprehensive grayscale chaos of the target suspected lesion area to achieve the integration of the local grayscale chaos of all pixels in the target suspected lesion area, which is not limited here.
[0088] The distance between the fitting centers of the fitting arcs of the two tracheal cartilage ring regions adjacent to the target suspected lesion region is negatively correlated to obtain the distance characteristic value of the target suspected lesion region. The larger the distance characteristic value, the more likely it is that the two tracheal cartilage ring regions adjacent to the target suspected lesion region were the same region before being destroyed by the target suspected lesion region, which further indicates that the suspected lesion region is more likely to be the real lesion region. The distance between the fitting centers can be measured by the Euclidean distance.
[0089] Then, the comprehensive grayscale chaos and distance eigenvalues are integrated and normalized to obtain the lesion possibility of the target suspected lesion area. The greater the lesion possibility, the more likely the target suspected lesion area is to be a real lesion area.
[0090] In the embodiment of the present invention, the integration of the comprehensive grayscale disorder degree and the distance feature value can be achieved by calculating the sum or product of the two, which is not limited here.
[0091] As an example, in one embodiment of the present invention, the expression of the lesion possibility of the target suspected lesion area may be specifically, for example, as follows:
[0092]
[0093] in, Indicates the lesion possibility of the target suspected lesion area; It represents the distance between the centers of the fitted arcs of the two tracheal cartilage ring regions adjacent to the target suspected lesion region; Indicates the distance feature value of the target suspected lesion area; Indicates the first The local grayscale chaos of each pixel; Indicates the number of pixels in the target suspected lesion area; Indicates the comprehensive grayscale chaos of the target suspected lesion area; represents the hyperbolic tangent function, which is used for normalization; Indicates the preset second adjustment parameter, which is used to prevent the denominator from being 0. The value range is In one embodiment of the present invention, Set to 0.01, The specific value of can also be set by the implementer according to the specific implementation scenario and is not limited here.
[0094] The lesion possibility of each suspected lesion area can be obtained by the same method as above. The greater the lesion possibility, the more likely the suspected lesion area is to be a real lesion area. Therefore, the suspected lesion area with a lesion possibility greater than the preset second threshold can be taken as a real lesion area, where the preset second threshold is set to 0.6. The specific value of the preset second threshold can also be set by the implementer according to the specific implementation scenario, and is not limited here.
[0095] At this point, the tracheal cartilage ring area and the true lesion area are extracted from the grayscale image.
[0096] Step S3: Select the nearest lesion area from all real lesion areas; take the midpoint of the line between the fitting centers of the fitting arcs of two tracheal cartilage ring areas adjacent to the nearest lesion area as the common fitting center; obtain the first straight line of the nearest lesion area and the initial swing angle of the bronchoscope according to the distance between the edge pixel point of the nearest lesion area and the common fitting center, and the number of pixel points in the nearest lesion area; perform straight line fitting on the fitting centers of the fitting arcs of all tracheal cartilage ring areas to obtain a second straight line; adjust the initial swing angle according to the angle between the first straight line and the second straight line to obtain the real swing angle of the bronchoscope.
[0097] During the process of advancing the bronchoscope in the bronchus, in order to prevent the bronchoscope from causing damage to the bronchus, it is necessary to ensure that the bronchoscope does not touch the actual lesion area closest to the bronchoscope. Therefore, it is necessary to first select a nearest lesion area closest to the bronchoscope from all the actual lesion areas. The nearest lesion area can be analyzed later to determine the angle at which the bronchoscope needs to be adjusted.
[0098] Preferably, in one embodiment of the present invention, the method for obtaining the nearest lesion area specifically includes:
[0099] From the imaging principle of near is big and far is small, we can know that the larger the area of the tracheal cartilage ring region, the more pixels there are, which means that the tracheal cartilage ring region is closer to the bronchoscope. Therefore, the tracheal cartilage ring region with the largest number of pixels can be used as the nearest cartilage ring region. The distance between the nearest cartilage ring region and the bronchoscope is the shortest, and then the nearest lesion region is selected from all real lesion regions. The distance between the center of mass of the nearest lesion region and the nearest cartilage ring region is the smallest. The distance between the centers of mass can be measured using the Euclidean distance, where the acquisition of the center of mass is a technical means well known to technicians in this field and will not be elaborated here.
[0100] Since the two tracheal cartilage ring areas adjacent to the most adjacent lesion area belonged to the same tracheal cartilage ring area before being destroyed, the midpoint of the line connecting the fitting centers of the fitting arcs of the two tracheal cartilage ring areas adjacent to the most adjacent lesion area can be used as the common fitting center. The common fitting center represents the center of the two tracheal cartilage ring areas before being destroyed. At the same time, the position of the common fitting center also represents the position of the bronchoscope. Since the real lesion area usually has a convex feature, the swing angle of the bronchoscope needs to be precisely controlled during the advancement of the bronchoscope to keep a certain distance between the bronchoscope and the edge of the most adjacent lesion area to prevent the bronchoscope from being damaged. The bronchoscope touches the nearest lesion area. At the same time, the more pixels there are in the nearest lesion area, the larger its volume is, and the greater the degree of adjustment of the swing angle of the bronchoscope is required to avoid touching the nearest lesion area. Therefore, the distance between the edge pixel points of the nearest lesion area and the common fitting circle center, as well as the number of pixels in the nearest lesion area can be analyzed to obtain the first straight line of the nearest lesion area and the initial swing angle of the bronchoscope. In the subsequent steps, the angle between the first straight line and the second straight line obtained subsequently can be used to adjust the initial swing angle to eliminate the interference of the tangential distortion generated in the grayscale image on the swing angle analysis.
[0101] Preferably, in one embodiment of the present invention, the method for acquiring the first straight line of the nearest lesion area and the initial swing angle of the bronchoscope specifically includes:
[0102] See also Figure 4 , which shows a flow chart of a method for obtaining the first straight line of the nearest lesion area and the initial swing angle of the bronchoscope provided by an embodiment of the present invention.
[0103] Step S301: The edge pixel point corresponding to the minimum distance between the nearest lesion area and the common fitting circle center is used as the outermost pixel point of the nearest lesion area; the straight line between the outermost pixel point and the common fitting circle center is used as the first straight line.
[0104] Since the lesion area on the inner wall of the bronchus has a convex feature, and the common fitting center can roughly represent the position of the bronchoscope, in order to avoid the bronchoscope touching the nearest lesion area during advancement, it is necessary to first find the pixel point closest to the common fitting center on the edge of the nearest lesion area, that is, the outermost pixel point, and then use the straight line where the outermost pixel point and the common fitting center are located as the first straight line. The first straight line is used for further adjustment of the swing angle of the bronchoscope in the subsequent process, wherein the distance between the edge pixel point of the nearest lesion area and the common fitting center can be measured by the Euclidean distance.
[0105] Step S302: negative correlation mapping is performed on the distance between the outermost pixel point and the common fitting circle center to obtain the proximity degree of the nearest lesion area.
[0106] The smaller the distance between the outermost pixel point and the common fitting circle center, the closer the outermost pixel point is to the bronchoscope. At this time, the angle of the bronchoscope needs to be adjusted to a greater extent to prevent the bronchoscope from touching the nearest lesion area. Therefore, the distance between the outermost pixel point and the common fitting circle center can be negatively correlated to obtain the degree of proximity to the nearest lesion area. The greater the degree of proximity, the closer the boundary of the nearest lesion area is to the bronchoscope.
[0107] Step S303: The number and proximity of the pixels in the nearest lesion area are integrated and normalized to obtain the adjustment degree of the bronchoscope; the preset maximum angle is weighted using the adjustment degree to obtain the initial swing angle of the bronchoscope.
[0108] The more pixels there are in the nearest lesion area, the larger the volume of the lesion protrusion is in reality. In order to prevent the bronchoscope from touching the nearest lesion area, the angle of the bronchoscope needs to be adjusted to a greater extent. Therefore, the number and proximity of pixels in the nearest lesion area can be integrated and normalized to obtain the adjustment degree of the bronchoscope. The greater the adjustment degree, the greater the degree of adjustment that needs to be made for the advancement angle of the bronchoscope. The adjustment degree is then used to weight the preset maximum angle to obtain the initial swing angle of the bronchoscope. The initial swing angle indicates the angle that needs to be adjusted during the advancement of the navigation. The preset maximum angle is set to 90 degrees. The specific value of the preset maximum angle can also be set by the implementer according to the specific implementation scenario, and is not limited here.
[0109] In the embodiment of the present invention, the combination of the two can be achieved by calculating the sum or product of the number and proximity of the pixels in the nearest lesion area, which is not limited here.
[0110] As an example, in one embodiment of the present invention, the expression of the initial swing angle of the bronchoscope may be specifically, for example, as follows:
[0111]
[0112]
[0113] in, Indicates the initial swing angle of the bronchoscope; Indicates the degree of adjustment of the bronchoscope; Indicates the preset maximum angle; Represents the distance between the outermost pixel and the center of the common fitting circle; It indicates the proximity of the most adjacent lesion area; Represents the number of pixels in the area closest to the lesion; represents the hyperbolic tangent function, which is used for normalization; Indicates the preset third adjustment parameter, which is used to prevent the denominator from being 0. The value range is In one embodiment of the present invention, Set to 0.01, The specific value of can also be set by the implementer according to the specific implementation scenario and is not limited here.
[0114] The range of motion of the bronchoscope in the trachea is limited, and the direction of the bronchoscope cannot always be kept consistent with the extension direction of the trachea. Therefore, when shooting, the image is prone to tangential distortion, resulting in a large error between the important data such as the distance obtained above and the actual situation, which in turn leads to a low accuracy of the initial swing angle obtained above. Further adjustment of the initial swing angle is required to eliminate the interference of the tangential distortion of the image on the navigation of the bronchoscope.
[0115] When the direction of the bronchoscope is inconsistent with the extension direction of the bronchus, resulting in tangential distortion in the grayscale image, the fitting centers corresponding to the fitting arcs of each tracheal cartilage ring area in the grayscale image will be roughly distributed in the same direction, which is opposite to the placement direction of the bronchoscope. Therefore, the embodiment of the present invention first uses the least squares method to perform straight line fitting on the fitting centers of the fitting arcs of all tracheal cartilage ring areas to obtain a second straight line, then all the fitting centers are distributed near the second straight line. At this time, the larger the angle between the second straight line and the first straight line obtained above, the more serious the tangential distortion phenomenon, and the above-calculated initial swing angle will be smaller than the actual situation. Therefore, the initial swing angle can be adjusted according to the angle between the first straight line and the second straight line to obtain the actual swing angle of the bronchoscope, thereby improving the accuracy of subsequent bronchoscope navigation.
[0116] Preferably, in one embodiment of the present invention, the method for obtaining the real swing angle of the bronchoscope specifically includes:
[0117] The angle between the first straight line and the second straight line is normalized to obtain the correction coefficient of the bronchoscope, wherein the value range of the angle between the first straight line and the second straight line is , the unit is degree; use the correction coefficient to weight the initial swing angle to obtain the angle adjustment amount of the bronchoscope; the sum of the initial swing angle and the angle adjustment amount is taken as the real swing angle of the bronchoscope.
[0118] As an example, in one embodiment of the present invention, the expression of the actual swing angle of the bronchoscope may be specifically, for example, as follows:
[0119]
[0120] in, Indicates the actual swing angle of the bronchoscope; Indicates the initial swing angle of the bronchoscope; represents the angle between the first straight line and the second straight line; represents the correction factor of the bronchoscope; Indicates the amount of angle adjustment of the bronchoscope; Represents the hyperbolic tangent function, which is used for normalization.
[0121] At this point, the actual swing angle of the bronchoscope before advancement was obtained.
[0122] Step S4: Navigate the bronchoscope according to the actual swing angle and the number of actual lesion areas.
[0123] When adjusting the placement direction of the bronchoscope before advancement with the real swing angle, it is also necessary to consider whether the bronchoscope needs to be withdrawn from the bronchus. While controlling the bronchoscope with the real swing angle, when the real swing angle is too large, the bronchoscope needs to be withdrawn in time to prevent the bronchoscope from touching the inner wall of the bronchus after adjustment at a larger angle and causing damage to it. At the same time, the more the number of real lesion areas, the more lesion areas there are on the inner wall of the bronchus. At this time, for safety reasons, the bronchus should not be pushed forward any further and should be withdrawn in time. Therefore, the real swing angle and the number of real lesion areas can be combined to accurately navigate the bronchoscope.
[0124] Preferably, in one embodiment of the present invention, the method for navigating a bronchoscope specifically includes:
[0125] The actual swing angle and the number of actual lesion areas are integrated and normalized to obtain the possibility of bronchoscope withdrawal. The greater the withdrawal possibility, the more necessary it is to withdraw the bronchoscope from the bronchus in a timely manner.
[0126] In the embodiment of the present invention, the integration of the real swing angle and the number of real lesion areas can be achieved by calculating the sum or product of the two, which is not limited here.
[0127] As an example, in one embodiment of the present invention, the expression for the possibility of bronchoscope withdrawal may be specifically, for example, as follows:
[0128]
[0129] in, Indicates the possibility of bronchoscope withdrawal; Indicates the actual swing angle of the bronchoscope; represents the number of true lesion areas; Represents the hyperbolic tangent function, which is used for normalization.
[0130] If the possibility of withdrawal is not greater than the preset withdrawal threshold, it means that the bronchoscope can continue to be pushed forward at this time, then the bronchoscope is adjusted along the direction of the common fitting circle center pointed by the outermost pixel point with the real swing angle, and the bronchoscope is pushed forward with the preset moving distance, wherein the direction in which the outermost pixel point points to the common fitting circle center is the direction away from the nearest lesion area, and adjusting the angle of the bronchus along this direction can prevent the bronchoscope from touching the nearest lesion area, wherein the preset moving distance has a value range of 1 to 5 mm. In one embodiment of the present invention, the preset moving distance is set to 2 mm, and the specific value of the preset moving distance can also be set by the implementer according to the specific implementation scenario, which is not limited here.
[0131] If the possibility of withdrawal is greater than the preset withdrawal threshold, it means that the bronchoscope cannot be further advanced at this time. In this case, the bronchoscope needs to be withdrawn from the bronchus to avoid secondary injury to the patient.
[0132] The preset withdrawal threshold value range is In one embodiment of the present invention, the preset withdrawal threshold is set to 0.8. The specific value of the preset withdrawal threshold can also be set by the implementer according to the specific implementation scenario, which is not limited here.
[0133] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0134] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.
Claims
1. A bronchoscopic assisted navigation method for respiratory infectious diseases, It is characterized in that The method comprises: Obtaining a grayscale image of the bronchus before the bronchoscope is moved, clustering the pixels in the grayscale image, and obtaining a plurality of segmented regions; Perform arc fitting on the pixel points in each segmented area to obtain a fitting arc, a fitting center of the fitting arc, and a fitting error; filter out the tracheal cartilage ring area from all segmented areas according to the change in the grayscale value of the pixel points on both sides of the fitting arc in each segmented area and the fitting error; take the segmented areas other than the tracheal cartilage ring area and adjacent to the tracheal cartilage ring area as suspected lesion areas, and filter out the real lesion area from all suspected lesion areas according to the distribution of the grayscale values of the pixel points in each suspected lesion area and the distance between the fitting centers of the fitting arcs of the two tracheal cartilage ring areas adjacent to each suspected lesion area; The nearest lesion area is selected from all the real lesion areas; the midpoint of the line between the fitting centers of the fitting arcs of two tracheal cartilage ring areas adjacent to the nearest lesion area is used as the common fitting center; according to the distance between the edge pixel point of the nearest lesion area and the common fitting center, and the number of pixel points in the nearest lesion area, the first straight line of the nearest lesion area and the initial swing angle of the bronchoscope are obtained; the fitting centers of the fitting arcs of all tracheal cartilage ring areas are fitted with a straight line to obtain a second straight line; according to the angle between the first straight line and the second straight line, the initial swing angle is adjusted to obtain the real swing angle of the bronchoscope; Navigating the bronchoscope according to the real swing angle and the number of real lesion areas; The method of screening out the tracheal cartilage ring region from all segmented regions comprises: Taking any segmentation area as a target segmentation area, taking any pixel point on the fitting arc of the target segmentation area as a target pixel point, and taking the line connecting the two intersection points of the normal line of the fitting arc of the target segmentation area at the target pixel point and the edge of the target segmentation area as a reference line segment of the target pixel point; taking the target pixel point as the segmentation point, dividing the reference line segment into a first line segment and a second line segment; Taking any one of the first line segment and the second line segment as the target line segment, starting from the end point of the target line segment at the edge of the target segmentation area, along the target line segment, calculating the difference between the grayscale value of each pixel point on the target line segment and the adjacent next pixel point, and obtaining the grayscale change of each pixel point on the target line segment; taking the accumulated value of the grayscale change of all pixels on the target line segment as the grayscale gradient value of the target line segment; taking the sum of the grayscale gradient values of the first line segment and the second line segment as the grayscale change characteristic value of the target pixel point; The accumulated value of the grayscale change characteristic values of all the pixels on the fitting arc of the target segmentation region is used as the comprehensive change characteristic value of the target segmentation region; Perform negative correlation mapping on the fitting error of the fitting arc of the target segmentation area to obtain the arc similarity of the target segmentation area; The comprehensive change characteristic value and the arc similarity are integrated and normalized to obtain the possibility of the tracheal cartilage ring in the target segmentation area; The segmented area whose tracheal cartilage ring possibility is greater than a preset first threshold is used as a tracheal cartilage ring area; The step of obtaining the first straight line of the nearest lesion area and the initial swing angle of the bronchoscope comprises: The edge pixel point corresponding to the minimum value of the distance between the most adjacent lesion area and the center of the common fitting circle is used as the outermost pixel point of the most adjacent lesion area; The straight line where the outermost pixel point and the common fitting circle center are located is taken as the first straight line; Performing negative correlation mapping on the distance between the outermost pixel point and the common fitting circle center to obtain the proximity degree of the nearest lesion area; The number of pixels in the most adjacent lesion area and the proximity are integrated and normalized to obtain the adjustment degree of the bronchoscope; The preset maximum angle is weighted using the adjustment degree to obtain the initial swing angle of the bronchoscope.
2. A bronchoscopic assisted navigation method for respiratory infectious diseases according to claim 1, It is characterized in that The obtaining of the fitting arc and the fitting center and fitting error of the fitting arc comprises: For any segmented area, the least square method is used to perform arc fitting on the pixel points in the segmented area to obtain the fitting arc and the fitting error of the fitting arc; The intersection points of the normal lines at each pixel point on the fitting arc are obtained, and the intersection point with the largest number of normal lines passing through is used as the fitting center of the fitting arc.
3. A bronchoscopic assisted navigation method for respiratory infectious diseases according to claim 1, It is characterized in that The method of screening out the real lesion area from all the suspected lesion areas comprises: Taking any suspected lesion area as the target suspected lesion area, taking any pixel point in the target suspected lesion area as the pixel point to be tested, analyzing the discrete degree of the grayscale value of each pixel point in a preset neighborhood centered on the pixel point to be tested, and obtaining the local grayscale disorder degree of the pixel point to be tested; The local grayscale disorder of all pixels in the target suspected lesion area is integrated to obtain the integrated grayscale disorder of the target suspected lesion area; Negative correlation mapping is performed on the distance between the centers of the fitted arcs of two tracheal cartilage ring regions adjacent to the target suspected lesion region to obtain the distance characteristic value of the target suspected lesion region; The comprehensive grayscale chaos degree and the distance characteristic value are integrated and normalized to obtain the lesion possibility of the target suspected lesion area; Based on the lesion possibility of each suspected lesion area, the real lesion area is screened out from all the suspected lesion areas.
4. A bronchoscopic assisted navigation method for respiratory infectious diseases according to claim 1, It is characterized in that The selecting the closest lesion region from all real lesion regions comprises: The tracheal cartilage ring area with the largest number of pixels is taken as the nearest cartilage ring area; The nearest lesion region is selected from all real lesion regions, and the distance between the center of mass of the nearest lesion region and the nearest cartilage ring region is the smallest.
5. A bronchoscopic assisted navigation method for respiratory infectious diseases according to claim 1, It is characterized in that The method of obtaining the true swing angle of the bronchoscope comprises: Normalizing the angle between the first straight line and the second straight line to obtain a correction coefficient of the bronchoscope; Using the correction coefficient, weighting the initial swing angle to obtain an angle adjustment amount of the bronchoscope; The sum of the initial swing angle and the angle adjustment amount is taken as the actual swing angle of the bronchoscope.
6. A bronchoscopic assisted navigation method for respiratory infectious diseases according to claim 1, It is characterized in that The navigating the bronchoscope according to the real swing angle and the number of real lesion areas includes: The actual swing angle and the number of actual lesion areas are integrated and normalized to obtain the possibility of bronchoscope withdrawal; If the withdrawal possibility is not greater than a preset withdrawal threshold, the bronchoscope is adjusted along the direction of the outermost pixel point pointing to the common fitting circle center at the real swing angle, and the bronchoscope is pushed forward at a preset moving distance; If the withdrawal possibility is greater than a preset withdrawal threshold, the bronchoscope is withdrawn from the bronchus.
7. A bronchoscopic assisted navigation method for respiratory infectious diseases according to claim 3, It is characterized in that The screening out of the real lesion area from all the suspected lesion areas based on the lesion possibility of each suspected lesion area comprises: The suspected lesion area whose lesion possibility is greater than a preset second threshold is regarded as a real lesion area.
8. A bronchoscopic assisted navigation method for respiratory infectious diseases according to claim 6, It is characterized in that The value range of the preset withdrawal threshold is .
Citation Information
Patent Citations
Human body cavity navigation method and system based on endoscope positioning
CN116236146A
Lung rehabilitation intelligent auxiliary system based on medical image analysis
CN117409004A
Bronchoscope image auxiliary optimization system based on machine learning
CN117522719A
Image processing apparatus, image processing program product, and image processing method
US20100092055A1
Method for segmenting medical images and detecting surface anomalies in anatomical structures
US6246784B1
