An intelligent navigation method for electronic bronchoscope

Abstract

The present invention relates to the field of image processing technology, and in particular to an intelligent navigation method for an electronic bronchoscope, comprising: obtaining two adjacent frames of bronchoscope grayscale images; obtaining a similar neighborhood window corresponding to each neighborhood window in a target grayscale image; obtaining the shadow expression degree of a pixel point; obtaining the mutation distance consistency of a pixel point; obtaining the mutation grayscale consistency of a pixel point; obtaining the modified shadow expression degree of a pixel point; using the modified shadow expression degree of a pixel point in a neighborhood window and a similar neighborhood window, modifying the grayscale value of a pixel point in a target grayscale image to obtain an enhanced bronchoscope grayscale image; and performing intelligent navigation according to the enhanced bronchoscope grayscale image. The present invention improves the navigation accuracy of a bronchoscope.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an intelligent navigation method for an electronic bronchoscope. Background Art

[0002] Electronic bronchoscopes are used to examine bronchial and lung lesions, while conventional bronchoscopes are difficult to reach the peripheral areas of the lungs. Therefore, electromagnetic positioning technology is used in medicine during electronic bronchoscope examinations to navigate to the peripheral pulmonary lesions in real time. During the navigation process, the images obtained by the bronchoscope are acquired in real time. However, the environment within the bronchus is complex and varies from person to person. The wrinkles in the bronchial wall under illumination can produce shadows, leading to navigation errors, misoperation, and even complications. Therefore, the images obtained by the bronchoscope need to be enhanced. Summary of the invention

[0003] In order to solve the above problems, the present invention provides an intelligent navigation method for an electronic bronchoscope, the method comprising:

[0004] Acquire two adjacent frames of bronchoscopic grayscale images; record any one frame of bronchoscopic grayscale image as a target grayscale image; record another frame of bronchoscopic grayscale image as a reference grayscale image;

[0005] The target grayscale image and the reference grayscale image are equally divided into a number of neighborhood windows; according to the grayscale distribution difference of the pixels between the neighborhood windows in the target grayscale image and the reference grayscale image, a similar neighborhood window corresponding to each neighborhood window in the target grayscale image is obtained in the reference grayscale image;

[0006] For any pixel in the target grayscale image, obtain the forward direction, mutation pixel and mutation distance of the pixel; in the forward direction of the pixel, according to the grayscale mutation of the mutation pixel of the pixel, obtain the shadow expression degree of the pixel; obtain the forward reference direction and mutation reference pixel of the pixel; in the forward reference direction of the pixel, according to the shadow expression degree of the pixel and the distance between the mutation pixels, obtain the mutation distance consistency of the pixel; according to the shadow expression degree of the mutation reference pixel of the pixel, obtain the mutation grayscale consistency of the pixel; correct the shadow expression degree according to the mutation grayscale consistency and mutation distance consistency of the pixel, and obtain the corrected shadow expression degree of the pixel;

[0007] The grayscale values ​​of the pixels in the target grayscale image are corrected by utilizing the modified shadow representation of the pixels in the neighborhood window and the similar neighborhood window to obtain an enhanced tracheal endoscope grayscale image; intelligent navigation is performed based on the enhanced tracheal endoscope grayscale image.

[0008] Preferably, the method of obtaining, in the reference grayscale image, a similar neighborhood window corresponding to each neighborhood window in the target grayscale image according to the difference in grayscale distribution of pixels between the neighborhood windows in the target grayscale image and the reference grayscale image comprises the following specific methods:

[0009] Any neighborhood window in the target grayscale image is recorded as the target neighborhood window; any neighborhood window in the reference grayscale image is recorded as the reference neighborhood window;

[0010] According to the grayscale distribution difference of the pixels between the target neighborhood window and the reference neighborhood window, the similarity between the target neighborhood window and the reference neighborhood window is obtained;

[0011] Among all the neighborhood windows in the reference grayscale image, the neighborhood window with the greatest similarity to the target neighborhood window is used as the similar neighborhood window of the target neighborhood window.

[0012] Preferably, the obtaining of the similarity between the target neighborhood window and the reference neighborhood window comprises the following specific methods:

[0013] According to the mean, variance and covariance of the gray values ​​of all pixels in the target neighborhood window and the reference neighborhood window, the structural similarity between the target neighborhood window and the reference neighborhood window is obtained;

[0014] According to the difference in grayscale values ​​of pixels between the target neighborhood window and the reference neighborhood window, the grayscale difference between the target neighborhood window and the reference neighborhood window is obtained;

[0015] The product of the structural similarity and the grayscale difference between the target neighborhood window and the reference neighborhood window is taken as the similarity between the target neighborhood window and the reference neighborhood window.

[0016] Preferably, the specific method of obtaining the moving direction of the pixel point, the mutation pixel point and the mutation distance includes:

[0017] For any neighborhood window in the target grayscale image, a straight line in any direction is drawn with the central pixel point of the neighborhood window as the starting point; for a straight line in any direction, if the mean value of the grayscale values ​​of all the pixels on the straight line in the direction is the minimum value among all the straight lines in the directions, the direction is taken as the bronchial advancing direction of the neighborhood window;

[0018] A clustering parameter a and a sequence parameter b are preset, and the pixel with the smallest gray value in the bronchial forward direction of each neighborhood window is recorded as the forward pixel of the neighborhood window; the forward pixel of all neighborhood windows in the target gray image is clustered to obtain a cluster, and the cluster center of the cluster is used as the forward path center of the target gray image;

[0019] For any pixel point in the target grayscale image, take the pixel point as the starting point, and record the direction of the line between the pixel point and the center of the forward path of the target grayscale image as the forward direction of the pixel point; in the forward direction of the pixel point, record the pixel point sequence composed of the first b pixel points and the last b pixel points of the pixel point as the grayscale change pixel point sequence of the pixel point; obtain the maximum value of the absolute value of the difference in grayscale values ​​between all any two pixel points in the grayscale change pixel point sequence of the pixel point, and use the two pixel points corresponding to the maximum value as the mutation pixel points of the pixel point; use the Euclidean distance between the two mutation pixel points as the mutation distance of the pixel point.

[0020] Preferably, the shadow expression degree of the pixel point is obtained according to the grayscale mutation of the pixel point in the forward direction of the pixel point, and the specific method includes:

[0021] For any pixel in the target grayscale image, any mutation pixel of the pixel is recorded as a first mutation pixel; another mutation pixel of the pixel is recorded as a second mutation pixel; in the forward direction of the pixel, the absolute value of the difference in grayscale value between the first mutation pixel and its adjacent pixel is recorded as a first difference absolute value; the absolute value of the difference in grayscale value between the second mutation pixel and its adjacent pixel is recorded as a second difference absolute value; and the normalized value of the sum of the first difference absolute value and the second difference absolute value is taken as the shadow expression degree of the pixel.

[0022] Preferably, the obtaining of the forward reference direction of the pixel point and the sudden change reference pixel point comprises the following specific methods:

[0023] For any pixel point in the target grayscale image, the direction perpendicular to the forward direction of the pixel point is recorded as the forward reference direction of the pixel point; in the forward reference direction of the pixel point, its two adjacent pixel points are recorded as the mutation reference pixel points of the pixel point.

[0024] Preferably, the method of obtaining the consistency of the mutation distance of the pixel points according to the shadow performance degree of the pixel points and the distance between the mutation pixels in the forward reference direction of the pixel points includes the following specific methods:

[0025] For any pixel point in the target grayscale image, any mutation reference pixel point of the pixel point is recorded as a first mutation reference pixel point; another mutation reference pixel point of the pixel point is recorded as a second mutation reference pixel point;

[0026] According to the shadow expression degree and the mutation distance, the difference in mutation distance between the first mutation reference pixel and the pixel point is obtained;

[0027] The reciprocal of the sum of the difference in mutation distance between the first mutation reference pixel and the pixel and the difference in mutation distance between the second mutation reference pixel and the pixel is taken as the mutation distance consistency of the pixel.

[0028] Preferably, the obtaining of the difference in mutation distance between the first mutation reference pixel and the pixel point comprises the following specific methods:

[0029] The sum of the shadow expression degree of the first mutation reference pixel and the shadow expression degree of the first mutation reference pixel is recorded as the first sum value; the ratio of the shadow expression degree of the first mutation reference pixel to the first sum value is recorded as the first ratio; the absolute value of the difference between the mutation distance of the first mutation reference pixel and the mutation distance of the pixel point is recorded as the mutation difference absolute value; the product of the first ratio and the mutation difference absolute value is taken as the mutation distance difference between the first mutation reference pixel and the pixel point.

[0030] Preferably, the step of obtaining the grayscale consistency of the pixel mutation according to the shadow expression degree of the pixel mutation reference pixel includes the following specific methods:

[0031] For any pixel point in the target grayscale image, any mutation reference pixel point of the pixel point is recorded as a first mutation reference pixel point; another mutation reference pixel point of the pixel point is recorded as a second mutation reference pixel point;

[0032] The absolute value of the difference between the shadow expression degree of the first mutation reference pixel and the shadow expression degree of the pixel is recorded as the first grayscale difference absolute value; the absolute value of the difference between the shadow expression degree of the second mutation reference pixel and the shadow expression degree of the pixel is recorded as the second grayscale difference absolute value; the reciprocal of the sum of the first grayscale difference absolute value and the second grayscale difference absolute value is taken as the mutation grayscale consistency of the pixel.

[0033] Preferably, the shadow expression degree is corrected according to the mutation gray consistency and mutation distance consistency of the pixel point to obtain the corrected shadow expression degree of the pixel point, including the specific method of:

[0034] For any pixel in the target grayscale image, the sum of the mutation grayscale consistency and the mutation distance consistency of the pixel is recorded as the correction factor; the normalized value of the product of the correction factor and the shadow expression degree of the pixel is used as the corrected shadow expression degree of the pixel.

[0035] The beneficial effects of the technical solution of the present invention are as follows: in the forward direction of the pixel point, the present invention obtains the shadow expression degree of the pixel point according to the grayscale mutation of the mutation pixel point of the pixel point; obtains the forward reference direction of the pixel point and the mutation reference pixel point; in the forward reference direction of the pixel point, according to the shadow expression degree of the pixel point and the distance between the mutation pixels, obtains the mutation distance consistency of the pixel point; according to the shadow expression degree of the mutation reference pixel point of the pixel point, obtains the mutation grayscale consistency of the pixel point; uses the corrected shadow expression degree of the pixel point in the neighborhood window and the similar neighborhood window to correct the grayscale value of the pixel point in the target grayscale image to obtain an enhanced tracheoendoscopic grayscale image; performs intelligent navigation based on the enhanced tracheoendoscopic grayscale image; thereby making the tracheoendoscopic grayscale image clearer and reducing the shadow blind area; thereby improving the navigation accuracy of the bronchial endoscope. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 A flowchart of the steps of an intelligent navigation method for an electronic bronchoscope according to the present invention;

[0038] Figure 2 The present invention is a characteristic relationship flow chart of an intelligent navigation method for an electronic bronchoscope. DETAILED DESCRIPTION

[0039] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the specific implementation, structure, features and effects of an intelligent navigation method for an electronic bronchoscope proposed according to the present invention, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0040] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0041] The specific scheme of the intelligent navigation method for electronic bronchoscope provided by the present invention is described in detail below with reference to the accompanying drawings.

[0042] See also Figure 1 , which shows a flowchart of a method for intelligent navigation of an electronic bronchoscope provided by an embodiment of the present invention, the method comprising the following steps:

[0043] Step S001: Acquire two adjacent frames of bronchial endoscope grayscale images.

[0044] Specifically, it is first necessary to collect two adjacent frames of bronchial endoscope grayscale images. The specific process is as follows:

[0045] The endoscope is inserted into the bronchus at a speed of 1 mm per second, and two adjacent frames of bronchial endoscope images are obtained at a frame rate of 30 fps; the two adjacent frames of bronchial endoscope images are subjected to median filtering, denoising and grayscale operations to obtain two adjacent frames of bronchial endoscope grayscale images. The median filtering and grayscale operations are prior arts, and are not described in detail in this embodiment.

[0046] So far, two adjacent frames of bronchial endoscope grayscale images are obtained through the above method.

[0047] Step S002: equally divide the target grayscale image and the reference grayscale image into a number of neighborhood windows; based on the difference in grayscale distribution of pixels between the neighborhood windows in the target grayscale image and the reference grayscale image, obtain in the reference grayscale image a similar neighborhood window corresponding to each neighborhood window in the target grayscale image.

[0048] It should be noted that, due to the different positions of the endoscope, the grayscale images of two adjacent frames of bronchoendoscopy are not completely consistent. Even the positions of windows showing the same image features are different. Therefore, it is necessary to first determine the positions of windows with the same features on different images. Windows with the same features are consistent in grayscale and structural characteristics formed with adjacent pixels. Therefore, the corresponding similar windows on two adjacent frames of bronchoendoscopy grayscale images can be determined based on the relationship between regional grayscale and structure.

[0049] Preferably, any frame of bronchoscopic grayscale image is recorded as a target grayscale image; another frame of bronchoscopic grayscale image is recorded as a reference grayscale image;

[0050] A window parameter n is preset, wherein this embodiment is described by taking n=5 as an example, and this embodiment is not specifically limited, wherein n is determined according to specific implementation conditions.

[0051] The target grayscale image and the reference grayscale image are equally divided into several neighborhood windows with a window size of n×n.

[0052] Preferably, in some implementations of the embodiments of the present invention, according to the grayscale distribution difference of pixels between the neighborhood windows in the target grayscale image and the reference grayscale image, a specific method for obtaining a similar neighborhood window corresponding to each neighborhood window in the target grayscale image in the reference grayscale image is:

[0053] Any neighborhood window in the target grayscale image is recorded as the target neighborhood window; any neighborhood window in the reference grayscale image is recorded as the reference neighborhood window;

[0054] According to the grayscale distribution difference of the pixels between the target neighborhood window and the reference neighborhood window, the similarity between the target neighborhood window and the reference neighborhood window is obtained;

[0055] Among all the neighborhood windows in the reference grayscale image, the neighborhood window with the greatest similarity to the target neighborhood window is used as the similar neighborhood window of the target neighborhood window.

[0056] Preferably, in some implementations of the embodiments of the present invention, the method for obtaining the similarity between the target neighborhood window and the reference neighborhood window includes:

[0057] According to the mean, variance and covariance of the gray values ​​of all pixels in the target neighborhood window and the reference neighborhood window, the structural similarity between the target neighborhood window and the reference neighborhood window is obtained;

[0058] According to the difference in grayscale values ​​of pixels between the target neighborhood window and the reference neighborhood window, the grayscale difference between the target neighborhood window and the reference neighborhood window is obtained;

[0059] The product of the structural similarity and the grayscale difference between the target neighborhood window and the reference neighborhood window is taken as the similarity between the target neighborhood window and the reference neighborhood window.

[0060] The specific formula is:

[0061]

[0062] Where E represents the similarity between the target neighborhood window and the reference neighborhood window; μmc represents the mean of the grayscale values ​​of all pixels in the target neighborhood window and the reference neighborhood window; μm represents the mean of the grayscale values ​​of all pixels in the target neighborhood window; μc represents the mean of the grayscale values ​​of all pixels in the reference neighborhood window; σmc represents the covariance of the grayscale values ​​of the pixels between the target neighborhood window and the reference neighborhood window; σm represents the variance of the grayscale values ​​of all pixels in the target neighborhood window; σc represents the variance of the grayscale values ​​of all pixels in the reference neighborhood window; HC represents the grayscale difference between the target neighborhood window and the reference neighborhood window; qm i Represents the gray value of the i-th pixel in the target neighborhood window; qci represents the gray value of the i-th pixel in the reference neighborhood window; || represents the absolute value; n is the preset window parameter; ω represents the preset hyperparameter, and this implementation presets ε=1 to prevent the denominator from being 0.

[0063] It should be noted that the greater the structural similarity between the target neighborhood window and the reference neighborhood window, the greater the similarity between the target neighborhood window and the reference neighborhood window; the smaller the grayscale difference between the target neighborhood window and the reference neighborhood window, the greater the similarity between the target neighborhood window and the reference neighborhood window.

[0064] At this point, the similar neighborhood window corresponding to each neighborhood window in the target grayscale image is obtained in the reference grayscale image through the above method.

[0065] Step S003: for any pixel in the target grayscale image, obtain the forward direction, mutation pixel and mutation distance of the pixel; in the forward direction of the pixel, obtain the shadow expression degree of the pixel according to the grayscale mutation of the mutation pixel of the pixel; obtain the forward reference direction and mutation reference pixel of the pixel; in the forward reference direction of the pixel, obtain the mutation distance consistency of the pixel according to the shadow expression degree of the pixel and the distance between the mutation pixels; obtain the mutation grayscale consistency of the pixel according to the shadow expression degree of the mutation reference pixel of the pixel; correct the shadow expression degree according to the mutation grayscale consistency and mutation distance consistency of the pixel, and obtain the corrected shadow expression degree of the pixel.

[0066] It should be noted that the positions at which the endoscope acquires two adjacent frames of bronchoscopic grayscale images are different, so the shadows formed by the light source on the wrinkle area are different. Therefore, when the matching points are fused, it is necessary to determine their fusion weights based on their shadow performance in the two adjacent frames of bronchoscopic grayscale images. The wrinkles in the bronchi are mainly distributed in the circumferential direction of the bronchi, and the shadows formed are also in the circumferential direction. Moreover, the shadows in the circumferential direction form a grayscale mutation with the normal bronchial wall in the forward direction of the bronchus, and also form a grayscale mutation with the wrinkle area.

[0067] 1. Get the shadow performance of the pixel.

[0068] It should be noted that, in the direction of the bronchus, as the distance from the light source becomes farther and farther, the path becomes darker and the grayscale gradually decreases; therefore, the direction of the bronchus can be determined based on the direction in which the grayscale decreases.

[0069] For any neighborhood window in the target grayscale image, a straight line in any direction is drawn with the central pixel point of the neighborhood window as the starting point; for a straight line in any direction, if the mean value of the grayscale values ​​of all the pixels on the straight line in the direction is the minimum value among all the straight lines in the directions, the direction is taken as the bronchial advancing direction of the neighborhood window;

[0070] A clustering parameter a and a sequence parameter b are preset, wherein this embodiment is described by taking a=1, b=20 as an example, and this embodiment is not specifically limited, wherein a and b are determined according to specific implementation conditions.

[0071] The pixel with the smallest gray value in the bronchial forward direction of each neighborhood window is recorded as the forward pixel of the neighborhood window; the forward pixel of all neighborhood windows in the target gray image is clustered using the k-means clustering algorithm with a k value of a to obtain a cluster cluster, and the cluster center of the cluster cluster is used as the forward path center of the target gray image;

[0072] For any pixel point in the target grayscale image, take the pixel point as the starting point, and record the direction of the line between the pixel point and the center of the forward path of the target grayscale image as the forward direction of the pixel point; in the forward direction of the pixel point, record the pixel point sequence composed of the first b pixel points and the last b pixel points of the pixel point as the grayscale change pixel point sequence of the pixel point; obtain the maximum value of the absolute value of the difference in grayscale values ​​between all any two pixel points in the grayscale change pixel point sequence of the pixel point, and use the two pixel points corresponding to the maximum value as the mutation pixel points of the pixel point; use the Euclidean distance between the two mutation pixel points as the mutation distance of the pixel point.

[0073] At this point, the moving direction of the pixel point, the mutation pixel point and the mutation distance are obtained.

[0074] It should be noted that, in the advancing direction of the pixel, the more obvious the grayscale mutation is, the stronger the contrast between the shadow and the normal area is, the deeper the shadow is, and therefore the more obvious the shadow of the pixel is.

[0075] Preferably, in some implementations of the embodiments of the present invention, in the forward direction of each pixel, according to the grayscale mutation of the mutation pixel of the pixel, the specific method for obtaining the shadow expression degree of each pixel is:

[0076] For any pixel in the target grayscale image, any mutation pixel of the pixel is recorded as a first mutation pixel; another mutation pixel of the pixel is recorded as a second mutation pixel; in the forward direction of the pixel, the absolute value of the difference in grayscale value between the first mutation pixel and its adjacent pixel is recorded as a first difference absolute value; the absolute value of the difference in grayscale value between the second mutation pixel and its adjacent pixel is recorded as a second difference absolute value; and the normalized value of the sum of the first difference absolute value and the second difference absolute value is taken as the shadow expression degree of the pixel.

[0077] The specific formula is:

[0078] P=norm(|q1-q1′|+|q2-q2′|)

[0079] Wherein, P represents the shadow performance degree of the pixel; q1 represents the gray value of the first mutation pixel; q2 represents the gray value of the second mutation pixel; q1' represents the gray value of the adjacent pixel of the first mutation pixel in the forward direction of the pixel; q2' represents the gray value of the adjacent pixel of the second mutation pixel in the forward direction of the pixel; || represents taking the absolute value; norm() represents the linear normalization function.

[0080] It should be noted that the larger the value of |q1-q1'| is, the more obvious the grayscale mutation of the pixel point is, and thus the more obvious the shadow of the pixel point is.

[0081] At this point, the shadow performance level of the pixel is obtained.

[0082] 2. Obtain the consistency of the mutation distance of pixels.

[0083] It should be noted that the consistency of mutation distance is reflected in the consistency of the distance between the mutation pixel and the adjacent pixel in the advancing direction. The closer the distance between the mutation pixels is, the stronger the consistency of the mutation situation is. At the same time, the grayscale mutation situation should also be consistent. The more consistent the grayscale change of the mutation pixel is, the stronger the consistency of the mutation situation is.

[0084] For any pixel point in the target grayscale image, the direction perpendicular to the forward direction of the pixel point is recorded as the forward reference direction of the pixel point; in the forward reference direction of the pixel point, its two adjacent pixel points are recorded as the mutation reference pixel points of the pixel point.

[0085] Preferably, in some implementations of the embodiments of the present invention, in the forward reference direction of each pixel point, according to the shadow performance degree of the pixel point and the distance between the sudden pixel points, the specific method for obtaining the consistency of the sudden distance of each pixel point is:

[0086] For any pixel point in the target grayscale image, any mutation reference pixel point of the pixel point is recorded as a first mutation reference pixel point; another mutation reference pixel point of the pixel point is recorded as a second mutation reference pixel point;

[0087] According to the shadow expression degree and the mutation distance, the difference in mutation distance between the first mutation reference pixel and the pixel point is obtained;

[0088] The reciprocal of the sum of the difference in mutation distance between the first mutation reference pixel and the pixel and the difference in mutation distance between the second mutation reference pixel and the pixel is taken as the mutation distance consistency of the pixel.

[0089] The specific formula is:

[0090]

[0091] Wherein, A represents the consistency of the mutation distance of the pixel point; M1 represents the difference in the mutation distance between the first mutation reference pixel point and the pixel point; M2 represents the difference in the mutation distance between the second mutation reference pixel point and the pixel point; ε represents a preset hyperparameter. In this implementation, ε=1 is preset to prevent the denominator from being 0.

[0092] Preferably, in some implementations of the embodiments of the present invention, the method for obtaining the difference in mutation distance between the first mutation reference pixel and the pixel point includes:

[0093] The sum of the shadow expression degree of the first mutation reference pixel and the shadow expression degree of the first mutation reference pixel is recorded as the first sum value; the ratio of the shadow expression degree of the first mutation reference pixel to the first sum value is recorded as the first ratio; the absolute value of the difference between the mutation distance of the first mutation reference pixel and the mutation distance of the pixel point is recorded as the mutation difference absolute value; the product of the first ratio and the mutation difference absolute value is taken as the mutation distance difference between the first mutation reference pixel and the pixel point.

[0094] The specific formula is:

[0095]

[0096] Wherein, M1 represents the difference in mutation distance between the first mutation reference pixel and the pixel; P1 represents the shadow performance degree of the first mutation reference pixel; P2 represents the shadow performance degree of the second mutation reference pixel; t represents the mutation distance of the pixel; t1 represents the mutation distance of the first mutation reference pixel; || represents taking the absolute value.

[0097] At this point, the consistency of the mutation distance of the pixels is obtained.

[0098] 3. Obtain the mutation grayscale consistency of pixels.

[0099] It should be noted that the smaller the difference in mutation distance between a pixel and its adjacent pixels, the more consistent the mutation distance. In addition, the different shadow performances of adjacent pixels have different effects on the consistency of the mutation distance of the pixels. The stronger the shadow performance, the greater the impact on the consistency of the corresponding mutation distance.

[0100] Preferably, in some implementations of the embodiments of the present invention, the specific method for obtaining the consistency of the mutation grayscale of each pixel point according to the shadow representation degree of the mutation reference pixel point of each pixel point is:

[0101] For any pixel point in the target grayscale image, any mutation reference pixel point of the pixel point is recorded as a first mutation reference pixel point; another mutation reference pixel point of the pixel point is recorded as a second mutation reference pixel point;

[0102] The absolute value of the difference between the shadow expression degree of the first mutation reference pixel and the shadow expression degree of the pixel is recorded as the first grayscale difference absolute value; the absolute value of the difference between the shadow expression degree of the second mutation reference pixel and the shadow expression degree of the pixel is recorded as the second grayscale difference absolute value; the reciprocal of the sum of the first grayscale difference absolute value and the second grayscale difference absolute value is taken as the mutation grayscale consistency of the pixel.

[0103] The specific formula is:

[0104]

[0105] Wherein, B represents the mutation gray consistency of the pixel; P1 represents the shadow expression degree of the first mutation reference pixel; P2 represents the shadow expression degree of the second mutation reference pixel; || represents taking the absolute value; ε represents the preset hyperparameter. In this implementation, ε=1 is preset to prevent the denominator from being 0.

[0106] At this point, the mutation grayscale consistency of the pixel points is obtained.

[0107] 4. Get the corrected shadow performance degree of the pixel.

[0108] Preferably, in some implementations of the embodiments of the present invention, the shadow expression degree is corrected according to the mutation gray consistency and mutation distance consistency of each pixel point, and the specific method for obtaining the corrected shadow expression degree of each pixel point is:

[0109] For any pixel in the target grayscale image, the sum of the mutation grayscale consistency and the mutation distance consistency of the pixel is recorded as the correction factor; the normalized value of the product of the correction factor and the shadow expression degree of the pixel is used as the corrected shadow expression degree of the pixel.

[0110] The specific formula is:

[0111] P′=norm((A+B)×P)

[0112] Where P' represents the degree of corrected shadow expression of the pixel; B represents the consistency of the mutation grayscale of the pixel; A represents the consistency of the mutation distance of the pixel; P represents the degree of shadow expression of the pixel; norm() represents the linear normalization function.

[0113] So far, the modified shadow expression degree of each pixel in the target grayscale image is obtained through the above method.

[0114] Step S004: using the modified shadow expression degree of the pixels in the neighborhood window and the similar neighborhood window, the grayscale value of the pixels in the target grayscale image is modified to obtain an enhanced tracheal endoscope grayscale image; and intelligent navigation is performed based on the enhanced tracheal endoscope grayscale image.

[0115] Preferably, in some implementations of the embodiments of the present invention, the grayscale value of the pixel in the target grayscale image is corrected by using the modified shadow expression degree of the pixel in the neighborhood window and the similar neighborhood window, and the specific orientation of the enhanced tracheoendoscopy grayscale image is obtained;

[0116] For the j-th pixel point in any neighborhood window in the target grayscale image, the product of the corrected shadow expression degree and the grayscale value of the j-th pixel point in the neighborhood window is recorded as the first corrected grayscale value of the j-th pixel point; the product of the corrected shadow expression degree and the grayscale value of the j-th pixel point in a similar neighborhood window of the neighborhood window is recorded as the second corrected grayscale value of the j-th pixel point; the sum of the first corrected grayscale value and the second corrected grayscale value is taken as the enhanced grayscale value of the j-th pixel point; and the enhanced tracheal endoscope grayscale image is obtained.

[0117] The enhanced endotracheal grayscale image is transmitted to the monitor, and subsequent navigation is performed based on the monitor screen.

[0118] See also Figure 2 , which shows a characteristic relationship flow chart of an intelligent navigation method for an electronic bronchoscope.

[0119] At this point, this embodiment is completed.

[0120] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An intelligent navigation method for electronic bronchoscope, characterized in that: The method comprises the following steps: Acquire two adjacent frames of bronchoscopic grayscale images; record any one frame of bronchoscopic grayscale image as a target grayscale image; record another frame of bronchoscopic grayscale image as a reference grayscale image; The target grayscale image and the reference grayscale image are equally divided into a number of neighborhood windows; according to the grayscale distribution difference of the pixels between the neighborhood windows in the target grayscale image and the reference grayscale image, a similar neighborhood window corresponding to each neighborhood window in the target grayscale image is obtained in the reference grayscale image; For any pixel in the target grayscale image, obtain the forward direction, mutation pixel and mutation distance of the pixel; in the forward direction of the pixel, according to the grayscale mutation of the mutation pixel of the pixel, obtain the shadow expression degree of the pixel; obtain the forward reference direction and mutation reference pixel of the pixel; in the forward reference direction of the pixel, according to the shadow expression degree of the pixel and the distance between the mutation pixels, obtain the mutation distance consistency of the pixel; according to the shadow expression degree of the mutation reference pixel of the pixel, obtain the mutation grayscale consistency of the pixel; correct the shadow expression degree according to the mutation grayscale consistency and mutation distance consistency of the pixel, and obtain the corrected shadow expression degree of the pixel; The grayscale values ​​of the pixels in the target grayscale image are corrected by utilizing the modified shadow representation of the pixels in the neighborhood window and the similar neighborhood window to obtain an enhanced tracheal endoscope grayscale image; intelligent navigation is performed based on the enhanced tracheal endoscope grayscale image.

2. The intelligent navigation method for electronic bronchoscope according to claim 1, characterized in that: The method of obtaining similar neighborhood windows corresponding to each neighborhood window in the target grayscale image in the reference grayscale image according to the grayscale distribution difference of pixels between the neighborhood windows in the target grayscale image and the reference grayscale image includes the following specific methods: Any neighborhood window in the target grayscale image is recorded as the target neighborhood window; any neighborhood window in the reference grayscale image is recorded as the reference neighborhood window; According to the grayscale distribution difference of the pixels between the target neighborhood window and the reference neighborhood window, the similarity between the target neighborhood window and the reference neighborhood window is obtained; Among all the neighborhood windows in the reference grayscale image, the neighborhood window with the greatest similarity to the target neighborhood window is used as the similar neighborhood window of the target neighborhood window.

3. The intelligent navigation method for electronic bronchoscope according to claim 2, characterized in that: The specific method of obtaining the similarity between the target neighborhood window and the reference neighborhood window is as follows: According to the mean, variance and covariance of the gray values ​​of all pixels in the target neighborhood window and the reference neighborhood window, the structural similarity between the target neighborhood window and the reference neighborhood window is obtained; According to the difference in grayscale values ​​of pixels between the target neighborhood window and the reference neighborhood window, the grayscale difference between the target neighborhood window and the reference neighborhood window is obtained; The product of the structural similarity and the grayscale difference between the target neighborhood window and the reference neighborhood window is taken as the similarity between the target neighborhood window and the reference neighborhood window.

4. The intelligent navigation method for electronic bronchoscope according to claim 1, characterized in that: The specific method of obtaining the moving direction of the pixel point, the mutation pixel point and the mutation distance includes: For any neighborhood window in the target grayscale image, a straight line in any direction is drawn with the central pixel point of the neighborhood window as the starting point; for a straight line in any direction, if the mean value of the grayscale values ​​of all the pixels on the straight line in the direction is the minimum value among all the straight lines in the directions, the direction is taken as the bronchial advancing direction of the neighborhood window; A clustering parameter a and a sequence parameter b are preset, and the pixel with the smallest gray value in the bronchial advancing direction of each neighborhood window is recorded as the advancing pixel of the neighborhood window; Cluster the forward pixel points of all neighborhood windows in the target grayscale image to obtain clusters, and use the cluster center of the cluster as the forward path center of the target grayscale image; For any pixel point in the target grayscale image, take the pixel point as the starting point, and record the direction of the line between the pixel point and the center of the forward path of the target grayscale image as the forward direction of the pixel point; in the forward direction of the pixel point, record the pixel point sequence composed of the first b pixel points and the last b pixel points of the pixel point as the grayscale change pixel point sequence of the pixel point; obtain the maximum value of the absolute value of the difference in grayscale values ​​between all any two pixel points in the grayscale change pixel point sequence of the pixel point, and use the two pixel points corresponding to the maximum value as the mutation pixel points of the pixel point; use the Euclidean distance between the two mutation pixel points as the mutation distance of the pixel point.

5. The intelligent navigation method for electronic bronchoscope according to claim 1, characterized in that: The specific method of obtaining the shadow performance degree of the pixel point according to the grayscale mutation of the pixel point in the forward direction of the pixel point is as follows: For any pixel in the target grayscale image, any mutation pixel of the pixel is recorded as a first mutation pixel; another mutation pixel of the pixel is recorded as a second mutation pixel; in the forward direction of the pixel, the absolute value of the difference in grayscale value between the first mutation pixel and its adjacent pixel is recorded as a first difference absolute value; the absolute value of the difference in grayscale value between the second mutation pixel and its adjacent pixel is recorded as a second difference absolute value; and the normalized value of the sum of the first difference absolute value and the second difference absolute value is taken as the shadow expression degree of the pixel.

6. The intelligent navigation method for electronic bronchoscope according to claim 1, characterized in that: The specific method of obtaining the forward reference direction of the pixel point and the mutation reference pixel point includes: For any pixel point in the target grayscale image, the direction perpendicular to the forward direction of the pixel point is recorded as the forward reference direction of the pixel point; in the forward reference direction of the pixel point, its two adjacent pixel points are recorded as the mutation reference pixel points of the pixel point.

7. The intelligent navigation method for electronic bronchoscope according to claim 1, characterized in that: The specific method of obtaining the consistency of the mutation distance of the pixel points according to the shadow performance degree of the pixel points and the distance between the mutation pixels in the forward reference direction of the pixel points is as follows: For any pixel point in the target grayscale image, any mutation reference pixel point of the pixel point is recorded as a first mutation reference pixel point; another mutation reference pixel point of the pixel point is recorded as a second mutation reference pixel point; According to the shadow expression degree and the mutation distance, the difference in mutation distance between the first mutation reference pixel and the pixel point is obtained; The reciprocal of the sum of the difference in mutation distance between the first mutation reference pixel and the pixel and the difference in mutation distance between the second mutation reference pixel and the pixel is taken as the mutation distance consistency of the pixel.

8. The intelligent navigation method for electronic bronchoscope according to claim 7, characterized in that: The specific method of obtaining the difference in mutation distance between the first mutation reference pixel and the pixel point is as follows: The sum of the shadow expression degree of the first mutation reference pixel and the shadow expression degree of the first mutation reference pixel is recorded as the first sum value; the ratio of the shadow expression degree of the first mutation reference pixel to the first sum value is recorded as the first ratio; the absolute value of the difference between the mutation distance of the first mutation reference pixel and the mutation distance of the pixel point is recorded as the mutation difference absolute value; the product of the first ratio and the mutation difference absolute value is taken as the mutation distance difference between the first mutation reference pixel and the pixel point.

9. The intelligent navigation method for electronic bronchoscope according to claim 1, characterized in that: The specific method of obtaining the grayscale consistency of the pixel mutation according to the shadow expression degree of the pixel mutation reference pixel is as follows: For any pixel point in the target grayscale image, any mutation reference pixel point of the pixel point is recorded as a first mutation reference pixel point; another mutation reference pixel point of the pixel point is recorded as a second mutation reference pixel point; The absolute value of the difference between the shadow expression degree of the first mutation reference pixel and the shadow expression degree of the pixel is recorded as the first grayscale difference absolute value; the absolute value of the difference between the shadow expression degree of the second mutation reference pixel and the shadow expression degree of the pixel is recorded as the second grayscale difference absolute value; the reciprocal of the sum of the first grayscale difference absolute value and the second grayscale difference absolute value is taken as the mutation grayscale consistency of the pixel.

10. The intelligent navigation method for electronic bronchoscope according to claim 1, characterized in that: The method of correcting the shadow expression degree according to the mutation gray consistency and mutation distance consistency of the pixel point to obtain the corrected shadow expression degree of the pixel point includes the following specific methods: For any pixel in the target grayscale image, the sum of the mutation grayscale consistency and the mutation distance consistency of the pixel is recorded as the correction factor; the normalized value of the product of the correction factor and the shadow expression degree of the pixel is used as the corrected shadow expression degree of the pixel.

Images