Endoscope image analysis method and device with sharpness significantly improved, storage medium and equipment

Abstract

The invention relates to an endoscope image analysis method and device with sharpness significance improved, a storage medium and equipment, and the method comprises the steps: correcting a sampling pixel value of a pixel point for any pixel point of a tomographic endoscope image sequence collected by an endoscope, and obtaining an endoscope corrected image; determining tracking pixel points corresponding to the endoscope correction image, and constructing a pixel value function according to correction pixel values of the pixel points in the endoscope correction image; at the tracking pixel point, a Hessian matrix corresponding to each scale window of the pixel value function is determined through a plurality of scale windows of different sizes; according to the pixel value function and the Hessian matrix corresponding to each scale window, determining the direction and the center point of the human tissue at the tracking pixel point; and according to the trend of each tracking pixel point, connecting the center points corresponding to the tracking pixel points, and generating an endoscope target image corresponding to the human tissue. The trend and characteristics of human tissues can be completely embodied, and the definition of endoscope images is improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of image data processing, and in particular, to an endoscopic image analysis method, apparatus, storage medium and device with significantly improved clarity. Background Art

[0002] In related scenarios, the images collected by the endoscope are provided to a pre-trained neural network model for image segmentation and target recognition, and target human tissues such as blood vessels, nerves, and suspected diseased tissues that are helpful for surgery are extracted from the images. However, the structure of human vascular images is complex, and there are also problems such as uneven brightness, weak contrast, and noise interference. As a result, after the input image is segmented, there is ambiguity in the direction and features, resulting in less than ideal endoscopic image effects. Summary of the invention

[0003] The purpose of the present disclosure is to provide an endoscopic image analysis method, device, storage medium and equipment with significantly improved clarity, aiming to fully reflect the contours and features of human tissue and improve the clarity of endoscopic images.

[0004] In order to achieve the above-mentioned purpose, a first aspect of the embodiments of the present disclosure provides an endoscopic image analysis method with significantly improved clarity, comprising:

[0005] For any pixel point in the tomographic endoscopic image sequence collected by the endoscope, the sampled pixel value of the pixel point is corrected to obtain an endoscopic corrected image;

[0006] Determine the tracking pixel points corresponding to the endoscope corrected image, and construct a pixel value function with the corrected pixel values ​​of each pixel point in the endoscope corrected image;

[0007] At the tracking pixel point, using a plurality of scale windows of different sizes, respectively determining a Hessian matrix corresponding to each scale window of the pixel value function;

[0008] Determine the direction and center point of the human tissue at the tracking pixel point according to the pixel value function and the Hessian matrix corresponding to each scale window;

[0009] According to the direction of each of the tracking pixel points, the center points corresponding to the tracking pixel points are connected to generate an endoscopic target image corresponding to the human tissue.

[0010] Optionally, the step of determining the direction and center point of the human tissue at the tracking pixel point according to the pixel value function and the Hessian matrix corresponding to each of the scale windows includes:

[0011] Performing eigenvalue decomposition on the Hessian matrix corresponding to each scale window to obtain an eigenvector matrix and an eigenvalue matrix corresponding to each scale window;

[0012] Determine the absolute value of the eigenvalue of each element in the eigenvalue matrix corresponding to each of the scale windows;

[0013] The eigenvalue matrix corresponding to the eigenvalue with the largest absolute value is used as the target eigenvalue matrix, and the eigenvalue with a value close to 0 in the target eigenvalue matrix is ​​used as the target eigenvalue;

[0014] According to the correlation between the eigenvectors in the eigenvector matrix and the eigenvalues ​​in the eigenvalue matrix, taking the eigenvector corresponding to the target eigenvalue in the eigenvector matrix as the target eigenvector;

[0015] Taking the direction of the target feature vector as the direction of the human tissue at the tracking pixel point;

[0016] The center point of the human tissue at the tracking pixel point is determined according to the direction of the human tissue at the tracking pixel point and the pixel value function.

[0017] Optionally, the step of determining the center point of the human tissue at the tracking pixel point according to the direction of the human tissue at the tracking pixel point and the pixel value function includes:

[0018] A vertical plane is made through the tracking pixel points to show the direction of the human tissue;

[0019] Determining a cross-sectional image at the tracking pixel point according to an intersection of the vertical plane and the pixel value function;

[0020] The centroid of the pixel points in the cross-sectional image is determined, and the centroid is used as the center point of the human tissue at the tracking pixel point.

[0021] Optionally, the step of determining the tracking pixel points corresponding to the endoscope corrected image comprises:

[0022] Taking the center point of the human tissue corresponding to the last tracking pixel point in the tomographic endoscope image sequence as the center of the sphere, a spherical surface with a radius R is constructed;

[0023] Constructing a pixel point set according to target pixel points on both the spherical surface and the pixel value function;

[0024] According to the connected areas after the spherical surface is cut by the pixel value function, clustering and deleting duplicate points of target pixel points in the pixel point set in each connected area to obtain a unique regional pixel point in each connected area;

[0025] Determine the connectivity between the regional pixel points and the central point in each connected region, and take the regional pixel points connected to the central point by connectivity characterization as connected pixel points;

[0026] Any of the connected pixel points is taken as a target connected pixel point. In the connected area, a preset step length is advanced from the center point of the human tissue corresponding to the previous tracking pixel point toward the target connected pixel point to obtain the tracking pixel point corresponding to the next frame image in the tomographic endoscope image sequence.

[0027] Optionally, the step of taking the regional pixel points connected to the central point by the connectivity representation as the connected pixel points comprises:

[0028] Dividing the path of the endoscope into a front path and a rear path according to the center point of the human tissue corresponding to the last tracking pixel point;

[0029] using the connected area on the rear path as a backup connected area;

[0030] Taking the spare connected area on the historical path of the endoscope for image acquisition this time as the target connected area;

[0031] The pixel points in the area where the connectivity representation in the target connected area is connected to the center point are removed, and the remaining pixel points in the area where the connectivity representation is connected to the center point are taken as connected pixel points.

[0032] Optionally, the step of determining, at the tracking pixel point, a Hessian matrix corresponding to each of the scale windows of the pixel value function using a plurality of preset scale windows, comprises:

[0033] Determine a Gaussian convolution kernel corresponding to the scale window according to the number of dimensions of the tomographic endoscope image, the scale value corresponding to the scale window, and the transpose corresponding to the corrected pixel value of the tracking pixel point;

[0034] In each dimension, determining the partial derivative of the Gaussian convolution kernel corresponding to each scale window;

[0035] According to the Gaussian convolution kernel and the corresponding partial derivative, a convolution operation is performed on the product of the square of the scale value corresponding to the scale window and the pixel value function, and a Hessian matrix corresponding to the pixel value function for each scale window is determined respectively.

[0036] Optionally, the step of correcting the sampled pixel value of any pixel point in the endoscopic image to be processed collected by the endoscope to obtain the endoscopic corrected image includes:

[0037] For any pixel point in a tomographic endoscopic image sequence acquired by an endoscope, obtaining a sampled pixel value of the pixel point, a sampled pixel value of a previous pixel point adjacent to the pixel point in the tomographic endoscopic image sequence, and a sampled pixel value of a subsequent pixel point adjacent to the pixel point;

[0038] Calculating a weighted average of the sampled pixel value of the pixel point, the sampled pixel value of the previous pixel point, and the sampled pixel value of the next pixel point to obtain a corrected pixel value of the pixel point;

[0039] The endoscope corrected image is obtained according to the corrected pixel values ​​corresponding to the pixel points in the tomographic endoscope image sequence.

[0040] According to a second aspect of the embodiments of the present disclosure, there is provided an endoscopic image analysis device with significantly improved clarity, comprising:

[0041] A pixel correction module is configured to correct a sampled pixel value of any pixel point in a sequence of tomographic endoscopic images collected by the endoscope to obtain an endoscopic corrected image;

[0042] a function construction module configured to determine the tracking pixel points corresponding to the endoscope corrected image, and to construct a pixel value function with the corrected pixel values ​​of each of the pixel points in the endoscope corrected image;

[0043] A matrix determination module is configured to determine, at the tracking pixel point, a plurality of scale windows of different sizes, respectively a Hessian matrix corresponding to each scale window of the pixel value function;

[0044] A direction and center point determination module is configured to determine the direction and center point of the human tissue at the tracking pixel point according to the pixel value function and the Hessian matrix corresponding to each scale window;

[0045] The image generation module is configured to connect the center points corresponding to the tracking pixel points according to the direction of each of the tracking pixel points to generate an endoscopic target image corresponding to the human tissue.

[0046] Optionally, the direction and center point determination module includes:

[0047] A decomposition submodule is configured to perform eigenvalue decomposition on the Hessian matrix corresponding to each scale window to obtain an eigenvector matrix and an eigenvalue matrix corresponding to each scale window;

[0048] A first determination submodule is configured to determine the absolute value of the eigenvalue of each element in the eigenvalue matrix corresponding to each of the scale windows;

[0049] The second determination submodule is configured to use the eigenvalue matrix where the eigenvalue corresponding to the maximum absolute value is located as the target eigenvalue matrix, and use the eigenvalue with a value close to 0 in the target eigenvalue matrix as the target eigenvalue;

[0050] A third determination submodule is configured to use the eigenvector corresponding to the target eigenvalue in the eigenvector matrix as the target eigenvector according to the association relationship between the eigenvectors in the eigenvector matrix and the eigenvalues ​​in the eigenvalue matrix;

[0051] A fourth determination submodule is configured to use the direction of the target feature vector as the direction of the human tissue at the tracking pixel point;

[0052] The fifth determination submodule is configured to determine the center point of the human tissue at the tracking pixel point according to the direction of the human tissue at the tracking pixel point and the pixel value function.

[0053] Optionally, the fifth determining submodule is configured to:

[0054] A vertical plane is made through the tracking pixel points to show the direction of the human tissue;

[0055] Determining a cross-sectional image at the tracking pixel point according to an intersection of the vertical plane and the pixel value function;

[0056] The centroid of the pixel points in the cross-sectional image is determined, and the centroid is used as the center point of the human tissue at the tracking pixel point.

[0057] Optionally, the function building module includes:

[0058] A spherical surface construction submodule is configured to construct a spherical surface with a radius R, taking the center point of the human tissue corresponding to the last tracking pixel point in the tomographic endoscope image sequence as the center of the sphere;

[0059] A pixel point set construction submodule is configured to construct a pixel point set according to target pixel points on both the spherical surface and the pixel value function;

[0060] A clustering submodule is configured to perform clustering and duplicate point deletion on target pixel points in the pixel point set in each connected area according to the connected areas after the sphere is cut by the pixel value function, so as to obtain a unique regional pixel point in each connected area;

[0061] A sixth determination submodule is configured to determine the connectivity between the regional pixel points and the central point in each connected region, and to use the regional pixel points connected to the central point with connectivity characteristics as connected pixel points;

[0062] The seventh determination submodule is configured to take any of the connected pixel points as a target connected pixel point, and in the connected area, advance a preset step length from the center point of the human tissue corresponding to the previous tracking pixel point toward the target connected pixel point to obtain the tracking pixel point corresponding to the next frame image in the tomographic endoscope image sequence.

[0063] Optionally, the sixth determining submodule is configured to:

[0064] Dividing the path of the endoscope into a front path and a rear path according to the center point of the human tissue corresponding to the last tracking pixel point;

[0065] using the connected area on the rear path as a backup connected area;

[0066] Taking the spare connected area on the historical path of the endoscope for image acquisition this time as the target connected area;

[0067] The pixel points in the area where the connectivity representation in the target connected area is connected to the center point are removed, and the remaining pixel points in the area where the connectivity representation is connected to the center point are taken as connected pixel points.

[0068] Optionally, the matrix determination module is configured to:

[0069] Determine a Gaussian convolution kernel corresponding to the scale window according to the number of dimensions of the tomographic endoscope image, the scale value corresponding to the scale window, and the transpose corresponding to the corrected pixel value of the tracking pixel point;

[0070] In each dimension, determining the partial derivative of the Gaussian convolution kernel corresponding to each scale window;

[0071] According to the Gaussian convolution kernel and the corresponding partial derivative, a convolution operation is performed on the product of the square of the scale value corresponding to the scale window and the pixel value function, and a Hessian matrix corresponding to the pixel value function for each scale window is determined respectively.

[0072] Optionally, the pixel correction module is configured as follows:

[0073] For any pixel point in a tomographic endoscopic image sequence acquired by an endoscope, obtaining a sampled pixel value of the pixel point, a sampled pixel value of a previous pixel point adjacent to the pixel point in the tomographic endoscopic image sequence, and a sampled pixel value of a subsequent pixel point adjacent to the pixel point;

[0074] Calculating a weighted average of the sampled pixel value of the pixel point, the sampled pixel value of the previous pixel point, and the sampled pixel value of the next pixel point to obtain a corrected pixel value of the pixel point;

[0075] The endoscope corrected image is obtained according to the corrected pixel values ​​corresponding to the pixel points in the tomographic endoscope image sequence.

[0076] According to a third aspect of an embodiment of the present disclosure, a non-temporary computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a first processor, implements the steps of the endoscopic image analysis method with significantly improved clarity as described in any one of the first aspects.

[0077] According to a fourth aspect of the embodiments of the present disclosure, an electronic device is provided, including:

[0078] a first memory having a computer program stored thereon;

[0079] The second processor is used to execute the computer program in the first memory to implement the steps of the endoscopic image analysis method with significantly improved clarity as described in any one of the first aspects.

[0080] Through the above technical solution, at least the following beneficial effects can be achieved:

[0081] For any pixel point in the sequence of tomographic endoscopic images collected by the endoscope, the sampled pixel value of the pixel point is corrected to obtain an endoscopic corrected image. The correction of the sampled pixel value of the pixel point can reduce the influence of the absorption of the development enhancer by the human body and the different concentration distribution of the development enhancer on the endoscopic image to a certain extent; determine the tracking pixel point corresponding to the endoscopic corrected image, and construct a pixel value function with the corrected pixel value of each pixel point in the endoscopic corrected image; at the tracking pixel point, use multiple scale windows of different sizes to respectively determine the Hessian matrix corresponding to each scale window of the pixel value function; according to the pixel value function and the Hessian matrix corresponding to each scale window, determine the direction and center point of the human tissue at the tracking pixel point, and the Hessian matrix can be determined by scale windows of different sizes, which can overcome the influence of different human tissues and different imaging methods on the determination of the direction and center point; according to the direction of each tracking pixel point, the center points corresponding to each tracking pixel point are connected to generate an endoscopic target image corresponding to the human tissue. Connecting the center points according to the direction to form a centerline image of a blood vessel, for example, can fully reflect the direction and characteristics of the human tissue and improve the clarity of the endoscopic image. This will help doctors to observe more accurately using the endoscope.

[0082] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0084] Figure 1 The present invention is a flowchart of an endoscopic image analysis method with significantly improved clarity according to an embodiment of the present invention.

[0085] Figure 2 This is an implementation shown according to an embodiment of the present disclosure. Figure 1 Flowchart of step S13 in FIG.

[0086] Figure 3 This is an implementation shown according to an embodiment of the present disclosure. Figure 1 Flowchart of step S14 in FIG.

[0087] Figure 4 This is an implementation shown according to an embodiment of the present disclosure. Figure 1 Flowchart of step S12 in FIG.

[0088] Figure 5 The invention is a block diagram of an endoscopic image analysis device with significantly improved clarity according to an embodiment of the present invention.

[0089] Figure 6 It is a block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0090] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0091] Before introducing the endoscopic image analysis method, device, storage medium and equipment with significantly improved clarity provided by the present disclosure, the application scenarios of the present disclosure are first introduced. When a doctor uses an endoscope to collect images of tissues, blood vessels, etc. in the human body, it is necessary to clearly display the images taken by the endoscope to avoid the doctor not understanding the direction of the human tissue, the degree of the disease, etc., resulting in the endoscope being pushed into the wrong direction or tissue in the body, thereby aggravating the patient's pain.

[0092] To this end, the disclosed embodiment provides an endoscopic image analysis method with significantly improved clarity, which aims to fully reflect the direction and characteristics of human tissues and improve the clarity of endoscopic images. Furthermore, it is beneficial for doctors to operate endoscopes for observation accurately and reduce the risk of doctors pushing the endoscope into the wrong direction or tissue in the body. Figure 1 As shown, the method includes the following steps.

[0093] In step S11, for any pixel point in the tomographic endoscopic image sequence acquired by the endoscope, the sampled pixel value of the pixel point is corrected to obtain an endoscopic corrected image.

[0094] In the disclosed embodiments, the method can be applied to a neural network model, for example, a sequence of tomographic endoscopic images acquired by an endoscope is input into a pre-trained neural network model to obtain an endoscopic target image output by the neural network model.

[0095] It can be explained here that, according to different endoscopic tasks, the tomographic endoscopic image sequence acquired by the endoscope can be input into different neural network models. For example, for the vascular image construction task, the tomographic endoscopic image sequence acquired by the endoscope can be input into a pre-trained vascular neural network model, wherein the vascular neural network model is trained by vascular annotated sample images.

[0096] In the embodiment of the present disclosure, the correction of the sampled pixel value of the pixel point may be to first perform pre-processing such as cleaning on the tomographic endoscope image, so as to repair obvious imaging defects. Further, the pre-processed tomographic endoscope image is filtered and corrected by a neighborhood filtering method. In step S11, the filtering correction is performed by the neighborhood filtering method, and the sampled pixel value of the pixel point is corrected for any pixel point in the to-be-processed endoscope image collected by the endoscope, so as to obtain the endoscope corrected image, including:

[0097] For any pixel point in the tomographic endoscopic image sequence acquired by the endoscope, a sampled pixel value of the pixel point, a sampled pixel value of a previous pixel point adjacent to the pixel point in the tomographic endoscopic image sequence, and a sampled pixel value of a subsequent pixel point adjacent to the pixel point are obtained.

[0098] The weighted average of the sampled pixel value of the pixel point, the sampled pixel value of the previous pixel point, and the sampled pixel value of the next pixel point is calculated to obtain the corrected pixel value of the pixel point.

[0099] According to the corrected pixel values ​​corresponding to the pixel points in the tomographic endoscope image sequence, an endoscope corrected image is obtained.

[0100] For example, for any pixel point P n (x, y), find the pixel P in the tomographic endoscope image sequence n (x, y) The previous pixel P at the same position directly adjacent to n-1 (x, y) and the previous pixel P n+1 (x, y), and then calculate the weighted average of the corrected pixel values ​​of the three pixels to obtain the pixel point P n The corrected pixel value at (x, y).

[0101] Any pixel point P can be calculated by the following formula n Corrected pixel value P' of (x, y) n (x, y):

[0102] P'n (x, y) = α1·P n-1 (x,y)+α2·P n (x, y)+α3·P n+1 (x, y)

[0103] Among them, α1, α2, α3 are weighting coefficients, which should satisfy α1+α2+α3=1. Each weighting coefficient can be adjusted by the doctor or preset by the system. Optionally, a reference value of a set of weighting coefficients is α1=0.1, α2=0.8, α3=0.1.

[0104] Furthermore, the filtered and corrected tomographic endoscopic image can be segmented through an image segmentation network such as CNN (Convolutional Neural Network), FCN (Fully Convolutional Network), SegNet (Semantic Network), and UNet (U-Network) to obtain an endoscopic corrected image that only contains, for example, a vascular network, a neural network, and a suspected lesion area.

[0105] In one embodiment, the endoscopic corrected image containing only the vascular network, neural network, and suspected lesion area may be smoothed by filtering algorithms such as Gaussian filtering and bilateral filtering to obtain a final endoscopic corrected image.

[0106] In step S12, the tracking pixel points corresponding to the endoscope corrected image are determined, and a pixel value function is constructed using the corrected pixel values ​​of each pixel point in the endoscope corrected image.

[0107] In the disclosed embodiment, the first tracking pixel point in the endoscope corrected image can be specified by the doctor, for example, by manually inputting the coordinates of the tracking pixel point, or can be specified by the system by default. Subsequent tracking pixel points in the endoscope corrected image can be determined based on the previous tracking pixel point.

[0108] In the disclosed embodiment, the coordinates of each pixel point in the endoscope corrected image are used as the variables of the function, and the corrected pixel value of each pixel point in the endoscope corrected image is used as the solution of the function to construct a pixel value function f(x).

[0109] In step S13, at the tracking pixel point, a plurality of scale windows of different sizes are used to respectively determine the Hessian matrix corresponding to each scale window of the pixel value function.

[0110] In the embodiment of the present disclosure, the tracking pixel point is taken as the current point x0, and the Hessian matrix H(x, s)=[h i，j (x, s)],

[0111] Among them, the scale window can be centered on the tracking pixel point, covering the tracking pixel point and the pixels around the tracking pixel point, and then multiple scale windows of different sizes can cover different areas around the tracking pixel point, and then multiple scale windows of different sizes can cover different numbers of pixels.

[0112] Furthermore, the multiple scale windows of different sizes may be multiple scale windows whose sizes are in geometric progression.

[0113] Optionally, in step S13, see Figure 2 As shown, at the tracking pixel point, the steps of respectively determining the Hessian matrix corresponding to each scale window of the pixel value function with a preset plurality of scale windows include:

[0114] In step S131, a Gaussian convolution kernel corresponding to the scale window is determined according to the number of dimensions of the tomographic endoscope image, the scale value corresponding to the scale window, and the transposition corresponding to the corrected pixel value of the tracking pixel point.

[0115] The number of dimensions of the tomographic endoscopic image is determined according to the number of dimensions of the image. For example, if the tomographic endoscopic image is a 2D image, the number of dimensions is 2; if the tomographic endoscopic image is a 3D image, the number of dimensions is 3.

[0116] In step S132, in each dimension, the partial derivative of the Gaussian convolution kernel corresponding to each scale window is determined.

[0117] In step S133, according to the Gaussian convolution kernel and the corresponding partial derivative, a convolution operation is performed on the product of the square of the scale value corresponding to the scale window and the pixel value function, and the Hessian matrix corresponding to the pixel value function for each scale window is determined respectively.

[0118] In the embodiment of the present disclosure, the Hessian matrix [h i，j (x, s)]:

[0119]

[0120] G(x,s)=(2πs 2 ) -D / 2 ×exp(-x T x / 2πs 2 )

[0121] Among them, G(x, s) is the Gaussian convolution kernel corresponding to the scale window s, is a convolution operation, T is the transpose of x, and D is the dimension of x. For example, if the tomographic endoscopic image is a 2D image, the value of D is 2; if the tomographic endoscopic image is a 3D image, the value of D is 3.

[0122] In step S14, the direction and center point of the human tissue at the tracking pixel point are determined according to the pixel value function and the Hessian matrix corresponding to each scale window.

[0123] Among them, the direction of the human tissue at the tracking pixel point is determined according to the pixel value function and the Hessian matrix corresponding to each scale window, and then the center point of the human tissue at the tracking pixel point is determined according to the direction of the human tissue at the tracking pixel point and the pixel value function.

[0124] In one embodiment, see Figure 3 As shown, in step S14, the step of determining the direction and center point of the human tissue at the tracking pixel point according to the pixel value function and the Hessian matrix corresponding to each scale window includes:

[0125] In step S141, the Hessian matrix corresponding to each scale window is subjected to eigenvalue decomposition to obtain an eigenvector matrix and an eigenvalue matrix corresponding to each scale window.

[0126] In the disclosed embodiment, the specific method of performing eigenvalue decomposition on the Hessian matrix is ​​a prior art and will not be described in detail here.

[0127] It can be understood that the Hessian matrix [h i，j (x, s)] = XΛX -1 , where X is the eigenvector matrix, X -1 is the inverse matrix of the eigenvector matrix X, Λ is the eigenvalue matrix, and Λ is a positive diagonal matrix.

[0128] From the basic knowledge of eigenvalue decomposition, it can be understood that there is a correlation between the eigenvectors in the eigenvector matrix and the eigenvalues ​​in the eigenvalue matrix.

[0129] In step S142, the absolute value of the eigenvalue of each element in the eigenvalue matrix corresponding to each scale window is determined.

[0130] In step S143, the eigenvalue matrix corresponding to the eigenvalue with the largest absolute value is used as the target eigenvalue matrix, and the eigenvalues ​​in the target eigenvalue matrix that are approximately 0 are used as the target eigenvalues.

[0131] For example, in the eigenvalue matrix corresponding to each scale window in the embodiment of the present disclosure, there are three eigenvalues ​​λ i, where i = 1, 2, 3. According to the characteristics of the eigenvalue matrix, one of the three eigenvalues ​​is approximately 0, and the other two eigenvalues ​​are approximately equal and their absolute values ​​are much greater than 0. Therefore, the three eigenvalues ​​are simplified to obtain λ1=0, |λ2|=|λ3|>0.

[0132] Furthermore, it is found through experiments that the positive and negative signs of the other two eigenvalues ​​mentioned above depend on the imaging method of the endoscope. For contrast images, such as CTA (Computerized Tomography Angiography) and MRA (Magnetic Resonance Angiography), the pixel value of blood vessels in human tissue is greater than the pixel value of the background, so λ2<0, and λ3<0<0; while for X-ray or DSA (Digital Subtraction Angiography), the pixel value of blood vessels in human tissue is less than the pixel value of the background, so λ2>0, and λ3>0. For the convenience of calculation, the absolute value of the eigenvalue of each element in the eigenvalue matrix is ​​determined, and the possible negative sign is discarded.

[0133] Furthermore, for a set of eigenvalues ​​corresponding to multiple scale windows, the set of values ​​with the largest absolute value is retained, and the values ​​corresponding to other scales are discarded. Taking two scale windows as an example, the eigenvalue corresponding to the first scale window is λ 11 =0,|λ 12 |=|λ 13 |=1, the eigenvalue corresponding to the second size window is λ 21 =0,|λ 22 |=|λ 23 |=2, the eigenvalue corresponding to the third size window is λ 31 =0,|λ 32 |=|λ 33 |=1.5, it is obvious that the absolute value of the eigenvalue corresponding to the second size window is the largest, so the eigenvalue matrix corresponding to the second size window is used as the target eigenvalue matrix.

[0134] Furthermore, the eigenvalue matrix corresponding to the second size window is approximately 0. 21 as the target feature value.

[0135] In step S144, according to the association between the eigenvectors in the eigenvector matrix and the eigenvalues ​​in the eigenvalue matrix, the eigenvector in the eigenvector matrix corresponding to the target eigenvalue is used as the target eigenvector.

[0136] The above embodiment is used for explanation. 21 The eigenvector ξ corresponding to the eigenvector matrix21 As the target eigenvector. The eigenvector matrix is ​​a matrix obtained by solving the linear equations of the Hessian matrix according to the specific values ​​of the eigenvalues.

[0137] In step S145, the direction of the target feature vector is used as the direction of the human tissue at the tracking pixel point.

[0138] It is easy to understand that each vector has a size and a direction. In the present disclosure, the direction of the target feature vector calculated by the Hessian matrix is ​​used as the direction of the human tissue at the tracking pixel point. For example, when the human tissue is a blood vessel, the direction of the target feature vector is used as the direction of the blood vessel at the tracking pixel point.

[0139] In step S146, the center point of the human tissue at the tracking pixel point is determined according to the direction of the human tissue at the tracking pixel point and the pixel value function.

[0140] Optionally, in step S146, the step of determining the center point of the human tissue at the tracking pixel point according to the direction of the human tissue at the tracking pixel point and the pixel value function includes:

[0141] A vertical plane is made by tracking pixel points to indicate the direction of human tissue.

[0142] For example, by tracking the pixel point x0, a plane P(x) perpendicular to the aforementioned direction is made.

[0143] According to the intersection of the vertical plane and the pixel value function, the profile image at the tracking pixel point is determined.

[0144] In the disclosed embodiment, the intersection of the vertical plane and the pixel value function is calculated to obtain the intersection pixel point, for example, B(x0)=P(x0)∩f(x). It can be understood that the intersection of the vertical plane and the pixel value function is a plurality of pixel points, that is, a plurality of intersection pixel points, and a cross-sectional image can be generated based on the plurality of intersection pixel points.

[0145] The centroid of the pixel points in the cross-sectional image is determined, and the centroid is used as the center point of the human tissue at the tracking pixel point.

[0146] In this step, the centroid of the pixel points in the cross-sectional image is calculated according to the coordinates of the pixel points in the cross-sectional image. Repeating the above steps to determine the center point and direction of each frame in the tomographic endoscopic image sequence collected by the endoscope can obtain a set of center points and directions.

[0147] In step S15, according to the direction of each tracking pixel point, the center points corresponding to the tracking pixel points are connected to generate an endoscopic target image corresponding to the human tissue.

[0148] In the disclosed embodiment, the center point of the next tracking pixel point is found from the center point of the previous tracking pixel point along the direction of the tracking pixel point, and the center point of the previous tracking pixel point is connected with the center point of the next tracking pixel point.

[0149] In one embodiment, the center points corresponding to the tracking pixel points are connected according to the direction of each tracking pixel point, and the connected tracking pixel points are labeled through augmented reality processing. For example, different human tissues are labeled with predefined colors and methods, and then an endoscopic target image corresponding to the human tissue is generated based on the labeled blood vessels, nerves, suspected lesion areas, etc.

[0150] The above technical solution aims at any pixel point in the sequence of tomographic endoscopic images collected by the endoscope, and corrects the sampled pixel value of the pixel point to obtain the endoscopic corrected image. The correction of the sampled pixel value of the pixel point can reduce the influence of the absorption of the development enhancer by the human body and the different concentration distribution of the development enhancer on the endoscopic image to a certain extent; determine the tracking pixel point corresponding to the endoscopic corrected image, and construct a pixel value function with the corrected pixel value of each pixel point in the endoscopic corrected image; at the tracking pixel point, use multiple scale windows of different sizes to respectively determine the Hessian matrix corresponding to each scale window of the pixel value function; according to the pixel value function and the Hessian matrix corresponding to each scale window, determine the direction and center point of the human tissue at the tracking pixel point, and the Hessian matrix can be determined by scale windows of different sizes, which can overcome the influence of different human tissues and different imaging methods on the determination of the direction and center point; according to the direction of each tracking pixel point, connect the center points corresponding to each tracking pixel point to generate the endoscopic target image corresponding to the human tissue. Connect the center points according to the direction to form a centerline image of a blood vessel, for example, which can fully reflect the direction and characteristics of the human tissue and improve the clarity of the endoscopic image. This is beneficial to the accuracy of the doctor's observation by operating the endoscope.

[0151] See also Figure 4 As shown, in step S12, the step of determining the tracking pixel points corresponding to the endoscope corrected image includes:

[0152] In step S121, a spherical surface with a radius of R is constructed with the center point of the human tissue corresponding to the last tracking pixel point in the tomographic endoscope image sequence as the center of the sphere.

[0153] In the disclosed embodiment, the radius R can be set on-site by the doctor based on experience, or the diameter of the blood vessel can be estimated and then set based on the diameter of the blood vessel. For example, the value of the radius R is greater than or equal to the diameter of the blood vessel.

[0154] Optionally, the center point of the human tissue corresponding to the last tracking pixel point is taken as ε0, and a spherical surface C(ε0) with a radius R is made with ε0 as the center.

[0155] In step S122, a pixel point set is constructed based on target pixel points on both the spherical surface and the pixel value function.

[0156] In the disclosed embodiment, all pixel points on the sphere C(ε0) are traversed to find the target pixel points that also belong to the pixel value function f(x), and the target pixel points construct a pixel point set F1.

[0157] In step S123, according to the connected areas after the sphere is cut by the pixel value function, the target pixel points in the pixel point set are clustered and duplicate points are deleted in each connected area to obtain a unique regional pixel point in each connected area.

[0158] It can be understood that there are multiple intersections between the sphere C(ε0) and the pixel value function, so the sphere C(ε0) will cut the pixel value function into two parts, namely, the part included in the sphere and the other part not included in the sphere. There are multiple pixel points in the part of the pixel value function included in the sphere, and connecting different pixel points will form different multiple connected areas. Further, in each connected area, the target pixel points in the pixel point set F1 are clustered, and the target pixel points that are repeated in each connected area and other connected areas are deleted, so that there are only unique pixel points in each connected area, and the regional pixel points corresponding to each connected area are obtained.

[0159] In step S124, the connectivity between the regional pixels and the center point in each connected region is determined, and the regional pixels whose connectivity characteristics are connected to the center point are regarded as connected pixels.

[0160] In the embodiment of the present disclosure, the corrected pixel value of the center point and the corrected pixel values ​​of the regional pixel points in each connected area are obtained, and when any connected area is adjacent to the center point and the corrected pixel values ​​of the regional pixel points in the connected area are the same as the corrected pixel value of the center point, the regional pixel points in the connected area are regarded as connected pixel points. The connected pixel points can be regarded as a set to obtain a connected pixel point set F2.

[0161] In step S125, any connected pixel point is taken as a target connected pixel point, and in the connected area, a preset step length is advanced from the center point of the human tissue corresponding to the previous tracking pixel point to the target connected pixel point to obtain the tracking pixel point corresponding to the next frame image in the tomographic endoscope image sequence.

[0162] In the disclosed embodiment, starting from the center point, the connectivity between the center point and the connected pixel points is determined in turn through the path search algorithm. If any connected pixel point is not connected to the center point, the connected pixel point is deleted from the connected pixel point set F2, and finally a pixel point set F3 connected to the center point is obtained.

[0163] It can be explained that if there is no connected pixel at the center point of the human tissue corresponding to any tracking pixel point, it means that the human tissue is terminated. For example, for a blood vessel, if there is no connected pixel at the center point of the blood vessel corresponding to any tracking pixel point, it means that the blood vessel is terminated, there is no branch and it will no longer extend; if there is a connected pixel at the center point of the human tissue corresponding to any tracking pixel point, it means that the human tissue has no branch and can continue to extend; if there are multiple connected pixels at the center point of the human tissue corresponding to any tracking pixel point, it means that the human tissue has branches. For example, for a blood vessel, if there are multiple connected pixels at the center point of the blood vessel corresponding to any tracking pixel point, it means that a fork branch appears in the blood vessel.

[0164] In the disclosed embodiment, the preset step length is used as a threshold parameter and can be manually set by a doctor or can be a system default value.

[0165] Optionally, in step S124, the step of taking the regional pixels connected to the center point by the connectivity representation as connected pixels includes:

[0166] According to the center point of the human tissue corresponding to the last tracking pixel point, the path of the endoscope is divided into a front path and a rear path.

[0167] The connected area on the rear path is used as the backup connected area.

[0168] The spare connected area on the historical path of the endoscope for image acquisition this time is used as the target connected area.

[0169] The pixels in the target connected area whose connectivity representation is connected to the center point are removed, and the remaining pixels in the area whose connectivity representation is connected to the center point are taken as connected pixels.

[0170] In the disclosed embodiment, the human tissue in the image taken by the endoscope is divided into a front path and a rear path with the center point as the boundary. The front path is a path that has not been explored yet, and the rear path is the path behind the endoscope during the current advancement process, including the explored path and the adjacent path of the explored path.

[0171] It can be understood that at least one connected pixel point is located on the rear path. According to the historical path of the endoscope this time, the rear path that overlaps with the historical path is deleted, and the connected pixel points on the deleted rear path are deleted together. The remaining connectivity representations and regional pixel points connected to the center point are used as connected pixel points.

[0172] In one embodiment, the present disclosure can also segment suspicious lesion targets related to the disease, such as inflammation and hyperplasia, from the tomographic endoscopy image.

[0173] During the operation, the endoscopic image analysis method with significantly improved clarity provided by the present invention is used to process the real-time images collected by the endoscope in real time, identify the blood vessels and suspected diseased tissues in the current endoscopic field of view, and then generate an endoscopic image containing only suspected diseased targets, which is called a suspected image.

[0174] Optionally, for the suspected image, the centroid position of the suspected lesion area is measured based on the image moment, and the diameter of the suspected lesion tissue is measured based on the circumscribed circle method. Then, based on the current position of the endoscope and the estimated distance between the suspected lesion area and the endoscope image part, the position coordinates of the suspected lesion area can be obtained.

[0175] In one embodiment, based on the position coordinates of the suspected lesion area, a data record is created for the suspected lesion area in the system database to store information such as the appearance image and measurement data of the suspected lesion area.

[0176] Optionally, on the human-computer interaction interface, when the obtained endoscopic image is displayed in real time, the outer contour of the suspected lesion area can be drawn on the endoscopic image, and measurement information about the suspected lesion area can be annotated.

[0177] Based on the same concept, the present disclosure also provides an endoscopic image analysis device with significantly improved clarity, see Figure 5 As shown, the endoscopic image analysis device 500 with significantly improved clarity includes: a pixel correction module 510, a function construction module 520, a matrix determination module 530, a direction and center point determination module 540 and an image generation module 550.

[0178] The pixel correction module 510 is configured to correct the sampled pixel value of any pixel in the tomographic endoscopic image sequence collected by the endoscope to obtain an endoscopic corrected image;

[0179] The function construction module 520 is configured to determine the tracking pixel points corresponding to the endoscope corrected image, and construct a pixel value function with the corrected pixel values ​​of each pixel point in the endoscope corrected image;

[0180] The matrix determination module 530 is configured to determine the Hessian matrix corresponding to each scale window of the pixel value function at the tracking pixel point using multiple scale windows of different sizes;

[0181] The direction and center point determination module 540 is configured to determine the direction and center point of the human tissue at the tracking pixel point according to the pixel value function and the Hessian matrix corresponding to each scale window;

[0182] The image generation module 550 is configured to connect the center points corresponding to each tracking pixel point according to the direction of each tracking pixel point, and generate an endoscopic target image corresponding to the human tissue.

[0183] Optionally, the direction and center point determination module 540 includes:

[0184] A decomposition submodule is configured to perform eigenvalue decomposition on the Hessian matrix corresponding to each scale window to obtain an eigenvector matrix and an eigenvalue matrix corresponding to each scale window;

[0185] A first determination submodule is configured to determine the absolute value of the eigenvalue of each element in the eigenvalue matrix corresponding to each scale window;

[0186] The second determination submodule is configured to use the eigenvalue matrix where the eigenvalue corresponding to the maximum absolute value is located as the target eigenvalue matrix, and use the eigenvalue in the target eigenvalue matrix that is approximately 0 as the target eigenvalue;

[0187] A third determination submodule is configured to use the eigenvector corresponding to the target eigenvalue in the eigenvector matrix as the target eigenvector according to the correlation between the eigenvectors in the eigenvector matrix and the eigenvalues ​​in the eigenvalue matrix;

[0188] A fourth determination submodule is configured to use the direction of the target feature vector as the direction of the human tissue at the tracking pixel point;

[0189] The fifth determination submodule is configured to determine the center point of the human tissue at the tracking pixel point according to the direction of the human tissue at the tracking pixel point and the pixel value function.

[0190] Optionally, the fifth determining submodule is configured to:

[0191] A vertical plane is made by tracking pixel points to show the direction of human tissue;

[0192] According to the intersection of the vertical plane and the pixel value function, the cross-sectional image at the tracking pixel point is determined;

[0193] The centroid of the pixel points in the cross-sectional image is determined, and the centroid is used as the center point of the human tissue at the tracking pixel point.

[0194] Optionally, the function building module 520 includes:

[0195] The spherical surface construction submodule is configured to construct a spherical surface with a radius R, taking the center point of the human tissue corresponding to the last tracked pixel point in the tomographic endoscope image sequence as the center of the sphere;

[0196] A pixel point set construction submodule is configured to construct a pixel point set according to target pixel points on both the spherical surface and the pixel value function;

[0197] The clustering submodule is configured to cluster and delete duplicate points of target pixel points in the pixel point set in each connected area according to the connected areas after the sphere is cut by the pixel value function, so as to obtain unique regional pixel points in each connected area;

[0198] A sixth determination submodule is configured to determine the connectivity between the regional pixels and the center point in each connected region, and to use the regional pixels connected to the center point as connected pixels.

[0199] The seventh determination submodule is configured to take any connected pixel point as a target connected pixel point, and advance a preset step length from the center point of the human tissue corresponding to the previous tracking pixel point to the target connected pixel point in the connected area to obtain the tracking pixel point corresponding to the next frame image in the tomographic endoscope image sequence.

[0200] Optionally, the sixth determining submodule is configured to:

[0201] According to the center point of the human tissue corresponding to the last tracking pixel point, the path of the endoscope is divided into a front path and a rear path;

[0202] The connected area on the rear path is used as a backup connected area;

[0203] The standby connected area on the historical path of the endoscope for image acquisition this time is used as the target connected area;

[0204] The pixels in the target connected area whose connectivity representation is connected to the center point are removed, and the remaining pixels in the area whose connectivity representation is connected to the center point are taken as connected pixels.

[0205] Optionally, the matrix determination module 530 is configured to:

[0206] Determine a Gaussian convolution kernel corresponding to the scale window according to the number of dimensions of the tomographic endoscope image, the scale value corresponding to the scale window, and the transpose corresponding to the corrected pixel value of the tracking pixel point;

[0207] In each dimension, determine the partial derivative of the Gaussian convolution kernel corresponding to each scale window;

[0208] According to the Gaussian convolution kernel and the corresponding partial derivative, a convolution operation is performed on the product of the square of the scale value corresponding to the scale window and the pixel value function, and the Hessian matrix corresponding to the pixel value function for each scale window is determined respectively.

[0209] Optionally, the pixel correction module 510 is configured to:

[0210] For any pixel point in the tomographic endoscopic image sequence acquired by the endoscope, obtain a sampled pixel value of the pixel point, a sampled pixel value of a previous pixel point adjacent to the pixel point in the tomographic endoscopic image sequence, and a sampled pixel value of a subsequent pixel point adjacent to the pixel point;

[0211] Calculate the weighted average of the sampled pixel value of the pixel point, the sampled pixel value of the previous pixel point, and the sampled pixel value of the next pixel point to obtain a corrected pixel value of the pixel point;

[0212] According to the corrected pixel values ​​corresponding to the pixel points in the tomographic endoscope image sequence, an endoscope corrected image is obtained.

[0213] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0214] Those skilled in the art should understand that the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, for example, multiple modules can be combined or integrated into one module. In addition, the modules described as separate components may or may not be physically separated. Moreover, each module may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. When implemented using hardware, it may be implemented in whole or in part in the form of an integrated circuit or chip.

[0215] An embodiment of the present disclosure also provides a non-temporary computer-readable storage medium having a computer program stored thereon, which, when executed by a first processor, implements the steps of the endoscopic image analysis method with significantly improved clarity as described in any one of the first aspects.

[0216] The present disclosure also provides an electronic device, including:

[0217] a first memory having a computer program stored thereon;

[0218] The second processor is used to execute the computer program in the first memory to implement the steps of the endoscopic image analysis method with significantly improved clarity as described in any one of the first aspects.

[0219] Figure 6 is a block diagram of an electronic device 600 according to an exemplary embodiment. The electronic device 600 can be configured as an endoscopic image analysis device with significantly improved clarity, such as Figure 6 As shown, the electronic device 600 may include: a third processor 601 , a second memory 602 . The electronic device 600 may also include one or more of a multimedia component 603 , an input / output (I / O) interface 604 , and a communication component 605 .

[0220] The third processor 601 is used to control the overall operation of the electronic device 600 to complete all or part of the steps in the above-mentioned endoscopic image analysis method with significantly improved clarity. The second memory 602 is used to store various types of data to support the operation of the electronic device 600, and these data may include, for example, instructions for any application or method used to operate on the electronic device 600, and application-related data, such as pictures, etc. The second memory 602 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, disk or optical disk. The multimedia component 603 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone, which is used to receive external audio signals. The received audio signal may be further stored in the second memory 602 or sent through the communication component 605. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 604 provides an interface between the third processor 601 and other interface modules, and the other interface modules may be keyboards, mice, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 605 is used for wired or wireless communication between the electronic device 600 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, is not limited here. Therefore, the corresponding communication component 605 may include: Wi-Fi module, Bluetooth module, NFC module, etc.

[0221] In an exemplary embodiment, the electronic device 600 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to perform the above-mentioned endoscopic image analysis method with significantly improved clarity.

[0222] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned endoscopic image analysis method with significantly improved clarity are implemented. For example, the computer-readable storage medium can be the above-mentioned second memory 602 including program instructions, and the above-mentioned program instructions can be executed by the third processor 601 of the electronic device 600 to complete the above-mentioned endoscopic image analysis method with significantly improved clarity.

[0223] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0224] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0225] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A method for analyzing an endoscopic image with significantly improved clarity, characterized in that: include: For any pixel point in the tomographic endoscopic image sequence collected by the endoscope, the sampled pixel value of the pixel point is corrected to obtain an endoscopic corrected image; Determine the tracking pixel points corresponding to the endoscope corrected image, and construct a pixel value function with the corrected pixel values ​​of each pixel point in the endoscope corrected image; At the tracking pixel point, using a plurality of scale windows of different sizes, respectively determining a Hessian matrix corresponding to each scale window of the pixel value function; Determine the direction and center point of the human tissue at the tracking pixel point according to the pixel value function and the Hessian matrix corresponding to each scale window; According to the direction of each of the tracking pixel points, the center points corresponding to the tracking pixel points are connected to generate an endoscopic target image corresponding to the human tissue.

2. The endoscopic image analysis method with improved clarity according to claim 1, characterized in that: The step of determining the direction and center point of the human tissue at the tracking pixel point according to the pixel value function and the Hessian matrix corresponding to each scale window comprises: Performing eigenvalue decomposition on the Hessian matrix corresponding to each scale window to obtain an eigenvector matrix and an eigenvalue matrix corresponding to each scale window; Determine the absolute value of the eigenvalue of each element in the eigenvalue matrix corresponding to each of the scale windows; The eigenvalue matrix corresponding to the eigenvalue with the largest absolute value is used as the target eigenvalue matrix, and the eigenvalue with a value close to 0 in the target eigenvalue matrix is ​​used as the target eigenvalue; According to the correlation between the eigenvectors in the eigenvector matrix and the eigenvalues ​​in the eigenvalue matrix, taking the eigenvector corresponding to the target eigenvalue in the eigenvector matrix as the target eigenvector; Taking the direction of the target feature vector as the direction of the human tissue at the tracking pixel point; The center point of the human tissue at the tracking pixel point is determined according to the direction of the human tissue at the tracking pixel point and the pixel value function.

3. The endoscopic image analysis method with improved clarity according to claim 2, characterized in that: The step of determining the center point of the human tissue at the tracking pixel point according to the direction of the human tissue at the tracking pixel point and the pixel value function comprises: A vertical plane is made through the tracking pixel points to indicate the direction of the human tissue; Determining a cross-sectional image at the tracking pixel point according to an intersection of the vertical plane and the pixel value function; The centroid of the pixel points in the cross-sectional image is determined, and the centroid is used as the center point of the human tissue at the tracking pixel point.

4. The endoscopic image analysis method with improved clarity according to claim 1, characterized in that: The step of determining the tracking pixel points corresponding to the endoscope corrected image comprises: Taking the center point of the human tissue corresponding to the last tracking pixel point in the tomographic endoscope image sequence as the center of the sphere, a spherical surface with a radius R is constructed; Constructing a pixel point set according to target pixel points on both the spherical surface and the pixel value function; According to the connected areas after the spherical surface is cut by the pixel value function, clustering and deleting duplicate points of target pixel points in the pixel point set in each connected area to obtain a unique regional pixel point in each connected area; Determine the connectivity between the regional pixel points and the central point in each connected region, and take the regional pixel points connected to the central point by connectivity characterization as connected pixel points; Any of the connected pixel points is taken as a target connected pixel point. In the connected area, a preset step length is advanced from the center point of the human tissue corresponding to the previous tracking pixel point toward the target connected pixel point to obtain the tracking pixel point corresponding to the next frame image in the tomographic endoscope image sequence.

5. The endoscopic image analysis method with improved clarity according to claim 4, characterized in that: The step of taking the pixel points in the area connected to the center point by the connectivity representation as the connected pixel points comprises: Dividing the path of the endoscope into a front path and a rear path according to the center point of the human tissue corresponding to the last tracking pixel point; using the connected area on the rear path as a backup connected area; Taking the spare connected area on the historical path of the endoscope for image acquisition this time as the target connected area; The pixel points in the area where the connectivity representation in the target connected area is connected to the center point are removed, and the remaining pixel points in the area where the connectivity representation is connected to the center point are taken as connected pixel points.

6. The endoscopic image analysis method with improved clarity according to claim 1, characterized in that: The step of determining, at the tracking pixel point, a Hessian matrix corresponding to each of the scale windows of the pixel value function using a plurality of preset scale windows, comprises: Determine a Gaussian convolution kernel corresponding to the scale window according to the number of dimensions of the tomographic endoscope image, the scale value corresponding to the scale window, and the transpose corresponding to the corrected pixel value of the tracking pixel point; In each dimension, determining the partial derivative of the Gaussian convolution kernel corresponding to each scale window; According to the Gaussian convolution kernel and the corresponding partial derivative, a convolution operation is performed on the product of the square of the scale value corresponding to the scale window and the pixel value function, and a Hessian matrix corresponding to the pixel value function for each scale window is determined respectively.

7. The endoscopic image analysis method with improved clarity according to any one of claims 1 to 6, characterized in that: The step of correcting the sampled pixel value of any pixel point in the endoscope image to be processed collected by the endoscope to obtain the endoscope corrected image includes: For any pixel point in a tomographic endoscopic image sequence acquired by an endoscope, obtaining a sampled pixel value of the pixel point, a sampled pixel value of a previous pixel point adjacent to the pixel point in the tomographic endoscopic image sequence, and a sampled pixel value of a subsequent pixel point adjacent to the pixel point; Calculating a weighted average of the sampled pixel value of the pixel point, the sampled pixel value of the previous pixel point, and the sampled pixel value of the next pixel point to obtain a corrected pixel value of the pixel point; The endoscope corrected image is obtained according to the corrected pixel values ​​corresponding to the pixel points in the tomographic endoscope image sequence.

8. An endoscopic image analysis device with significantly improved clarity, characterized in that: include: A pixel correction module is configured to correct a sampled pixel value of any pixel point in a sequence of tomographic endoscopic images collected by the endoscope to obtain an endoscopic corrected image; a function construction module configured to determine the tracking pixel points corresponding to the endoscope corrected image, and to construct a pixel value function with the corrected pixel values ​​of each of the pixel points in the endoscope corrected image; A matrix determination module is configured to determine, at the tracking pixel point, a plurality of scale windows of different sizes, respectively a Hessian matrix corresponding to each scale window of the pixel value function; A direction and center point determination module is configured to determine the direction and center point of the human tissue at the tracking pixel point according to the pixel value function and the Hessian matrix corresponding to each scale window; The image generation module is configured to connect the center points corresponding to the tracking pixel points according to the direction of each of the tracking pixel points to generate an endoscopic target image corresponding to the human tissue.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by the first processor, the steps of the endoscopic image analysis method with significantly improved clarity as described in any one of claims 1 to 7 are implemented.

10. An electronic device, characterized in that: include: a first memory having a computer program stored thereon; A second processor is used to execute the computer program in the first memory to implement the steps of the endoscopic image analysis method with significantly improved clarity as described in any one of claims 1-7.

Images