Endoscope image analysis method, apparatus, storage medium, and device with significantly improved sharpness

Abstract

The present disclosure relates to an endoscope image analysis method, device, storage medium and equipment with improved definition, comprising: for any pixel point of a tomographic endoscope image sequence collected by an endoscope, correcting the sampling pixel value of the pixel point to obtain an endoscope corrected image; determining a tracking pixel point corresponding to the endoscope corrected image, and constructing a pixel value function with the corrected pixel value of each pixel point in the endoscope corrected image; at the tracking pixel point, determining a Hessian matrix corresponding to each scale window of the pixel value function with a plurality of scale windows of different sizes; determining the direction and center point of the human tissue at the tracking pixel point according to the pixel value function and the Hessian matrix corresponding to each scale window; and connecting the center points corresponding to each tracking pixel point to generate an endoscope target image corresponding to the human tissue according to the direction of each tracking pixel point. The direction and features of the human tissue can be fully reflected, and the definition of the endoscope image is improved.

Description

Technical Field

[0001] This disclosure relates to the field of image data processing technology, and more specifically, to a method, apparatus, storage medium, and device for analyzing endoscopic images with significantly improved clarity. Background Technology

[0002] In relevant scenarios, images acquired by endoscopy are provided to a pre-trained neural network model for image segmentation and target recognition. The model extracts target human tissues that are helpful for surgery, such as blood vessels, nerves, and suspected lesions. However, human blood vessel images have complex structures and also suffer from uneven brightness, weak contrast, and noise interference. This results in blurred direction and features after segmenting the input image, leading to less than ideal endoscopy image quality. Summary of the Invention

[0003] The purpose of this disclosure is to provide an endoscopic image analysis method, apparatus, storage medium, and device with significantly improved clarity, designed to fully represent the contours and features of human tissues and improve the clarity of endoscopic images.

[0004] To achieve the above objectives, a first aspect of this disclosure provides an endoscopic image analysis method with significantly improved clarity, comprising:

[0005] For any pixel in the tomographic endoscopic image sequence acquired by the endoscope, the sampled pixel value of the pixel is corrected to obtain the corrected endoscopic image.

[0006] Determine the tracking pixel points corresponding to the endoscope correction image, and construct a pixel value function using the correction pixel values ​​of each pixel point in the endoscope correction image;

[0007] At the tracked pixel, the Hessian matrix corresponding to the pixel value function for each scale window is determined using multiple scale windows of different sizes.

[0008] Based on the pixel value function and the Hessian matrix corresponding to each scale window, the orientation and center point of the human tissue at the tracking pixel are determined.

[0009] Based on the direction of each tracking pixel, the center points corresponding to each tracking pixel are connected to generate an endoscopic target image corresponding to the human tissue.

[0010] Optionally, the step of determining the orientation and center point of the human tissue at the tracking pixel based on the pixel value function and the Hessian matrix corresponding to each scale window includes:

[0011] Eigenvalue decomposition is performed on the Hessian matrix corresponding to each scale window to obtain the eigenvector matrix and eigenvalue matrix corresponding to each scale window.

[0012] Determine the absolute value of the eigenvalue of each element in the eigenvalue matrix corresponding to each scale window;

[0013] The eigenvalue matrix containing the eigenvalue corresponding to the eigenvalue with the largest absolute value is taken as the target eigenvalue matrix, and the eigenvalues ​​in the target eigenvalue matrix that take values ​​approximately 0 are taken as the target eigenvalues.

[0014] Based on the correlation between the eigenvectors in the eigenvector matrix and the eigenvalues ​​in the eigenvalue matrix, the eigenvector in the eigenvector matrix that corresponds to the target eigenvalue is taken as the target eigenvector;

[0015] The direction of the target feature vector is taken as the orientation of the human tissue at the tracking pixel;

[0016] The center point of the human tissue at the tracked pixel is determined based on the direction of the human tissue at the tracked pixel and the pixel value function.

[0017] Optionally, the step of determining the center point of the human tissue at the tracking pixel based on the orientation of the human tissue at the tracking pixel and the pixel value function includes:

[0018] A plane perpendicular to the orientation of the human tissue is drawn through the tracking pixel.

[0019] The profile image at the tracked pixel is determined based on the intersection of the vertical plane and the pixel value function.

[0020] Determine the centroid of the pixels in the cross-sectional image, and use the centroid as the center point of the human tissue at the tracked pixel.

[0021] Optionally, the step of determining the tracking pixel point corresponding to the endoscopic correction image includes:

[0022] Using the center point of the human tissue corresponding to the previous tracking pixel in the tomographic endoscopy image sequence as the center of the sphere, a sphere with radius R is constructed;

[0023] Construct a set of pixels based on the target pixels that are simultaneously on the sphere and the pixel value function;

[0024] Based on the connected regions after the sphere is cut by the pixel value function, the target pixels in the pixel point set in each connected region are clustered and duplicate points are deleted to obtain unique region pixels in each connected region.

[0025] Determine the connectivity between the region pixels and the center point in each connected region, and use the region pixels whose connectivity is characterized to be connected to the center point as connected pixels;

[0026] Taking any of the connected pixels as the target connected pixel, in the connected region, advance a preset step length from the center point of the human tissue corresponding to the previous tracked pixel towards the target connected pixel to obtain the tracked pixel corresponding to the next frame image in the tomographic endoscopy image sequence.

[0027] Optionally, the step of taking the region pixels whose connectivity representation is connected to the center point as connected pixels includes:

[0028] Based on the center point of the human tissue corresponding to the previous tracked pixel, the path of the endoscope is divided into a front path and a rear path.

[0029] The connected regions located on the rear path are designated as backup connected regions.

[0030] The spare connected region on the historical path of the endoscope's current image acquisition is used as the target connected region;

[0031] Remove the pixels in the target connected region whose connectivity representation is connected to the center point, and take the remaining pixels in the connected region whose connectivity representation is connected to the center point as connected pixels.

[0032] Optionally, the step of determining the Hessian matrix corresponding to the pixel value function for each preset scale window at the tracked pixel point includes:

[0033] The Gaussian convolution kernel corresponding to the scale window is determined based on the number of dimensions of the endoscopic image, the scale value corresponding to the scale window, and the transpose corresponding to the corrected pixel value of the tracked pixel.

[0034] In each dimension, determine the partial derivative of the Gaussian convolution kernel corresponding to each scale window;

[0035] Based on the Gaussian convolution kernel and its corresponding partial derivative, a convolution operation is performed on the product of the square of the scale value corresponding to the scale window and the pixel value function to determine the Hessian matrix of the pixel value function for each scale window.

[0036] Optionally, the step of correcting the sampled pixel value of any pixel in the endoscope image to be processed, to obtain a corrected endoscope image, includes:

[0037] For any pixel in the tomographic endoscopic image sequence acquired by the endoscope, obtain the sampled pixel value of the pixel, the sampled pixel value of the preceding pixel in the tomographic endoscopic image sequence, and the sampled pixel value of the following pixel.

[0038] The corrected pixel value of the pixel is obtained by calculating the weighted average of the sampled pixel value of the pixel, the sampled pixel value of the previous pixel, and the sampled pixel value of the next pixel.

[0039] The corrected endoscopic image is obtained based on the corrected pixel values ​​corresponding to the pixels in the tomographic endoscopic image sequence.

[0040] A second aspect of this disclosure provides an endoscopic image analysis apparatus with significantly improved clarity, comprising:

[0041] The pixel correction module is configured to correct the sampled pixel value of any pixel in the sequence of tomographic endoscopic images acquired by the endoscope, thereby obtaining an endoscope-corrected image.

[0042] The function construction module is configured to determine the tracking pixel points corresponding to the endoscope correction image, and construct a pixel value function using the correction pixel values ​​of each pixel point in the endoscope correction image;

[0043] The matrix determination module is configured to determine the Hessian matrix corresponding to the pixel value function for each scale window at the tracked pixel point, using multiple scale windows of different sizes.

[0044] The orientation and center point determination module is configured to determine the orientation and center point of the human tissue at the tracking pixel based on the pixel value function and the Hessian matrix corresponding to each scale window.

[0045] The image generation module is configured to connect the center points corresponding to each tracking pixel according to the direction of each tracking pixel to generate an endoscopic target image corresponding to the human tissue.

[0046] Optionally, the orientation and center point determination module includes:

[0047] The decomposition submodule is configured to perform eigenvalue decomposition on the Hessian matrix corresponding to each scale window to obtain the eigenvector matrix and eigenvalue matrix corresponding to each scale window.

[0048] The first determining submodule is configured to determine the absolute value of the eigenvalue of each element in the eigenvalue matrix corresponding to each scale window;

[0049] The second determining submodule is configured to take the eigenvalue matrix containing the eigenvalue corresponding to the eigenvalue with the largest absolute value as the target eigenvalue matrix, and take the eigenvalue in the target eigenvalue matrix that takes a value approximately 0 as the target eigenvalue.

[0050] The third determining submodule is configured to select the feature vector in the feature vector matrix that corresponds to the target feature value as the target feature vector based on the correlation between the feature vector in the feature vector matrix and the feature value in the feature value matrix.

[0051] The fourth determining submodule is configured to use the direction of the target feature vector as the orientation of the human tissue at the tracking pixel.

[0052] The fifth determining submodule is configured to determine the center point of the human tissue at the tracking pixel based on the orientation of the human tissue at the tracking pixel and the pixel value function.

[0053] Optionally, the fifth determining submodule is configured as follows:

[0054] A plane perpendicular to the orientation of the human tissue is drawn through the tracking pixel.

[0055] The profile image at the tracked pixel is determined based on the intersection of the vertical plane and the pixel value function.

[0056] Determine the centroid of the pixels in the cross-sectional image, and use the centroid as the center point of the human tissue at the tracked pixel.

[0057] Optionally, the function building module includes:

[0058] The spherical construction submodule is configured to construct a sphere with radius R, using the center point of the human tissue corresponding to the previous tracked pixel in the tomographic endoscopy image sequence as the center of the sphere;

[0059] The pixel set construction submodule is configured to construct a pixel set based on target pixels that are simultaneously on the sphere and the pixel value function;

[0060] The clustering submodule is configured to perform clustering and duplicate point deletion on the target pixels in the set of pixels in each connected region based on the connected regions after the sphere is cut by the pixel value function, so as to obtain unique region pixels in each connected region.

[0061] The sixth determining submodule is configured to determine the connectivity between the region pixels in each connected region and the center point, and to use the region pixels whose connectivity characteristics are connected to the center point as connected pixels;

[0062] The seventh determination submodule is configured to take any of the connected pixels as the target connected pixel, and advance a preset step size from the center point of the human tissue corresponding to the previous tracked pixel in the connected region towards the target connected pixel to obtain the tracked pixel corresponding to the next frame image in the tomographic endoscopy image sequence.

[0063] Optionally, the sixth determining submodule is configured as follows:

[0064] Based on the center point of the human tissue corresponding to the previous tracked pixel, the path of the endoscope is divided into a front path and a rear path.

[0065] The connected regions located on the rear path are designated as backup connected regions.

[0066] The spare connected region on the historical path of the endoscope's current image acquisition is used as the target connected region;

[0067] Remove the pixels in the target connected region whose connectivity representation is connected to the center point, and take the remaining pixels in the connected region whose connectivity representation is connected to the center point as connected pixels.

[0068] Optionally, the matrix determination module is configured to:

[0069] The Gaussian convolution kernel corresponding to the scale window is determined based on the number of dimensions of the endoscopic image, the scale value corresponding to the scale window, and the transpose corresponding to the corrected pixel value of the tracked pixel.

[0070] In each dimension, determine the partial derivative of the Gaussian convolution kernel corresponding to each scale window;

[0071] Based on the Gaussian convolution kernel and its corresponding partial derivative, a convolution operation is performed on the product of the square of the scale value corresponding to the scale window and the pixel value function to determine the Hessian matrix of the pixel value function for each scale window.

[0072] Optionally, the pixel correction module is configured to:

[0073] For any pixel in the tomographic endoscopic image sequence acquired by the endoscope, obtain the sampled pixel value of the pixel, the sampled pixel value of the preceding pixel in the tomographic endoscopic image sequence, and the sampled pixel value of the following pixel.

[0074] The corrected pixel value of the pixel is obtained by calculating the weighted average of the sampled pixel value of the pixel, the sampled pixel value of the previous pixel, and the sampled pixel value of the next pixel.

[0075] The corrected endoscopic image is obtained based on the corrected pixel values ​​corresponding to the pixels in the tomographic endoscopic image sequence.

[0076] A third aspect of this disclosure provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a first processor, implements the steps of the endoscopic image analysis method with significantly improved clarity as described in any of the first aspects.

[0077] A fourth aspect of this disclosure provides an electronic device, comprising:

[0078] The first memory, on which the computer program is stored;

[0079] A second processor is configured to execute the computer program in the first memory to implement the steps of the endoscopic image analysis method with significantly improved clarity as described in any of the first aspects.

[0080] The above technical solution can achieve at least the following beneficial effects:

[0081] For any pixel in a sequence of tomographic endoscopic images acquired by an endoscope, the sampled pixel value is corrected to obtain a corrected endoscopic image. This pixel value correction can, to some extent, reduce the impact of contrast enhancer absorption by the human body and varying contrast enhancer concentrations on the endoscopic image. Tracking pixels corresponding to the corrected endoscopic image are determined, and a pixel value function is constructed using the corrected pixel values ​​of each pixel in the corrected image. At each tracking pixel, the Hessian matrix corresponding to the pixel value function is determined for each scale window of different sizes. Based on the pixel value function and the Hessian matrix corresponding to each scale window, the orientation and center point of the human tissue at the tracking pixel are determined. The ability to determine the Hessian matrix using scale windows of different sizes overcomes the influence of different human tissues and imaging methods on the determination of orientation and center point. Based on the orientation of each tracking pixel, the center points corresponding to each tracking pixel are connected to generate the endoscopic target image corresponding to the human tissue. Connecting the center points according to the orientation forms, for example, a centerline image of a blood vessel, which can fully represent the orientation and features of the human tissue, improving the clarity of the endoscopic image. This, in turn, improves the accuracy of doctors' observations when using endoscopes.

[0082] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0083] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings:

[0084] Figure 1 This is a flowchart illustrating an endoscopic image analysis method with significantly improved clarity according to embodiments of the present disclosure.

[0085] Figure 2 This is an implementation shown according to an embodiment of the present disclosure. Figure 1 The flowchart for step S13.

[0086] Figure 3 This is an implementation shown according to an embodiment of the present disclosure. Figure 1 The flowchart for step S14.

[0087] Figure 4 This is an implementation shown according to an embodiment of the present disclosure. Figure 1 The flowchart for step S12.

[0088] Figure 5 This is a block diagram of an endoscopic image analysis device with significantly improved clarity, according to an embodiment of the present disclosure.

[0089] Figure 6 This is a block diagram of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0090] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.

[0091] Before introducing the endoscopic image analysis method, apparatus, storage medium and device with significantly improved clarity provided in this disclosure, let me first introduce the application scenarios of this disclosure. When doctors use endoscopes to collect images of tissues, blood vessels and other structures in the human body, it is necessary to clearly display the images captured by the endoscope to avoid doctors not understanding the direction of human tissues, the degree of lesions, etc., which could lead to the endoscope being pushed in the wrong direction or tissue in the body, thus aggravating the patient's pain.

[0092] Therefore, this disclosure provides an endoscopic image analysis method with significantly improved clarity, aiming to fully represent the orientation and characteristics of human tissues and improve the clarity of endoscopic images. This, in turn, improves the accuracy of doctors' endoscopic observation and reduces the risk of doctors advancing the endoscope in the wrong direction or tissue within the body. See [link to relevant documentation]. Figure 1 As shown, the method includes the following steps.

[0093] In step S11, for any pixel in the tomographic endoscopic image sequence acquired by the endoscope, the sampled pixel value of the pixel is corrected to obtain the corrected endoscopic image.

[0094] In this embodiment of the disclosure, the method can be applied to a neural network model, for example, by inputting a sequence of tomographic endoscopic images acquired by an endoscope into a pre-trained neural network model to obtain the endoscopic target image output by the neural network model.

[0095] It can be noted that, depending on the endoscopic task, the sequence of tomographic endoscopic images acquired by the endoscope can be input into different neural network models. For example, for the task of constructing blood vessel images, the sequence of tomographic endoscopic images acquired by the endoscope can be input into a pre-trained blood vessel neural network model, which is trained using blood vessel labeled sample images.

[0096] In this embodiment of the disclosure, the correction of the sampled pixel value of a pixel can be achieved by first performing preprocessing such as cleaning on the endoscopic image to repair obvious imaging defects. Further, a neighborhood filtering method is used to filter and correct the preprocessed endoscopic image. In step S11, the step of correcting the sampled pixel value of any pixel in the endoscopic image to be processed acquired by the endoscope, using a neighborhood filtering method, to obtain the corrected endoscopic image, includes:

[0097] For any pixel in the endoscopic tomographic image sequence acquired by the endoscope, obtain the sampled pixel value of the pixel, the sampled pixel value of the preceding pixel in the endoscopic tomographic image sequence, and the sampled pixel value of the following pixel.

[0098] The corrected pixel value of a pixel is obtained by calculating the weighted average of the sampled pixel value of the pixel, the sampled pixel value of the previous pixel, and the sampled pixel value of the next pixel.

[0099] The corrected endoscopic image is obtained by using the corrected pixel values ​​corresponding to the pixels in the tomographic endoscopic image sequence.

[0100] For example, for any pixel P n (x, y), find the pixel P in the sequence of endoscopic images. n The pixel P at the same position directly adjacent to (x, y) n-1 (x, y) and the previous pixel P n+1 (x, y), and then calculate the weighted average of the corrected pixel values ​​of these three pixels to obtain pixel P. n Corrected pixel values ​​for (x, y).

[0101] Any pixel P can be obtained using the following formula n The corrected pixel value P' of (x, y) n (x, y):

[0102] P'n (x, y) = α1·P n-1 (x, y) + α2·P n (x, y) + α3·P n+1 (x, y)

[0103] Wherein, α1, α2, and α3 are weighting coefficients, which should satisfy α1 + α2 + α3 = 1. Each weighting coefficient can be adjusted by the doctor or preset by the system. Optionally, a set of reference values ​​for the weighting coefficients is α1 = 0.1, α2 = 0.8, and α3 = 0.1.

[0104] Furthermore, image segmentation networks such as CNN (Convolutional Neural Network), FCN (Fully Convolutional Network), SegNet (Semantic Network), and UNet (U-Network) can be used to segment the filtered and corrected tomographic endoscopic images, resulting in endoscopic corrected images that contain only vascular networks, neural networks, and suspected lesion areas.

[0105] In one implementation, filtering algorithms such as Gaussian filtering and bilateral filtering can be used to smooth the endoscopic correction image containing only vascular networks, neural networks, and suspected lesion areas to obtain the final endoscopic correction image.

[0106] In step S12, the tracking pixel points corresponding to the endoscope correction image are determined, and a pixel value function is constructed using the correction pixel values ​​of each pixel point in the endoscope correction image.

[0107] In this embodiment of the disclosure, the first tracking pixel in the current endoscopic correction image can be specified by the doctor, for example, by manually inputting the coordinates of the tracking pixel, or it can be specified by the system default. Subsequent tracking pixels in the endoscopic correction image can be determined based on the previous tracking pixel.

[0108] In this embodiment of the disclosure, the coordinates of each pixel in the endoscope-corrected image are used as variables of the function, and the corrected pixel values ​​of each pixel in the endoscope-corrected image are used as solutions of the function to construct a pixel value function f(x).

[0109] In step S13, at the tracked pixel point, the Hessian matrix corresponding to each scale window is determined for the pixel value function at each scale window with multiple scale windows of different sizes.

[0110] In this embodiment of the disclosure, the tracking pixel is taken as the current point x0, and at the position x = x0, the Hessian matrix H(x, s) of the function f(x) is calculated using a scale window s of different sizes. i，j [(x, s)],

[0111] The scale window can be centered on the tracking pixel and cover the tracking pixel and the pixels around the tracking pixel. Thus, multiple scale windows of different sizes can cover different areas around the tracking pixel, and thus multiple scale windows of different sizes can cover different numbers of pixels.

[0112] Furthermore, multiple scale windows of different sizes can be multiple scale windows whose sizes are in a geometric sequence.

[0113] Optionally, in step S13, see Figure 2 As shown, at the tracking pixel point, the steps for determining the Hessian matrix of the pixel value function for each preset scale window include:

[0114] In step S131, the Gaussian convolution kernel corresponding to the scale window is determined based on the number of dimensions of the endoscopic image, the scale value corresponding to the scale window, and the transpose corresponding to the corrected pixel value of the tracked pixel.

[0115] The number of dimensions of a tomographic endoscopy image is determined by the number of dimensions of the image. For example, if the tomographic endoscopy image is a 2D image, the number of dimensions is 2; if the tomographic endoscopy image is a 3D image, the number of dimensions is 3.

[0116] In step S132, the partial derivative of the Gaussian convolution kernel corresponding to each scale window is determined in each dimension.

[0117] In step S133, based on the Gaussian convolution kernel and its corresponding partial derivative, a convolution operation is performed on the product of the square of the scale value corresponding to the scale window and the pixel value function, thereby determining the Hessian matrix of the pixel value function for each scale window.

[0118] In this embodiment of the disclosure, the Hessian matrix [h] corresponding to each scale window of the pixel value function can be determined by the following formula: i，j [(x, s)]:

[0119]

[0120] G(x, s) = (2πs) 2 ) -D / 2 ×exp(-x T x / 2πs 2 )

[0121] Where G(x, s) is the Gaussian convolution kernel corresponding to the scale window s. For convolution operation, T is the transpose of x, and D is the dimension of x. For example, if the endoscopic image is a 2D image, then the value of D is 2; if the endoscopic image is a 3D image, then the value of D is 3.

[0122] In step S14, the orientation and center point of the human tissue at the tracking pixel are determined based on the pixel value function and the Hessian matrix corresponding to each scale window.

[0123] Specifically, the orientation of the human tissue at the tracking pixel is determined based on the pixel value function and the Hessian matrix corresponding to each scale window. Then, the center point of the human tissue at the tracking pixel is determined based on the orientation of the human tissue at the tracking pixel and the pixel value function.

[0124] In one implementation, see Figure 3 As shown, in step S14, the step of determining the orientation and center point of the human tissue at the tracking pixel point based on the pixel value function and the Hessian matrix corresponding to each scale window includes:

[0125] In step S141, the Hessian matrix corresponding to each scale window is decomposed into eigenvalues ​​to obtain the eigenvector matrix and eigenvalue matrix corresponding to each scale window.

[0126] The specific method for eigenvalue decomposition of the Hessian matrix in this embodiment is existing technology and will not be described in detail here.

[0127] It is understandable that the Hessian matrix [h] corresponds to each scale window. i，j [(x, s)]=XΛX -1 Where X is the eigenvector matrix, X -1 Let X be the inverse of the eigenvector matrix X, Λ be the eigenvalue matrix, and Λ be the diagonal matrix.

[0128] As can be understood from the basic knowledge of eigenvalue decomposition, there is a correlation between the eigenvectors in the eigenvector matrix and the eigenvalues ​​in the eigenvalue matrix.

[0129] In step S142, the absolute value of the eigenvalue of each element in the eigenvalue matrix corresponding to each scale window is determined.

[0130] In step S143, the eigenvalue matrix containing the eigenvalue corresponding to the eigenvalue with the largest absolute value is taken as the target eigenvalue matrix, and the eigenvalues ​​in the target eigenvalue matrix that take values ​​approximately 0 are taken as the target eigenvalues.

[0131] For example, in each scale window embodiment of this disclosure, there are 3 eigenvalues ​​λ in the eigenvalue matrix. iLet i = 1, 2, 3. According to the properties of the eigenvalue matrix, one of the three eigenvalues ​​is approximately 0, and the other two eigenvalues ​​are approximately equal with absolute values ​​much greater than 0. Therefore, after simplifying the three eigenvalues, we get λ1 = 0 and |λ2| = |λ3| > 0.

[0132] Furthermore, experiments revealed that the signs of the other two eigenvalues ​​mentioned earlier depend on the imaging method of the endoscope. For contrast-enhanced images, such as CTA (Computerized Tomography Angiography) and MRA (Magnetic Resonance Angiography), the pixel values ​​of blood vessels in human tissue are greater than the pixel values ​​of the background; therefore, λ2 < 0 and λ3 < 0 < 0. However, for X-ray or DSA (Digital Subtraction Angiography) images, the pixel values ​​of blood vessels in human tissue are less than the pixel values ​​of the background; therefore, λ2 > 0 and λ3 > 0. For ease of calculation, the absolute value of the eigenvalue of each element in the eigenvalue matrix is ​​determined, and any possible negative signs are discarded.

[0133] Furthermore, for a set of feature values ​​corresponding to multiple scale windows, the set with the largest absolute value is retained, while the values ​​corresponding to other scales are discarded. Taking two-size windows as an example, the feature value corresponding to the first size window is λ. 11 =0,|λ 12 |=|λ 13 |=1, the eigenvalue corresponding to the second-sized window is λ 21 =0,|λ 22 |=|λ 23 |=2, the eigenvalue corresponding to the third-sized window is λ 31 =0,|λ 32 |=|λ 33 =1.5. Obviously, the absolute value of the eigenvalues ​​corresponding to the second-sized window is the largest. Therefore, the eigenvalue matrix corresponding to the second-sized window is used as the target eigenvalue matrix.

[0134] Furthermore, the eigenvalues ​​of λ in the eigenvalue matrix corresponding to the second-sized window are approximately 0. 21 As the target feature value.

[0135] In step S144, based on the correlation between the eigenvectors in the eigenvector matrix and the eigenvalues ​​in the eigenvalue matrix, the eigenvectors in the eigenvector matrix that correspond to the target eigenvalues ​​are taken as the target eigenvectors.

[0136] The above embodiments will be used for illustration, and λ will be used for illustration. 21 The eigenvector ξ corresponding to the eigenvector matrix21 As the target eigenvectors, the eigenvector matrix is ​​obtained by solving the linear equations of the Hessian matrix based on the specific values ​​of the eigenvalues.

[0137] In step S145, the direction of the target feature vector is used as the direction of the human tissue at the tracking pixel.

[0138] It is easy to understand that each vector has magnitude and direction. In this disclosure, the direction of the target feature vector calculated by the Hessian matrix is ​​used as the direction of the human tissue at the tracking pixel. For example, when the human tissue is a blood vessel, the direction of the target feature vector is used as the direction of the blood vessel at the tracking pixel.

[0139] In step S146, the center point of the human tissue at the tracking pixel is determined based on the direction of the human tissue at the tracking pixel and the pixel value function.

[0140] Optionally, in step S146, the step of determining the center point of the human tissue at the tracking pixel based on the direction of the human tissue at the tracking pixel and the pixel value function includes:

[0141] The vertical plane that tracks the direction of human tissue is determined by the pixels.

[0142] For example, construct a plane P(x) perpendicular to the aforementioned direction by tracking pixel x0.

[0143] The profile image at the tracking pixel is determined based on the intersection of the vertical plane and the pixel value function.

[0144] In this embodiment of the disclosure, the intersection of the vertical plane and the pixel value function is obtained to obtain the intersection pixel points, for example, B(x0)=P(x0)∩f(x). It can be understood that the intersection of the vertical plane and the pixel value function is multiple pixel points, that is, multiple intersection pixel points. A profile image can be generated based on the multiple intersection pixel points.

[0145] Determine the centroid of each pixel in the cross-sectional image and use it as the center point of the human tissue at the tracking pixel.

[0146] In this step, the centroid of each pixel in the cross-sectional image is calculated based on its coordinates. Repeating this process determines the center point and orientation of each frame in the endoscopic tomographic image sequence, resulting in a set of center points and orientations.

[0147] In step S15, the center points corresponding to each tracked pixel are connected according to the direction of each tracked pixel to generate an endoscopic target image corresponding to human tissue.

[0148] In this embodiment of the disclosure, the center point of the next tracking pixel is found along the direction of the previous tracking pixel from the center point of the previous tracking pixel, and the center point of the previous tracking pixel is connected to the center point of the next tracking pixel.

[0149] In one implementation, the center points corresponding to each tracked pixel are connected according to the direction of each tracked pixel. The connected tracked pixels are then labeled using augmented reality processing. For example, different human tissues are labeled using predefined colors and methods. Based on the labeled blood vessels, nerves, suspected lesion areas, etc., an endoscopic target image of the human tissue is generated.

[0150] The above technical solution corrects the sampled pixel values ​​of any pixel in the endoscopic tomographic image sequence acquired by an endoscope, resulting in a corrected endoscopic image. This pixel value correction can reduce the impact of contrast enhancer absorption by the human body and varying contrast enhancer concentrations on the endoscopic image. The solution then identifies the tracking pixels corresponding to the corrected endoscopic image and constructs a pixel value function using the corrected pixel values ​​of each pixel. At each tracking pixel, the Hessian matrix corresponding to the pixel value function is determined for each scale window of different sizes. Based on the pixel value function and the Hessian matrix for each scale window, the orientation and center point of the human tissue at the tracking pixel are determined. The ability to determine the Hessian matrix using scale windows of different sizes overcomes the influence of different human tissues and imaging methods on the determination of orientation and center point. Finally, the center points of each tracking pixel are connected according to its orientation to generate an endoscopic target image of the human tissue. Connecting the center points according to the orientation creates, for example, a centerline image of a blood vessel, which fully reflects the orientation and features of the human tissue, improving the clarity of the endoscopic image. This, in turn, improves the accuracy of doctors' observations when using endoscopes.

[0151] See Figure 4 As shown, in step S12, the step of determining the tracking pixel point corresponding to the endoscopic correction image includes:

[0152] In step S121, a sphere with radius R is constructed with the center point of the human tissue corresponding to the previous tracked pixel in the endoscopic image sequence as the center.

[0153] In this embodiment of the disclosure, the radius R can be set by the doctor on-site based on experience, or the diameter of the blood vessel can be estimated and then set according to the diameter of the blood vessel. For example, the value of the radius R is greater than or equal to the diameter of the blood vessel.

[0154] Optionally, the center point of the human tissue corresponding to the previous tracked pixel is taken as ε0, and a sphere C(ε0) with radius R is drawn with ε0 as the center.

[0155] In step S122, a set of pixels is constructed based on the target pixels that are simultaneously on the sphere and the pixel value function.

[0156] In this embodiment of the disclosure, all pixels on the sphere C(ε0) are traversed to find the target pixel that simultaneously belongs to the pixel value function f(x), and the target pixel is used to construct a pixel set F1.

[0157] In step S123, based on the connected regions cut by the sphere-to-pixel value function, the target pixels in the pixel set in each connected region are clustered and duplicate points are removed to obtain unique region pixels in each connected region.

[0158] It is understandable that the sphere C(ε0) intersects with the pixel value function at multiple points. Therefore, the sphere C(ε0) divides the pixel value function into two parts: one contained within the sphere and the other not contained within the sphere. The part of the pixel value function contained within the sphere contains multiple pixels. Connecting these different pixels forms multiple connected regions. Furthermore, within each connected region, the target pixels in the pixel set F1 are clustered, and target pixels that are duplicated in other connected regions are removed. Thus, each connected region contains only a unique pixel, resulting in the region pixels corresponding to each connected region.

[0159] In step S124, the connectivity between the region pixels and the center point in each connected region is determined, and the region pixels whose connectivity is characterized to be connected to the center point are taken as connected pixels.

[0160] In this embodiment of the disclosure, the corrected pixel value of the center point and the corrected pixel value of the regional pixel in each connected region are obtained. When any connected region is adjacent to the center point and the corrected pixel value of the regional pixel in the connected region is the same as the corrected pixel value of the center point, the regional pixel in the connected region is taken as the connected pixel. The connected pixel can be taken as a set to obtain the connected pixel set F2.

[0161] In step S125, any connected pixel is taken as the target connected pixel. In the connected region, a preset step size is taken from the center point of the human tissue corresponding to the previous tracked pixel to the target connected pixel to obtain the tracked pixel corresponding to the next frame image in the tomographic endoscopy image sequence.

[0162] In this embodiment, starting from the center point, the connectivity between the center point and connected pixels is determined sequentially using a path search algorithm. If any connected pixel is not connected to the center point, it is removed from the set of connected pixels F2, ultimately resulting in the set of pixels F3 connected to the center point.

[0163] It can be explained that if the center point of any tracked pixel corresponding to a human tissue has no connected pixels, it means that the human tissue terminates. For example, for a blood vessel, if the center point of any tracked pixel corresponding to a blood vessel has no connected pixels, it means that the blood vessel terminates, has no branches, and does not extend further. If the center point of any tracked pixel corresponding to a human tissue has one connected pixel, it means that the human tissue has no branches and can continue to extend. If the center point of any tracked pixel corresponding to a human tissue has multiple connected pixels, it means that the human tissue has branches. For example, for a blood vessel, if the center point of any tracked pixel corresponding to a blood vessel has multiple connected pixels, it means that the blood vessel has branched.

[0164] In this embodiment of the disclosure, the preset step size is used as a threshold parameter, which can be manually set by the doctor or be the system default value.

[0165] Optionally, in step S124, the step of taking the region pixels whose connectivity representation is connected to the center point as connected pixels includes:

[0166] Based on the center point of the human tissue corresponding to the previous tracked pixel, the endoscope path is divided into a front path and a rear path.

[0167] The connected regions on the subsequent path are designated as backup connected regions.

[0168] The spare connected region on the historical path of the endoscope's current image acquisition is used as the target connected region.

[0169] Remove the pixels in the target connected region whose connectivity representation is connected to the center point, and take the remaining pixels in the connected region whose connectivity representation is connected to the center point as connected pixels.

[0170] In this embodiment of the disclosure, the human tissue in the image obtained by the endoscope is divided into a front path and a rear path, with the center point as the boundary. The front path is the path that has not yet been explored, while the rear path is the path behind the endoscope during the current advancement process, including the explored path and the adjacent path of the explored path.

[0171] It is understandable that at least one connected pixel is located on the back path. Based on the current historical path of the endoscope, the back path that overlaps with the historical path is deleted, and the connected pixels on the deleted back path are also deleted. The remaining pixels representing the connectivity of the region connected to the center point are taken as connected pixels.

[0172] In one embodiment, this disclosure can also segment suspicious lesions related to the condition, such as inflammation and hyperplasia, from endoscopic images.

[0173] During the surgery, the endoscopic image analysis method with significantly improved clarity provided in this disclosure is used to process the real-time images acquired by the endoscope in real time, identify blood vessels and suspected lesions in the current endoscopic field of view, and then generate an endoscopic image containing only suspected lesions, called a suspected image.

[0174] Optionally, for a suspected lesion image, the centroid of the suspected lesion region is measured based on image moments, and the diameter of the suspected lesion tissue is measured using the circumscribed circle method. Then, based on the current position of the endoscope and the estimated distance between the suspected lesion region and the endoscopic image section, the positional coordinates of the suspected lesion region can be obtained.

[0175] In one implementation, based on the location coordinates of the suspected lesion area, a data record is created for the suspected lesion area in the system database to store information such as the appearance image and measurement data of the suspected lesion area.

[0176] Optionally, when the obtained endoscopic image is displayed in real time on the human-computer interaction interface, the outline of the suspected lesion area can be drawn on the endoscopic image, and measurement information about the suspected lesion area can be marked.

[0177] Based on the same concept, this disclosure also provides an endoscopic image analysis device with significantly improved clarity, see [link to relevant documentation]. Figure 5 As shown, the endoscope image analysis device 500 with significantly improved clarity includes: a pixel correction module 510, a function construction module 520, a matrix determination module 530, a direction and center point determination module 540, and an image generation module 550.

[0178] The pixel correction module 510 is configured to correct the sampled pixel value of any pixel in the tomographic endoscopy image sequence acquired by the endoscope, so as to obtain the endoscope correction image.

[0179] The function construction module 520 is configured to determine the tracking pixel points corresponding to the endoscope correction image and construct a pixel value function using the correction pixel values ​​of each pixel point in the endoscope correction image.

[0180] The matrix determination module 530 is configured to determine the Hessian matrix for each scale window of the pixel value function at the tracked pixel point, using multiple scale windows of different sizes.

[0181] The orientation and center point determination module 540 is configured to determine the orientation and center point of human tissue at the tracking pixel based on the pixel value function and the Hessian matrix corresponding to each scale window.

[0182] The image generation module 550 is configured to connect the center points corresponding to each tracked pixel according to the direction of each tracked pixel to generate an endoscopic target image of human tissue.

[0183] Optionally, the routing and center point determination module 540 includes:

[0184] The decomposition submodule is configured to perform eigenvalue decomposition on the Hessian matrix corresponding to each scale window to obtain the eigenvector matrix and eigenvalue matrix corresponding to each scale window.

[0185] The first determining submodule is configured to determine the absolute value of the eigenvalue of each element in the eigenvalue matrix corresponding to each scale window;

[0186] The second determination submodule is configured to take the eigenvalue matrix containing the eigenvalue corresponding to the eigenvalue with the largest absolute value as the target eigenvalue matrix, and take the eigenvalues ​​in the target eigenvalue matrix that take values ​​approximately 0 as the target eigenvalues.

[0187] The third determination submodule is configured to select the eigenvector in the eigenvector matrix that corresponds to the target eigenvalue as the target eigenvector based on the relationship between the eigenvectors in the eigenvector matrix and the eigenvalues ​​in the eigenvalue matrix.

[0188] The fourth determination submodule is configured to use the direction of the target feature vector as the orientation of the human tissue at the tracking pixel.

[0189] The fifth determination submodule is configured to determine the center point of the human tissue at the tracking pixel based on the direction of the human tissue at the tracking pixel and the pixel value function.

[0190] Optionally, the fifth determining submodule is configured as follows:

[0191] The vertical plane that tracks the direction of human tissue is obtained by tracking the pixels;

[0192] The profile image at the tracking pixel is determined based on the intersection of the vertical plane and the pixel value function.

[0193] Determine the centroid of each pixel in the cross-sectional image and use it as the center point of the human tissue at the tracking pixel.

[0194] Optionally, function construction module 520 includes:

[0195] The sphere construction submodule is configured to construct a sphere with radius R, centered at the center point of the human tissue corresponding to the previous tracked pixel in the tomographic endoscopy image sequence.

[0196] The pixel set construction submodule is configured to construct a pixel set based on target pixels that are simultaneously on the sphere and the pixel value function.

[0197] The clustering submodule is configured to perform clustering and duplicate point removal on the target pixels in the pixel set within each connected region based on the connected regions cut by the sphere surface pixel value function, thereby obtaining unique region pixels within each connected region.

[0198] The sixth determination submodule is configured to determine the connectivity between region pixels and the center point in each connected region, and to use the region pixels whose connectivity is characterized to be connected to the center point as connected pixels.

[0199] The seventh determination submodule is configured to take any connected pixel as the target connected pixel, and advance a preset step size from the center point of the human tissue corresponding to the previous tracked pixel to the target connected pixel in the connected region, so as to obtain the tracked pixel corresponding to the next frame image in the tomographic endoscopy image sequence.

[0200] Optionally, the sixth determining submodule is configured as follows:

[0201] Based on the center point of the human tissue corresponding to the previous tracked pixel, the path of the endoscope is divided into the front path and the back path.

[0202] Use the connected regions on the subsequent path as backup connected regions.

[0203] The spare connected region on the historical path of the endoscope's current image acquisition is used as the target connected region;

[0204] Remove the pixels in the target connected region whose connectivity representation is connected to the center point, and take the remaining pixels in the connected region whose connectivity representation is connected to the center point as connected pixels.

[0205] Optionally, the matrix determination module 530 is configured as follows:

[0206] The Gaussian convolution kernel corresponding to the scale window is determined based on the number of dimensions of the endoscopic image, the scale value corresponding to the scale window, and the transpose corresponding to the corrected pixel value of the tracked pixel.

[0207] In each dimension, determine the partial derivative of the Gaussian convolution kernel corresponding to each scale window;

[0208] Based on the Gaussian convolution kernel and its corresponding partial derivative, a convolution operation is performed on the product of the square of the scale value corresponding to the scale window and the pixel value function to determine the Hessian matrix of the pixel value function for each scale window.

[0209] Optionally, the pixel correction module 510 is configured as follows:

[0210] For any pixel in the tomographic endoscopy image sequence acquired by the endoscope, obtain the sampled pixel value of the pixel, the sampled pixel value of the preceding pixel adjacent to the pixel in the tomographic endoscopy image sequence, and the sampled pixel value of the following pixel adjacent to the pixel.

[0211] The corrected pixel value of a pixel is obtained by calculating the weighted average of the sampled pixel value of the pixel, the sampled pixel value of the previous pixel, and the sampled pixel value of the next pixel.

[0212] The corrected endoscopic image is obtained by using the corrected pixel values ​​corresponding to the pixels in the tomographic endoscopic image sequence.

[0213] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0214] Those skilled in the art should understand that the device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and other division methods may exist in actual implementation. For instance, multiple modules may be combined or integrated into one module. Furthermore, the modules described as separate components may or may not be physically separated. Each module can be implemented wholly or partially through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented wholly or partially in the form of a computer program product. When implemented in hardware, it can be implemented wholly or partially in the form of an integrated circuit or chip.

[0215] This disclosure also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a first processor, implements the steps of the endoscope image analysis method with significantly improved clarity as described in any one of the first aspects.

[0216] This disclosure also provides an electronic device, including:

[0217] The first memory, on which the computer program is stored;

[0218] A second processor is configured to execute the computer program in the first memory to implement the steps of the endoscopic image analysis method with significantly improved clarity as described in any of the first aspects.

[0219] Figure 6 This is a block diagram illustrating an electronic device 600 according to an exemplary embodiment. The electronic device 600 can be configured as an endoscopic image analysis device with significantly improved clarity, such as... Figure 6 As shown, the electronic device 600 may include: a third processor 601 and a second memory 602. The electronic device 600 may also include one or more of the following: a multimedia component 603, an input / output (I / O) interface 604, and a communication component 605.

[0220] The third processor 601 controls the overall operation of the electronic device 600 to complete all or part of the steps in the aforementioned endoscopic image analysis method with significantly improved clarity. The second memory 602 stores various types of data to support the operation of the electronic device 600. This data may include, for example, instructions for any application or method operating on the electronic device 600, and application-related data, such as images. The second memory 602 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. Multimedia component 603 may include a screen and an audio component. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in a second memory 602 or transmitted via communication component 605. The audio component also includes at least one speaker for outputting audio signals. I / O interface 604 provides an interface between the third processor 601 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 605 is used for wired or wireless communication between the electronic device 600 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IoT, eMTC, or other 5G technologies, or combinations thereof, is not limited here. Therefore, the corresponding communication component 605 may include: a Wi-Fi module, a Bluetooth module, an NFC module, etc.

[0221] In an exemplary embodiment, the electronic device 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the aforementioned endoscopic image analysis method with significantly improved clarity.

[0222] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the endoscopic image analysis method with significantly improved clarity described above. For example, the computer-readable storage medium may be the second memory 602 including the program instructions described above, which may be executed by a third processor 601 of the electronic device 600 to complete the endoscopic image analysis method with significantly improved clarity described above.

[0223] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.

[0224] It should also be noted that the various specific technical features described in the above embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.

[0225] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. An endoscope image analysis method with significantly improved sharpness, characterized by, The method comprises the following steps: For any pixel point in the tomoscopy image sequence collected by the endoscope, the sampling pixel value of the pixel point is corrected to obtain an endoscope corrected image; A tracking pixel point corresponding to the endoscope corrected image is determined, and a pixel value function is constructed using the corrected pixel values of each pixel point in the endoscope corrected image; At the tracking pixel point, a plurality of scale windows of different sizes are used to determine the Hessian matrix corresponding to each scale window of the pixel value function; According to the pixel value function and the Hessian matrix corresponding to each scale window, the direction and center point of the human tissue at the tracking pixel point are determined; According to the direction of each tracking pixel point, the center points corresponding to each tracking pixel point are connected to generate an endoscopic target image corresponding to the human tissue; The step of determining the tracking pixel point corresponding to the endoscope corrected image comprises: A sphere with a center point of the human tissue corresponding to the last tracking pixel point in the tomoscopy image sequence as the center and a radius R is constructed; According to the target pixel points on the sphere and the pixel value function, a pixel point set is constructed; According to the connected regions after the sphere cuts the pixel value function, the target pixel points in the pixel point set are clustered and repeated point deletion is performed in each connected region to obtain unique regional pixel points in each connected region; The connectivity of the regional pixel points in each connected region and the center point is determined, and the regional pixel points connected with the center point are taken as connected pixel points; Any connected pixel point is taken as a target connected pixel point, and a preset step length is advanced from the center point of the human tissue corresponding to the last tracking pixel point to the target connected pixel point in the connected region to obtain a tracking pixel point corresponding to the next frame image in the tomoscopy image sequence.

2. The endoscope image analysis method for sharpness saliency enhancement according to claim 1, characterized by, The step of determining the direction and center point of the human tissue at the tracking pixel point according to the pixel value function and the Hessian matrix corresponding to each scale window comprises: Eigenvalue decomposition is performed on the Hessian matrix corresponding to each scale window to obtain a feature vector matrix and an eigenvalue matrix corresponding to each scale window; The absolute values of the eigenvalues of each element in the eigenvalue matrix corresponding to each scale window are determined; The eigenvalue matrix in which the eigenvalue corresponding to the maximum absolute value is taken as a target eigenvalue matrix, and the eigenvalue with a value close to 0 in the target eigenvalue matrix is taken as a target eigenvalue; According to the correlation between the feature vectors in the feature vector matrix and the eigenvalues in the eigenvalue matrix, the feature vector corresponding to the target eigenvalue in the feature vector matrix is taken as a target feature vector; The direction of the target feature vector is taken as the direction of the human tissue at the tracking pixel point; The center point of the human tissue at the tracking pixel point is determined according to the direction of the human tissue at the tracking pixel point and the pixel value function.

3. The endoscope image analysis method for sharpness saliency enhancement according to claim 2, characterized by, The step of determining the center point of the human tissue at the tracking pixel point according to the direction of the human tissue at the tracking pixel point and the pixel value function comprises: a vertical plane perpendicular to the direction of the human tissue at the tracking pixel point; determining a profile image at the tracking pixel point according to an intersection of the vertical plane and the pixel value function; determining a center point of the human tissue at the tracking pixel point as a centroid of the pixel points in the profile image.

4. The endoscope image analysis method for sharpness saliency enhancement according to claim 1, characterized by, The step of determining the connected pixel points as the region pixel points connected with the center point according to the connectivity representation, comprises: dividing a path of the endoscope into a front path and a rear path according to the center point of the human tissue corresponding to the last tracking pixel point, wherein the front path is a path that has not been explored by the endoscope, and the rear path is a path behind the endoscope in the current advancing process; determining a connected region on the rear path as a standby connected region; determining a standby connected region on a historical path of the endoscope in the current image acquisition process as a target connected region; removing region pixel points connected with the center point from the target connected region according to the connectivity representation, and determining the remaining region pixel points connected with the center point as the connected pixel points.

5. The endoscope image analysis method for sharpness saliency enhancement according to claim 1, characterized by, The step of determining the Hessian matrix corresponding to each scale window of the pixel value function at the tracking pixel point according to a plurality of preset scale windows, comprises: determining a Gaussian convolution kernel corresponding to each scale window according to a number of dimensions of the tomographic endoscope image, a scale value corresponding to the scale window, and a transpose corresponding to the modified pixel value of the tracking pixel point; determining a partial derivative of the Gaussian convolution kernel corresponding to each scale window in each dimension; performing convolution operation on a product of a square of the scale value corresponding to the scale window and the pixel value function according to the Gaussian convolution kernel and the corresponding partial derivative, to determine the Hessian matrix corresponding to each scale window of the pixel value function.

6. The endoscope image analysis method for sharpness saliency enhancement according to any one of claims 1 to 5, characterized by, The step of modifying a sampling pixel value of any pixel point in a tomographic endoscope image sequence collected by an endoscope to obtain an endoscope modified image, comprises: obtaining the sampling pixel value of the pixel point, a sampling pixel value of a previous pixel point adjacent to the pixel point, and a sampling pixel value of a next pixel point adjacent to the pixel point in the tomographic endoscope image sequence collected by the endoscope; calculating a weighted average of the sampling pixel value of the pixel point, the sampling pixel value of the previous pixel point, and the sampling pixel value of the next pixel point to obtain a modified pixel value of the pixel point; obtaining the endoscope modified image according to the modified pixel value corresponding to the pixel point in the tomographic endoscope image sequence.

7. An endoscope image analysis apparatus with significantly improved definition, characterized by comprises: a pixel modification module configured to modify a sampling pixel value of any pixel point in a tomographic endoscope image sequence collected by an endoscope to obtain an endoscope modified image; a function construction module configured to determine a tracking pixel point corresponding to the endoscope modified image, and to construct a pixel value function with modified pixel values of the pixel points in the endoscope modified image; The matrix determination module is configured to determine, at the tracking pixel point, a plurality of scale windows with different sizes, a Hessian matrix corresponding to each scale window for the pixel value function; The direction and center point determination module is configured to determine the direction and center point of the human tissue at the tracking pixel point according to the pixel value function and the Hessian matrix corresponding to each scale window; The image generation module is configured to connect the center points corresponding to each tracking pixel point according to the direction of each tracking pixel point, to generate an endoscopic target image corresponding to the human tissue. The function construction module is configured to: take the center point of the human tissue corresponding to the previous tracking pixel point in the tomographic endoscopic image sequence as the center of a sphere, and construct a sphere with a radius R; According to the target pixel point on the sphere and the pixel value function, a pixel point set is constructed; According to the connected region after the sphere cuts the pixel value function, the target pixel point in the pixel point set is clustered and repeated point deletion is performed in each connected region, to obtain unique regional pixel points in each connected region; The connectivity between the regional pixel points and the center point in each connected region is determined, and the regional pixel points connected with the center point are taken as connected pixel points; Any connected pixel point is taken as a target connected pixel point, and a preset step length is advanced from the center point of the human tissue corresponding to the previous tracking pixel point to the target connected pixel point in the connected region, to obtain a tracking pixel point corresponding to a next frame of image in the tomographic endoscopic image sequence.

8. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the first processor to implement the steps of the endoscopic image analysis method with improved definition significance according to any one of claims 1-6.

9. An electronic device, comprising: Comprise: A first memory having a computer program stored thereon; A second processor configured to execute the computer program in the first memory to implement the steps of the endoscopic image analysis method with improved definition significance according to any one of claims 1-6.

Images