Esophageal squamous intraepithelial neoplasia image recognition contrast method
By quantifying vascular and cellular characteristics and combining hemodynamic and SMS features, a multi-dimensional identification model was constructed, which solved the problems of subjectivity and neglect of dynamic features in the diagnosis of esophageal squamous intraepithelial neoplasia and achieved efficient and accurate lesion identification.
Patent Information
- Application Number
- CN202610614280.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-05-07
- Publication Date
- 2026-07-31
AI Technical Summary
Existing technologies for the diagnosis of esophageal squamous intraepithelial neoplasia suffer from problems such as strong subjectivity, reliance on qualitative description, neglect of dynamic features, and lack of multi-feature fusion mechanisms, resulting in insufficient accuracy and robustness in identification.
By acquiring dynamic video images, the hemodynamic recognition unit quantifies the vascular respiration state, and combined with cell morphology and SMS features, a machine learning/deep learning recognition model is constructed to achieve multi-dimensional quantification and fusion recognition of vascular morphology, cell morphology and fusion features.
It improves the objectivity and consistency of diagnosis of esophageal squamous intraepithelial neoplasia, reduces the interference of respiratory motion, enhances the ability to identify diseased blood vessels, improves the stability and accuracy of identification, and realizes real-time automatic lesion identification.
Smart Images
Figure CN122492607A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical artificial intelligence technology, and in particular to a method for image recognition and comparison of esophageal squamous intraepithelial neoplasia. Background Technology
[0002] Esophageal squamous intraepithelial neoplasia (SIN) is a precancerous lesion of esophageal cancer, and early diagnosis is crucial for treatment and prognosis. Currently, confocal endoscopy (CLE) plays a vital role in real-time imaging of the esophageal mucosa, particularly in observing intraepithelial vascular morphology and cellular structure. Among existing techniques, the surface maturations score (SMS) has been proposed as a diagnostic reference method based on CLE images. This method qualitatively scores four surface maturations phenomena—"halo," "gradient," "polarity," and "compass effect"—ranging from 0 to 4 points, with an SMS score of 0 serving as a diagnostic criterion for esophageal SIN. Based on this, the surface maturations score can be applied to planar CLE images and provides a simple and easy-to-use diagnostic standard for CLE in clinical practice. However, the SMS method has the following main problems: 1. Highly subjective: SMS relies on the doctor's visual judgment of image features and is easily influenced by the observer's experience and subjective consciousness; 2. Primarily qualitative description: SMS does not perform quantitative analysis on features such as blood vessel morphology and cell dynamic changes, and cannot fully utilize the multi-dimensional information in the images; 3. Ignoring dynamic features: The dynamic changes in blood vessels and cells caused by physiological activities such as respiration are not considered, which affects the accuracy and robustness of identification; 4. Lack of multi-feature fusion mechanism: Existing methods do not systematically integrate vascular morphology, cell morphology and SMS features, making it difficult to achieve multi-dimensional comprehensive judgment. Summary of the Invention
[0003] The purpose of this invention is to provide an image recognition and comparison method for esophageal squamous intraepithelial neoplasia, so as to solve or at least partially solve the above-mentioned technical problems existing in the prior art.
[0004] To achieve the above-mentioned objectives, the technical solution provided by this invention is as follows: A method for image recognition and comparison of esophageal squamous intraepithelial neoplasia, the method comprising: S101. Acquire dynamic video images through a confocal endoscope and input them into a hemodynamic-based respiratory state recognition unit to identify the vascular respiratory state in different image frames of the dynamic video images. S102. Input all image frames of the dynamic video and their corresponding vascular respiration status into the vascular anomaly quantization unit to calculate the morphology-eddy current fusion value of the blood vessel. S103. Input all image frames of the dynamic video and their corresponding vascular respiration status into the cell heteromorphism quantification unit to calculate the cell morphology quantification value. S104. Input all image frames of the dynamic video and their corresponding vascular respiration status into the SMS feature quantization unit to calculate the SMS features of the blood vessels. S105. Input the calculation results of steps S102 to S104 into the comparison and recognition unit, and construct and train machine learning / deep learning recognition models based on cell morphology, blood vessel morphology, SMS features and fusion features respectively to identify esophageal squamous epithelial tumors.
[0005] Furthermore, step S101 specifically includes the following operations: S201. Identify the blood flow velocity in blood vessels in dynamic video images; S202. Based on the periodicity of blood flow velocity, the time period from the start to the end of the dynamic video is divided into several respiratory cycles. S203. For each respiratory cycle, identify its inspiratory and expiratory phases; S204. Identify the end point of the inhalation phase; S205. Identify the end-expiratory point of the expiratory phase.
[0006] Furthermore, step S201 specifically includes the following operations: S301. Divide the dynamic video image into multiple static image frames; S302. Input the image frame into the trained image segmentation model to segment out the blood vessel region in the image frame; S303. Using an image tracking algorithm, the displacement of red blood cells in each blood vessel in all vascular regions is calculated, and the blood flow velocity is further calculated by combining the time taken for the red blood cell displacement.
[0007] Furthermore, step S102 specifically includes the following operations: S401. Calculate the dynamic tortuosity value and dynamic area correction coefficient of the blood vessel through the tortuosity quantization module; S402. Calculate the dynamic Reynolds correction value of all blood vessels along the vessel centerline using the Reynolds number correction module; S403. Calculate the morphology-eddy current fusion value of the blood vessel based on the dynamic tortuosity value, dynamic area correction coefficient, and dynamic Reynolds correction value.
[0008] Furthermore, the bending quantization module is specifically used to perform the following operations: S501. Determine the tortuosity value of each blood vessel at the end of inhalation and the end of exhalation, and further calculate the dynamic tortuosity value of the blood vessel based on the tortuosity value at the end of inhalation and the end of exhalation. S502. Calculate the cross-sectional area correction coefficient of each blood vessel at the end of inhalation and the end of exhalation, and calculate the dynamic area correction coefficient of the blood vessel based on the cross-sectional area correction coefficient of the blood vessel at the end of inhalation and the end of exhalation.
[0009] Furthermore, the cell heterogeneity quantification unit is specifically used to perform the following operations: S601. The cell displacement amplitude quantization module calculates the quantized value of the cell displacement amplitude based on the coordinate position of the cell in different respiratory cycles. S602. Using the deformation recovery rate quantification module, calculate the morphological recovery rate quantification value of the cell based on the ratio of the long axis to the short axis of the cell at the end of inspiration and the end of expiration in each respiratory cycle, as well as the average value of the ratio of the long axis to the short axis of the cell throughout the entire respiratory cycle. S603. Calculate the dynamic sharpness quantization value of each cell's local image using the dynamic sharpness index quantization module.
[0010] Furthermore, step S104 specifically includes the following operations: S701. Calculate the halo attenuation rate of the corresponding area for each blood vessel using the halo attenuation module; S702. Calculate the cell thickness attenuation index of the corresponding region of each blood vessel through the cell thickness thinning module. S703. Calculate the ratio of the major and minor axes of the ellipse fitting of the halo region corresponding to each blood vessel through the halo extension module, and record it as the halo major and minor axis ratio. S704. Calculate the variance of the principal direction angles of all IPCL halos.
[0011] Furthermore, a machine learning / deep learning recognition model based on cell morphology, blood vessel morphology, and SMS features is constructed and trained, specifically including the following operations: S801. Decompose the CLE images of multiple examiners into frames to obtain CLE images for constructing training and test sets; S802. Based on the blood vessel morphology-eddy current fusion value calculation method in step S102, a one-dimensional vector of blood vessel morphology is generated for each CLE image. The training set is input into a machine learning model or a deep learning model for training to obtain a recognition model based on blood vessel morphology. The test set is then classified to obtain the classification accuracy. S803. Based on the cell morphology quantification calculation method in step S103, a two-dimensional cell morphology matrix is generated for each CLE image. The training set is input into a machine learning model or a deep learning model for training to obtain a cell morphology-based recognition model. The test set is then classified to obtain the classification accuracy. S804. Based on the SMS feature calculation method in step S104, generate a two-dimensional matrix of SMS features for each CLE image, input the training set into the machine learning model or deep learning model for training, obtain a recognition model based on SMS features, and perform classification recognition on the test set to obtain the classification accuracy.
[0012] Furthermore, a machine learning / deep learning recognition model based on fused features is constructed and trained, specifically including the following operations: S901. Transpose the one-dimensional vector of blood vessel morphology into a column vector, and then concatenate and expand it with the two-dimensional matrix of SMS features to obtain the expanded two-dimensional matrix. S902. Based on the extended two-dimensional matrix and the cell morphology two-dimensional matrix, a dual-input model is constructed. The dual input channels of the dual-input model adopt a 2D CNN structure, and then feature concatenation is performed to obtain a recognition model based on fused features. S903. The classification accuracy is obtained by classifying and recognizing the test set through a recognition model based on fusion features.
[0013] Furthermore, in step S105, esophageal squamous epithelioma is identified by comparing the classification accuracy of identification models based on cell morphology, blood vessel morphology, SMS features, and fusion features on the same test set, and using the identification model with the highest classification accuracy as the final identification model for identifying esophageal squamous epithelioma.
[0014] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention uses a vascular atypia quantification unit, a cell atypia quantification unit, and an SMS feature quantification unit to dynamically quantify vascular morphology, cell morphology, and surface maturation characteristics, respectively, transforming traditional qualitative descriptions into calculable quantitative indicators, which significantly improves the objectivity and consistency of diagnosis.
[0015] 2. This invention innovatively combines vascular morphology with hemodynamic parameters. By calculating the morphology-eddy current fusion value, it more comprehensively characterizes the dynamic changes of blood vessels during the respiratory cycle, thereby enhancing the ability to identify diseased blood vessels.
[0016] 3. This invention uses hemodynamic-based respiratory state recognition to extract image features at key time points such as the end of inspiration and the end of expiration, effectively reducing the interference of respiratory motion on image analysis and improving the stability and reliability of feature extraction.
[0017] 4. This invention provides multiple recognition paths based on blood vessel morphology, cell morphology, SMS features, and multi-feature fusion, and sets up a recognition comparison unit for model training and testing. By comparing the recognition effects of different approaches, the optimal recognition strategy can be selected to improve the overall recognition performance and robustness of the system.
[0018] 5. The method provided by this invention can be integrated into a CLE image processing system to achieve real-time and automatic lesion identification, reduce reliance on doctors' experience, and help to achieve rapid and accurate screening and diagnosis of esophageal squamous intraepithelial neoplasia in clinical practice. Attached Figure Description
[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only preferred embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0020] Figure 1 This is a schematic diagram of the overall structure of an image recognition and comparison method for esophageal squamous intraepithelial neoplasia provided in an embodiment of the present invention.
[0021] Figure 2 This is a schematic diagram of halo attenuation quantization provided in an embodiment of the present invention.
[0022] Figure 3 This is a schematic diagram of the dual-input model structure provided in an embodiment of the present invention. Detailed Implementation
[0023] The principles and features of the present invention are described below with reference to the accompanying drawings. The listed embodiments are only used to explain the present invention and are not intended to limit the scope of the present invention.
[0024] Reference Figure 1 This embodiment provides a method for image recognition and comparison of esophageal squamous intraepithelial neoplasia, the method comprising the following steps: S101. Acquire dynamic video images through a confocal endoscope and input them into a hemodynamic-based respiratory state recognition unit to identify the vascular respiratory state in different image frames of the dynamic video images.
[0025] Confocal endoscopic images are significantly affected by factors such as respiration, especially vasoconstriction and vasodilation, and changes in cell morphology. This step involves analyzing the acquired dynamic video images within the t0-t1 time period. Note that the t0-t1 time period must include at least several complete respiratory cycles, with a minimum of three complete cycles.
[0026] In a preferred embodiment, step S101 specifically includes the following operations: S201. Identify the blood flow velocity in blood vessels within a dynamic video image. This specifically includes the following operations: S301. Divide the dynamic video into multiple static image frames.
[0027] S302. Input the image frame into the trained image segmentation model to segment out the blood vessel region in the image frame.
[0028] S303. Using an image tracking algorithm, the displacement of red blood cells in each blood vessel in all vascular regions is monitored. Calculations were performed, and the blood flow velocity was further calculated by combining the time taken for red blood cell displacement. .
[0029] S202. Based on the periodicity of blood flow velocity, the time interval t0~t1 from the start to the end of the dynamic video is divided into several respiratory cycles. A complete respiratory cycle includes an inhalation phase and an exhalation phase.
[0030] S203. For each respiratory cycle, identify its inspiratory and expiratory phases, which can be represented as:
[0031] in, t This indicates a change in velocity, equivalent to acceleration. Based on experience, its value can be set to .
[0032] S204. Identify the end point of the inhalation phase.
[0033] In this embodiment, the inhalation phase is satisfied with This indicates that the moment is the end of the inhalation. This represents the blood flow velocity at time t.
[0034] S205. Identify the end-expiratory point of the expiratory phase.
[0035] In this embodiment, the exhalation phase is satisfied with This is the point at the end of exhalation.
[0036] S102. Input all image frames of the dynamic video and their corresponding vascular respiration status into the vascular anomaly quantization unit to calculate the morphology-eddy current fusion value of the blood vessels.
[0037] In a preferred embodiment, step S102 specifically includes the following operations: S401. Calculate the dynamic tortuosity value and dynamic area correction coefficient of blood vessels through the tortuosity quantization module.
[0038] In this embodiment, the bending quantization module is specifically used to perform the following operations: S501. Determine the tortuosity value of each blood vessel at the end of inhalation and the end of exhalation, and further calculate the dynamic tortuosity value of the blood vessel based on the tortuosity value at the end of inhalation and the end of exhalation.
[0039] The formula for calculating the bending value is as follows:
[0040] In the above formula, Let be the length of the i-th blood vessel. Let be the area of the minimum bounding rectangle of the i-th blood vessel. Then the dynamic curvature value of the i-th blood vessel = The dynamic bending value is the result of dividing the bending value of the i-th blood vessel at the end of inspiration by the bending value at the end of expiration.
[0041] S502. Calculate the cross-sectional area correction coefficient of each blood vessel at the end of inhalation and the end of exhalation, and calculate the dynamic area correction coefficient of the blood vessel based on the cross-sectional area correction coefficient of the blood vessel at the end of inhalation and the end of exhalation.
[0042] In this embodiment, the cross-sectional area correction factor along the blood vessel centerline direction... The calculation formula is as follows:
[0043] In the above formula, J represents the total number of pixel position indices on the center line of the blood vessel, j represents the pixel position index on the center line of the blood vessel, and mean(·) represents the average value. Furthermore, the dynamic area correction coefficient for the i-th blood vessel = The dynamic area correction coefficient is the result of dividing the cross-sectional area correction coefficient at the end of inhalation of the i-th blood vessel by the cross-sectional area correction coefficient at the end of exhalation.
[0044] S402. Calculate the dynamic Reynolds correction value of all blood vessels along the vessel centerline using the Reynolds number correction module.
[0045] In this embodiment, the formula for calculating the Reynolds correction factor for blood flow at the current location along the centerline of the blood vessel is as follows:
[0046] In the above formula, ρ≈ plasma density (e.g., ρ can be taken as 1.025 g / cm³). = Flow velocity (μm / s), D = vessel diameter (μm), η ≈ plasma viscosity (e.g., η can be 1.2 mPa·s), where D is calculated based on the Zhang-Suen thinning algorithm for the centerline of each vessel region. A line is drawn through each point on the centerline and intersects the vessel wall at two points. The Euclidean distance between these two points is the diameter D of the vessel segment at its current position. Here, i represents the i-th vessel, and j represents the position of a pixel on the centerline. Then, the dynamic Reynolds correction value for the i-th vessel = ,in .
[0047] S403. Calculate the morphology-eddy current fusion value of the blood vessel based on the dynamic tortuosity value, dynamic area correction coefficient, and dynamic Reynolds correction value.
[0048] In this embodiment, the morphology-eddy current fusion value of the i-th blood vessel The calculation formula is as follows:
[0049] In the above formula, The curvature sensitivity coefficient of the i-th blood vessel is usually... .
[0050] S103. Input all image frames of the dynamic video and their corresponding vascular respiration status into the cell heteromorphism quantification unit to calculate the cell morphology quantification value.
[0051] In a preferred embodiment, the cell heterogeneity quantification unit is specifically used to perform the following operations: S601. The cell displacement amplitude quantization module calculates the quantized value of the cell displacement amplitude based on the coordinate position of the cell in different respiratory cycles.
[0052] Displacement amplitude quantization value = , where N is the total number of respiratory cycles, k is the kth respiratory cycle, and i is the ith cell. This represents the x-axis coordinate position of the i-th blood vessel during the k-th respiratory cycle. This represents the y-axis coordinate position of the i-th blood vessel during the k-th respiratory cycle.
[0053] S602. Using the deformation recovery rate quantification module, calculate the quantified value of cell morphology recovery rate based on the ratio of the long axis to the short axis of cells at the end of inspiration and the end of expiration in each respiratory cycle, as well as the average value of the ratio of the long axis to the short axis of cells throughout the entire respiratory cycle.
[0054] Quantitative value of morphological recovery rate = , This represents the mean ratio of the long axis to the short axis of the cell in the i-th blood vessel during the k-th respiratory cycle. This represents the ratio of the long axis to the short axis of the cell at the end of the inspiratory phase in the i-th blood vessel during the k-th respiratory cycle. The ratio of the long axis to the short axis of the cells in the i-th blood vessel at the end of the expiration phase of the k-th respiratory cycle.
[0055] S603. Calculate the dynamic sharpness quantization value of each cell's local image using the dynamic sharpness index quantization module.
[0056] For each cell local image, its single-frame sharpness quantification The expression is as follows:
[0057] In the above formula, This is the second inverse of the edge-enhanced image. Var (·) represents variance. Then the dynamic sharpness metric for each cell's local image = .in This represents the clarity threshold at the end of inhalation. This represents the clarity threshold at the end of expiration. and It can be obtained through specific experiments. This represents the single-frame sharpness quantization value at the end of the inhalation time. This represents the total number of frames in all local cell images analyzed at the end of inhalation.
[0058] S104. Input all image frames of the dynamic video and their corresponding vascular respiration status into the SMS feature quantization unit to calculate the SMS features of the blood vessels.
[0059] In a preferred embodiment, step S104 specifically includes the following operations: S701, calculate the halo attenuation rate of the corresponding area of each blood vessel through the halo attenuation module.
[0060] In this implementation, the halo attenuation module first locates the centroid of each blood vessel based on threshold segmentation, edge detection, or a deep learning model, such as... Figure 2 Point O in the image is used. Then, samples are taken at equal angular intervals from the center outwards. To avoid jagged edges, sub-pixel interpolation can be performed based on cv2.remap. Next, the gradient or attenuation rate is calculated using a fitted curve, with the corresponding formula as follows:
[0061] In the above formula, Indicates the halo attenuation gradient exponent. Indicates the intensity value at the center of the blood vessel. This represents the intensity value at a distance d. This represents the diameter of the blood vessel. Finally, the halo range is determined; the corresponding expression is:
[0062] In the above formula, Indicates the effective radius of the halo. This represents the gray intensity at a radial distance r. This represents the average grayscale value of the background. represents the standard deviation of background grayscale, and r represents the radial distance variable.
[0063] S702. Calculate the cell thickness attenuation index of the corresponding region of each blood vessel through the cell thickness thinning module.
[0064] In this embodiment, the cell thickness thinning module first overlays the blood vessel segmentation map and the cell segmentation map obtained in step S302 to obtain an overlay map, and then fills the overlay map with the pixel values from the original CLE image frame to obtain a filled blood vessel-cell segmentation map. For the filled blood vessel-cell segmentation map, concentric ring regions (each ring width 5-10 μm) are divided outwards along the blood vessel edge, and the average grayscale gradient of each ring is calculated; then, a fitted exponential decay model is calculated. , This represents the average gray intensity value at a radial distance d from the edge of the blood vessel, and the attenuation coefficient k is a quantification index.
[0065] S703. Calculate the ratio of the major and minor axes of the ellipse fitting of the halo region corresponding to each blood vessel through the halo extension module, and record it as the halo major and minor axis ratio.
[0066] S704. Calculate the variance of the principal direction angles of all IPCL halos.
[0067] S105. Input the calculation results of steps S102 to S104 into the comparison and recognition unit, and construct and train machine learning / deep learning recognition models based on cell morphology, blood vessel morphology, SMS features and fusion features respectively to identify esophageal squamous epithelial tumors.
[0068] As a preferred implementation, a machine learning / deep learning recognition model based on cell morphology, blood vessel morphology, and SMS features is constructed and trained, specifically including the following operations: S801. Decompose the CLE images of multiple examiners into frames to obtain CLE images for constructing training and test sets.
[0069] S802. Based on the blood vessel morphology-eddy current fusion value calculation method in step S102, a one-dimensional vector of blood vessel morphology is generated for each CLE image. The length of the one-dimensional vector of blood vessel morphology can be set to a finite size, such as 1000 (this is just an example, and a reasonable value can be set according to the actual situation). If the number of blood vessel roots in each image in the training set and the test set is less than 1000, the remaining positions are padded with 0. The training set is input into a machine learning model or a deep learning model for training to obtain a blood vessel morphology-based recognition model. The test set is then used for classification recognition to obtain the classification accuracy.
[0070] S803. Based on the cell morphology quantification calculation method in step S103, a two-dimensional cell morphology matrix is generated for each CLE image. The two-dimensional cell morphology matrix has 3 columns (corresponding to the displacement amplitude quantization value, morphology restoration rate quantization value, and dynamic clarity quantization value, respectively). The number of rows in the two-dimensional cell morphology matrix is the number of cells, which can be set to a finite size, such as a length of 1000 (this is just an example; a reasonable value can be set according to the actual situation). If the number of cells in each image in the training set and the test set is less than 1000, the remaining rows are padded with 0. The training set is input into a machine learning model or a deep learning model for training to obtain a cell morphology-based recognition model. The test set is then used for classification to obtain the classification accuracy. When using the machine learning model, the two-dimensional cell morphology matrix is flattened into a one-dimensional vector before being input into the machine learning model.
[0071] S804. Based on the SMS feature calculation method in step S104, an SMS feature two-dimensional matrix is generated for each CLE image. Specifically, based on the image frame obtained in step S103, the halo attenuation rate, cell thickness attenuation index, halo major-minor axis ratio, and halo polarity angle variance corresponding to I blood vessels can be obtained, resulting in an I-row, 4-column SMS feature two-dimensional matrix, where the fourth column of each row is fixed as the halo polarity angle variance. The number of rows in the SMS feature matrix is the number of cells, which can be set to a finite size, such as a length of 1000 (this is just an example; a reasonable value can be set according to the actual situation). If the number of cells in each image in the training and test sets is less than 1000, the remaining rows are padded with 0s. The training set is input into a machine learning model (such as SVM, random forest, etc.) or a deep learning model (such as 2DCNN, etc.) for training to obtain a recognition model based on SMS features. The test set is then used for classification to obtain the classification accuracy. When using machine learning, the SMS feature two-dimensional matrix is flattened into a one-dimensional vector before being input into the machine learning model.
[0072] Building and training a machine learning / deep learning recognition model based on fused features includes the following operations: S901. Transpose the one-dimensional vector of blood vessel morphology of length I obtained in step S802 into a column vector, and concatenate it with the SMS feature two-dimensional matrix of length I to obtain an expanded two-dimensional matrix of length I and length 5. The number of rows in the expanded two-dimensional matrix is the number of blood vessel roots, which can be set to a finite size, such as a length of 1000 (this is just an example, and a reasonable value can be set according to the actual situation). If the number of cells in each image in the training set and the test set does not reach 1000, the remaining rows are padded with 0.
[0073] S902. Based on the extended two-dimensional matrix and the cell morphology two-dimensional matrix, a dual-input model is constructed. The dual input channels of the dual-input model adopt a 2D CNN structure, and then feature concatenation is performed to obtain a recognition model based on fused features.
[0074] For example, the feature concatenation uses the Concate feature concatenation method, and the dual-input model structure is as follows: Figure 3 As shown.
[0075] S903. The classification accuracy is obtained by classifying and recognizing the test set through a recognition model based on fusion features.
[0076] Based on the aforementioned embodiments, in step S105, esophageal squamous epithelioma is identified by comparing the classification accuracy of identification models based on cell morphology, blood vessel morphology, SMS features, and fusion features on the same test set, and using the identification model with the highest classification accuracy as the final identification model for identifying esophageal squamous epithelioma.
[0077] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. An esophageal squamous intraepithelial neoplasia image recognition contrast method, characterized by, The method includes: S101. Acquire dynamic video images through a confocal endoscope and input them into a hemodynamic-based respiratory state recognition unit to identify the vascular respiratory state in different image frames of the dynamic video images. S102. Input all image frames of the dynamic video and their corresponding vascular respiration status into the vascular anomaly quantization unit to calculate the morphology-eddy current fusion value of the blood vessel. S103. Input all image frames of the dynamic video and their corresponding vascular respiration status into the cell heteromorphism quantification unit to calculate the cell morphology quantification value. S104. Input all image frames of the dynamic video and their corresponding vascular respiration status into the SMS feature quantization unit to calculate the SMS features of the blood vessels. S105. Input the calculation results of steps S102 to S104 into the comparison and recognition unit, and construct and train machine learning / deep learning recognition models based on cell morphology, blood vessel morphology, SMS features and fusion features respectively to identify esophageal squamous epithelial tumors.
2. The method for image recognition and comparison of esophageal squamous intraepithelial neoplasia according to claim 1, characterized in that, Step S101 specifically includes the following operations: S201. Identify the blood flow velocity in blood vessels in dynamic video images; S202. Based on the periodicity of blood flow velocity, the time period from the start to the end of the dynamic video is divided into several respiratory cycles. S203. For each respiratory cycle, identify its inspiratory and expiratory phases; S204. Identify the end point of the inhalation phase; S205. Identify the end-expiratory point of the expiratory phase.
3. The method for image recognition and comparison of esophageal squamous intraepithelial neoplasia according to claim 2, characterized in that, Step S201 specifically includes the following operations: S301. Divide the dynamic video image into multiple static image frames; S302. Input the image frame into the trained image segmentation model to segment out the blood vessel region in the image frame; S303. Using an image tracking algorithm, the displacement of red blood cells in each blood vessel in all vascular regions is calculated, and the blood flow velocity is further calculated by combining the time taken for the red blood cell displacement.
4. The method for image recognition and comparison of esophageal squamous intraepithelial neoplasia according to claim 3, characterized in that, Step S102 specifically includes the following operations: S401. Calculate the dynamic tortuosity value and dynamic area correction coefficient of the blood vessel through the tortuosity quantization module; S402. Calculate the dynamic Reynolds correction value of all blood vessels along the vessel centerline using the Reynolds number correction module; S403. Calculate the morphology-eddy current fusion value of the blood vessel based on the dynamic tortuosity value, dynamic area correction coefficient, and dynamic Reynolds correction value.
5. The method for image recognition and comparison of esophageal squamous intraepithelial neoplasia according to claim 4, characterized in that, The bending quantization module is specifically used to perform the following operations: S501. Determine the tortuosity value of each blood vessel at the end of inhalation and the end of exhalation, and further calculate the dynamic tortuosity value of the blood vessel based on the tortuosity value at the end of inhalation and the end of exhalation. S502. Calculate the cross-sectional area correction coefficient of each blood vessel at the end of inhalation and the end of exhalation, and calculate the dynamic area correction coefficient of the blood vessel based on the cross-sectional area correction coefficient of the blood vessel at the end of inhalation and the end of exhalation.
6. The method for image recognition and comparison of esophageal squamous intraepithelial neoplasia according to claim 5, characterized in that, The cell heterogeneity quantification unit is specifically used to perform the following operations: S601. The cell displacement amplitude quantization module calculates the quantized value of the cell displacement amplitude based on the coordinate position of the cell in different respiratory cycles. S602. Using the deformation recovery rate quantification module, calculate the morphological recovery rate quantification value of the cell based on the ratio of the long axis to the short axis of the cell at the end of inspiration and the end of expiration in each respiratory cycle, as well as the average value of the ratio of the long axis to the short axis of the cell throughout the entire respiratory cycle. S603. Calculate the dynamic sharpness quantization value of each cell's local image using the dynamic sharpness index quantization module.
7. The method for image recognition and comparison of esophageal squamous intraepithelial neoplasia according to claim 6, characterized in that, Step S104 specifically includes the following operations: S701. Calculate the halo attenuation rate of the corresponding area for each blood vessel using the halo attenuation module; S702. Calculate the cell thickness attenuation index of the corresponding region of each blood vessel through the cell thickness thinning module. S703. Calculate the ratio of the major and minor axes of the ellipse fitting of the halo region corresponding to each blood vessel through the halo extension module, and record it as the halo major and minor axis ratio. S704. Calculate the variance of the principal direction angles of all IPCL halos.
8. The method for image recognition and comparison of esophageal squamous intraepithelial neoplasia according to claim 7, characterized in that, Constructing and training a machine learning / deep learning recognition model based on cell morphology, blood vessel morphology, and SMS features includes the following operations: S801. Decompose the CLE images of multiple examiners into frames to obtain CLE images for constructing training and test sets; S802. Based on the blood vessel morphology-eddy current fusion value calculation method in step S102, a one-dimensional vector of blood vessel morphology is generated for each CLE image. The training set is input into a machine learning model or a deep learning model for training to obtain a recognition model based on blood vessel morphology. The test set is then classified to obtain the classification accuracy. S803. Based on the cell morphology quantification calculation method in step S103, a two-dimensional cell morphology matrix is generated for each CLE image. The training set is input into a machine learning model or a deep learning model for training to obtain a cell morphology-based recognition model. The test set is then classified to obtain the classification accuracy. S804. Based on the SMS feature calculation method in step S104, generate a two-dimensional matrix of SMS features for each CLE image, input the training set into the machine learning model or deep learning model for training, obtain a recognition model based on SMS features, and perform classification recognition on the test set to obtain the classification accuracy.
9. The method for image recognition and comparison of esophageal squamous intraepithelial neoplasia according to claim 8, characterized in that, Building and training a machine learning / deep learning recognition model based on fused features includes the following operations: S901. Transpose the one-dimensional vector of blood vessel morphology into a column vector, and then concatenate and expand it with the two-dimensional matrix of SMS features to obtain the expanded two-dimensional matrix. S902. Based on the extended two-dimensional matrix and the cell morphology two-dimensional matrix, a dual-input model is constructed. The dual input channels of the dual-input model adopt a 2D CNN structure, and then feature concatenation is performed to obtain a recognition model based on fused features. S903. The classification accuracy is obtained by classifying and recognizing the test set through a recognition model based on fusion features.
10. The method for image recognition and comparison of esophageal squamous intraepithelial neoplasia according to claim 9, characterized in that, In step S105, esophageal squamous epithelioma is identified by comparing the classification accuracy of identification models based on cell morphology, blood vessel morphology, SMS features, and fusion features on the same test set, and using the identification model with the highest classification accuracy as the final identification model for identifying esophageal squamous epithelioma.