Artificial intelligence-based confocal endoscopy method for early cancer risk assessment in digestive tract

Abstract

The present application relates to the technical field of early cancer risk assessment of digestive tract, and particularly relates to a confocal endoscope early cancer risk assessment method of digestive tract based on artificial intelligence. The method acquires macroscopic images and microscopic images of the digestive tract and a reference position of the confocal probe in the macroscopic images in real time; according to the differences in gray scale and reference position distribution between the macroscopic images at the current moment and the previous adjacent moment and the differences in shape and position distribution of the cell regions in the microscopic images at the current moment and the previous adjacent moment, a comprehensive relative displacement vector of the confocal probe at the current moment is acquired; based on the reference position at the previous adjacent moment of the current moment and the comprehensive relative displacement vector, a corrected reference position of the confocal probe in the macroscopic images at the current moment is acquired. The present application acquires the corrected reference position of the confocal probe in the macroscopic images in real time, and effectively improves the accuracy of the early cancer lesion site positioning of the digestive tract.

Description

Technical Field

[0001] This invention relates to the field of early gastrointestinal cancer risk assessment technology, specifically to an artificial intelligence-based confocal endoscopy method for early gastrointestinal cancer risk assessment. Background Technology

[0002] Early gastrointestinal cancer refers to early-stage malignant tumors occurring in the mucosal and submucosal layers of the digestive tract. The cancerous tissue has a shallow depth of invasion and has not metastasized extensively. Minimally invasive treatments such as endoscopic resection can achieve extremely high cure rates and good prognoses. Therefore, early detection and accurate risk assessment of early gastrointestinal cancer are crucial for reducing the mortality rate of gastrointestinal cancers and saving social medical costs.

[0003] Currently, the mainstream clinical approach for assessing early gastrointestinal cancers is the combination of conventional endoscopy and probe-type confocal laser microscopy. The confocal probe can be inserted into the digestive tract through the biopsy channel of the conventional endoscope, enabling collaborative work. Typically, the area where the confocal probe is located is marked in real-time on the macroscopic image of the conventional endoscope to assist physicians in spatial localization. However, during actual examinations, physiological peristalsis occurs in the patient's digestive tract, causing the microscopic area captured by the confocal probe to shift relative to the fixed position of the conventional endoscope lens. This results in the inability to correlate microscopic cellular abnormalities with macroscopic mucosal locations, increasing the risk of localization errors in early gastrointestinal cancers. Furthermore, the confocal probe is designed to adhere to the mucosa for imaging and is not fixedly connected to it. The mucosal surface has a certain degree of smoothness, and when peristalsis causes mucosal displacement, relative sliding can easily occur between the probe and the mucosa. This leads to a discrepancy between the cellular area actually observed by the probe and the expected area calculated based solely on overall digestive tract peristalsis, further increasing the risk of localization errors in early gastrointestinal cancers. Therefore, existing methods for assessing early gastrointestinal cancers cannot accurately locate lesions, severely impacting the accuracy of subsequent treatment. Summary of the Invention

[0004] To address the technical problem that existing methods for assessing early gastrointestinal cancer cannot accurately determine the location of lesions, the present invention aims to provide an artificial intelligence-based confocal endoscopic method for assessing the risk of early gastrointestinal cancer. The specific technical solution adopted is as follows: This invention provides an artificial intelligence-based confocal endoscopy method for assessing the risk of early gastrointestinal cancer, comprising the following steps: Real-time acquisition of macroscopic and microscopic images of the digestive tract; real-time acquisition of the reference position of the confocal probe in the macroscopic image; Based on the differences in grayscale distribution and reference position distribution between the macroscopic images at the current time and the previous adjacent time, the first displacement vector of the confocal probe at the current time and the degree of peristalsis of the digestive tract at the current time are obtained. Based on the differences in shape and position of cell regions in the microscopic images at the current time and the previous adjacent time, the second displacement vector and sliding degree of the confocal probe at the current time are obtained; Based on the first displacement vector, the degree of creep, the second displacement vector, and the degree of sliding, the comprehensive relative displacement vector of the confocal probe at the current moment is obtained; Based on the reference position at the previous adjacent time and the comprehensive relative displacement vector, the corrected reference position of the confocal probe in the macroscopic image at the current time is obtained.

[0005] Furthermore, the method for obtaining the first displacement vector is as follows: The preset window region of the reference position in the macroscopic image at the previous adjacent time is taken as the first reference window region; the preset window regions of the reference position and its preset neighboring points in the macroscopic image at the current time are both taken as the second reference window region. For any second reference window region, the average difference in grayscale values ​​between pixels at the same position in the first reference window region and the second reference window region is taken as the degree of inconsistency between the first reference window region and the second reference window region. The second reference window region corresponding to the minimum degree of inconsistency is used as the matching window region of the first reference window region. The position within the matching window area that corresponds to the reference position within the first reference window area is taken as the suspected reference position at the current moment. The vector pointing from the reference position within the first reference window area to the suspected reference position is used as the first displacement vector of the confocal probe at the current moment.

[0006] Furthermore, the method for obtaining the degree of peristalsis is as follows: Overlay the macroscopic images at the current time and the previous adjacent time to obtain the gray value difference between pixels at the same position, and use them as the first difference. The proportion of first differences with a value of 0 among all first differences is used as the first degree of similarity. The pixels that overlap with the macroscopic image at the current moment after the macroscopic image at the previous adjacent moment is shifted according to the first displacement vector are all taken as the analysis pixels. The difference in grayscale value between each analyzed pixel and its overlapping pixels is used as the second difference. The proportion of second differences that are 0 among all second differences is taken as the second degree of similarity. The difference between the second degree of similarity and the first degree of similarity is used as the analytical difference; wherein, the second degree of similarity is greater than or equal to the first degree of similarity. The normalized product of the magnitude of the first displacement vector and the analysis difference is used as the degree of peristalsis of the lower digestive tract at the current moment.

[0007] Furthermore, the method for obtaining the second displacement vector is as follows: Each cell region in the microscopic image is obtained by edge detection algorithm. One cell region is randomly selected from the microscopic image at the current time and the previous adjacent time and grouped into a cell region pair. The degree of consistency of the cell region pair is obtained based on the similarity in shape between the two cell regions in the pair and the similarity in the distribution of other cell regions within the local area. The cell region pair corresponding to the highest degree of consistency is taken as the target cell region pair; The vector pointing from the centroid of the target cell region at the previous time step to the centroid of the cell region at the current time step is used as the second displacement vector of the confocal probe at the current time step.

[0008] Furthermore, the method for obtaining the degree of consistency is as follows: For any pair of cell regions, the two cell regions in the pair are aligned with their centroids to obtain the overlap area as the first reference area of ​​the pair, and the sum of the non-overlapping areas of the two cell regions is obtained as the second reference area of ​​the pair. Based on the negative correlation result of the second reference area and the first reference area, the first matching analysis value of the pair of cell regions is obtained. The distance between the centroids of the two cell regions in the cell region pair is taken as the first analysis distance; the distance between the centroids of the target neighboring cell regions of the two cell regions in the cell region pair is taken as the second analysis distance; the result of negatively correlated between the first analysis distance and the second analysis distance is taken as the second matching analysis value of the cell region pair. The product of the first and second matching analysis values ​​is taken as the degree of consistency of the cell region pair.

[0009] Furthermore, the method for obtaining the target neighboring cell region of the cell region is as follows: For any cell region in any microscopic image, the distance between the centroid of that cell region and the centroid of other cell regions in the same microscopic image is obtained and used as the first distance. All other cell regions corresponding to the smallest first distance are taken as the reference neighbor cell regions of that cell region; For any reference neighboring cell region, the area difference between the reference neighboring cell region and the cell region is used as the morphological similarity analysis value between the reference neighboring cell region and the cell region. The reference neighboring cell region corresponding to the smallest morphological similarity analysis value is taken as the target neighboring cell region of that cell region.

[0010] Furthermore, the method for obtaining the degree of sliding is as follows: The positions corresponding to the centroids of all cell regions in the microscopic image of the current time and the previous adjacent time are all used as the reference centroids for movement. For any moving reference centroid, when the moving reference centroid is located in the cell region of the microscopic image at the current time, the cell region corresponding to the moving reference centroid in the microscopic image at the previous adjacent time and the cell region in the microscopic image at the current time are constructed as a reference cell pair. The ratio of the number of reference cell pairs to the total number of moving reference centroids is used as the first characteristic value; The mean of the consistency of all reference cell pairs is used as the second characteristic value; The normalized product of the first eigenvalue, the second eigenvalue, and the magnitude of the second displacement vector is used as the degree of sliding of the confocal probe at the current moment.

[0011] Furthermore, the method for obtaining the comprehensive relative displacement vector is as follows: The product of the degree of creep and the first displacement vector is used as the corrected first displacement vector; The ratio of the second displacement vector to the magnification of the confocal probe relative to the conventional endoscope is used as the quantized second displacement vector; The product of the degree of slip and the quantized second displacement vector is used as the corrected second displacement vector; The sum of the corrected first displacement vector and the corrected second displacement vector is used as the comprehensive relative displacement vector of the confocal probe at the current moment.

[0012] Furthermore, the method for obtaining the corrected reference position is as follows: The position of the reference position at the previous adjacent time step is shifted according to the comprehensive relative displacement vector and used as the corrected reference position of the confocal probe in the macroscopic image at the current time step.

[0013] Furthermore, the centroid of the cell region is obtained through the irregular shape centroid calculation method.

[0014] The present invention has the following beneficial effects: This invention first obtains the first displacement vector of the confocal probe and the degree of peristalsis of the digestive tract at the current moment based on the differences in grayscale distribution and reference position distribution between the macroscopic images at the current moment and the previous adjacent moment. This accurately reflects the positional offset of the confocal probe and the peristalsis of the digestive tract in the macroscopic image at the current moment, preparing for subsequent accurate correction of the reference position at the current moment. Furthermore, based on the differences in shape and position distribution of cell regions in the microscopic images at the current moment and the previous adjacent moment, it obtains the second displacement vector and the degree of sliding of the confocal probe at the current moment. This accurately reflects the displacement and sliding of the confocal probe relative to the digestive tract in the microscopic image at the current moment. Furthermore, this also prepares for accurate correction of the reference position at the current moment. Based on the first displacement vector, the degree of peristalsis, the second displacement vector, and the degree of sliding, the comprehensive relative displacement vector of the confocal probe at the current moment is obtained, accurately achieving the organic integration of the macroscopic and microscopic scales. This precisely reflects the relative displacement vector of the actual position change of the confocal probe, which is beneficial for accurately correcting the position of the confocal probe in the macroscopic image at the current moment. Furthermore, based on the reference position at the previous adjacent moment and the comprehensive relative displacement vector, the corrected reference position of the confocal probe in the macroscopic image at the current moment is accurately obtained, effectively improving the accuracy of locating early gastrointestinal cancer lesions and thus effectively improving the accuracy of subsequent treatment. Attached Figure Description

[0015] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A schematic flowchart illustrating an artificial intelligence-based confocal endoscopic method for assessing the risk of early gastrointestinal cancer, provided as an embodiment of the present invention; Figure 2 This is a structural diagram of an artificial intelligence-based confocal endoscopic gastrointestinal cancer early risk assessment system provided in one embodiment of the present invention; Figure 3 This is a schematic diagram of a computer device provided according to an embodiment of the present invention. Detailed Implementation

[0017] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of the artificial intelligence-based confocal endoscopic method for early gastrointestinal cancer risk assessment proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0019] The following description, in conjunction with the accompanying drawings, details the specific scheme of the artificial intelligence-based confocal endoscopic method for assessing the risk of early gastrointestinal cancer provided by this invention.

[0020] Example 1: This invention proposes an artificial intelligence-based confocal endoscopic risk assessment method for early gastrointestinal cancers. Please refer to [link to relevant documentation]. Figure 1 The diagram illustrates a schematic flowchart of an artificial intelligence-based confocal endoscopic method for assessing the risk of early gastrointestinal cancer, according to an embodiment of the present invention. The method includes the following steps: Step S1: Acquire macroscopic and microscopic images of the digestive tract in real time; acquire the reference position of the confocal probe in the macroscopic image in real time.

[0021] Specifically, this embodiment uses one patient as an example for analysis; all subsequent patient references refer to this patient and will not be repeated. To accurately analyze the patient's digestive tract information and enable timely and accurate assessment of the risk of early digestive tract cancer, the patient first assumes a supine position under the guidance of medical staff, cooperating to fix the patient's position on the operating table in preparation for subsequent endoscopic examination. Then, a collaborative operation mode using a conventional endoscope and a probe-type confocal laser microscopy endoscope is employed to acquire macroscopic and microscopic images of the digestive tract in real time. It should be noted that the confocal probe is pre-installed inside the biopsy channel of the conventional endoscope and is slowly inserted into the patient's digestive tract along with the conventional endoscope. Once the conventional endoscope reaches the target area of ​​the digestive tract to be screened, preliminary macroscopic image acquisition is completed using the conventional endoscope. When fine observation at the cellular level is required in local suspicious areas (such as identifying precancerous features like goblet cell metaplasia and abnormal proliferation), the confocal probe is then extended from the biopsy channel, adhering closely to the digestive tract mucosa to capture high-resolution microscopic images. To dynamically track the state of the digestive tract mucosa and avoid positional deviations caused by peristalsis, this embodiment sets the time interval between two adjacent image acquisitions to 1 second. Implementers can set the time interval between two adjacent image acquisitions according to actual conditions; no limitation is imposed here. Macroscopic and microscopic images are acquired synchronously.

[0022] After acquisition, the macroscopic and microscopic images are first processed using noise reduction algorithms to eliminate interference from equipment noise and light fluctuations. Then, image enhancement processing is applied to optimize detail clarity, ensuring image quality meets the requirements of subsequent analysis. This completes the preprocessing of the macroscopic and microscopic images. It should be noted that all subsequent macroscopic and microscopic images are preprocessed versions. The image preprocessing process is well-known and will not be elaborated upon further.

[0023] The preprocessed images will be uploaded to the early gastrointestinal cancer risk assessment system in real time. The system will automatically mark the location of the gastrointestinal mucosa corresponding to the microscopic image in the central area of ​​the conventional endoscopic macroscopic image. That is, the reference position of the confocal probe in the macroscopic image will be obtained in real time. The core of this step is to establish the initial spatial relationship between the microscopic image and the macroscopic image, so as to provide a precise reference point for subsequent correction of positional deviation caused by gastrointestinal peristalsis.

[0024] Step S2: Based on the difference in grayscale distribution and the difference in reference position distribution between the macroscopic images at the current time and the previous adjacent time, obtain the first displacement vector of the confocal probe at the current time and the degree of peristalsis of the digestive tract at the current time.

[0025] Specifically, continuous peristalsis in the digestive tract is a key physiological activity for maintaining normal digestive function. Through rhythmic contractions and relaxations, the digestive tract grinds food, propels it into the lumen, and absorbs nutrients. This process occurs throughout the body's daily physiological cycle and cannot be completely stopped by endoscopy. However, in endoscopic procedures for early gastrointestinal cancer screening, this physiological peristalsis exhibits unpredictable randomness. This is because the confocal probe needs to be closely attached to the digestive tract mucosa to capture cellular images. Its marked position in the macroscopic view of a conventional endoscope must precisely correspond to the actual mucosal area. However, peristalsis causes displacement of the mucosa and the confocal probe attached to it, directly leading to a deviation between the marked position in the confocal image and the actual mucosal position in the conventional endoscope. If this is not corrected, it will seriously affect the subsequent localization and diagnosis of early cancer areas. Therefore, before performing positional correlation correction between confocal images and conventional endoscopic images, it is necessary to analyze the specific state of gastrointestinal peristalsis.

[0026] During gastrointestinal endoscopy, once a conventional endoscope is inserted into the target area through the body's natural cavities, its physical position remains relatively fixed. This stabilizes the endoscope's posture and prevents interference with observation due to its own movement. Consequently, the field of view of a conventional endoscope remains constant because its hardware parameters are predetermined. Once the endoscope's position is fixed, the boundaries and size of the local area of ​​the digestive tract that its lens can capture do not change; it can only continuously cover the digestive tract wall within the same spatial range. However, in reality, the physiological peristalsis of the human digestive tract does not stop during the examination. The mucosal tissue continues to move under the rhythmic contractions of the digestive tract's smooth muscles. This movement does not change the position and field of view of the conventional endoscope, but it does cause displacement of the local digestive tract mucosa that was originally within the endoscopic field of view. That is, the mucosal tissue shifts in position relative to the fixed endoscope lens, ultimately resulting in a continuous change in the local area of ​​the digestive tract captured in the endoscopic image, rather than the endoscope itself or its field of view moving. In order to analyze the peristalsis of the digestive tract in real time and to accurately correct the position of the confocal probe, this embodiment obtains the first displacement vector of the confocal probe and the degree of peristalsis of the digestive tract at the current moment based on the difference in grayscale distribution and the difference in reference position distribution between the macroscopic images at the current moment and the previous adjacent moment. This initially reflects the positional offset of the confocal probe and the peristalsis of the digestive tract at the current moment.

[0027] Preferably, in one feasible implementation of this embodiment, the method for obtaining the first displacement vector is as follows: a preset window region of the reference position in the macroscopic image at the previous adjacent time point is used as the first reference window region; in this embodiment, the size of the preset window region is set to be... The corresponding pixel is the center point. The implementer can set the size of the preset window area and its corresponding pixels according to the actual situation, which is not limited here. The preset window area of ​​the reference position and its preset neighboring points in the macroscopic image at the current moment is used as the second reference window area. In this embodiment, the preset neighboring point is set as the 8 neighboring pixels of the reference position. The implementer can set the preset neighboring point according to the actual situation, which is not limited here. For any second reference window area, the mean of the absolute values ​​of the difference between the gray values ​​of the pixels at the same position in the first reference window area and the second reference window area is used as the degree of inconsistency between the first reference window area and the second reference window area. The smaller the degree of inconsistency, the more consistent the pixel distribution in the second reference window area and the first reference window area. Then, the second reference window area corresponding to the minimum degree of inconsistency is used as the matching window area of ​​the first reference window area. Then, the position in the matching window area corresponding to the reference position in the first reference window area is used as the suspected reference position at the current moment. The vector from the reference position in the first reference window area to the suspected reference position is used as the first displacement vector of the confocal probe at the current moment.

[0028] Preferably, in one feasible embodiment, the method for obtaining the degree of creep is as follows: Overlay the macroscopic images at the current moment and the previous adjacent moment, and obtain the absolute value of the difference in grayscale values ​​between pixels at the same position, all of which are taken as the first difference; the proportion of the first difference that is 0 in all first differences is taken as the first similarity; the larger the first similarity, the more similar the macroscopic images at the current moment and the previous adjacent moment are; the pixels overlapping with the macroscopic image at the current moment after being shifted according to the first displacement vector are all taken as analysis pixels; the absolute value of the difference in grayscale values ​​between each analysis pixel and its overlapping pixels is obtained, all of which are taken as the second difference; the proportion of the second difference that is 0 in all second differences is taken as the first similarity. The first similarity is used as the second degree of similarity. The greater the second similarity, the more consistent the macroscopic image at the current moment is with the macroscopic image at the previous adjacent moment after being shifted by the first displacement vector. The difference between the second similarity and the first similarity is used as the analytical difference. The greater the analytical difference, the more significant the improvement in the overlap between the macroscopic image at the current moment and the previous adjacent moment after being corrected by the first displacement vector, that is, the higher the fit between the first displacement vector and the actual peristaltic direction and amplitude of the digestive tract. The second similarity must be greater than or equal to the first similarity. It is known that the larger the magnitude of the first displacement vector, the greater the actual intensity of peristalsis. Therefore, in this embodiment, the normalized result of the product of the magnitude of the first displacement vector and the analytical difference is used as the degree of peristalsis of the digestive tract at the current moment. This embodiment normalizes the product of the magnitude of the first displacement vector and the analysis difference using the max-min normalization method. Here, the dataset corresponding to the max-min normalization method is the product of the magnitudes of all first displacement vectors and the analysis differences calculated within a preset time window (e.g., the previous 10 seconds or the previous 20 frames) ending at the current time. It should be noted that if the maximum and minimum values ​​within the preset time window are equal, the creep degree is directly defined as 0 to avoid division by zero errors. The method for obtaining the vector magnitude and the max-min normalization method are well-known techniques and will not be elaborated further.

[0029] Step S3: Based on the differences in shape and position of cell regions in the microscopic images at the current time and the previous adjacent time, obtain the second displacement vector and sliding degree of the confocal probe at the current time.

[0030] Specifically, it is known that the contact between the confocal probe and the digestive tract is an adherence rather than a fixation, meaning that the confocal probe does not form a tight connection with the digestive tract mucosa. Furthermore, the digestive tract mucosa itself has a certain degree of smoothness. When peristalsis causes the mucosa to shift, the friction between the mucosa and the confocal probe is insufficient to completely constrain their synchronous movement. This results in the confocal probe not being able to fully follow the rhythm of peristalsis and only partially responding to the mucosa's displacement trend. If the first displacement vector is directly used as the final relative displacement of the confocal probe, the relative motion error between the probe and the mucosa will be ignored, leading to deviations in subsequent position marking. Therefore, further correction is needed by combining the microscopic images captured by the confocal probe itself.

[0031] The core function of a confocal probe is to conform to the digestive tract wall mucosa to acquire cellular images. The accuracy of its observation area directly depends on the stability of its relative position with the mucosa. When the digestive tract wall moves with peristalsis, the confocal probe will initially move under the influence of mucosal friction. However, during this process, due to factors such as mucosal smoothness, fluctuations in local peristaltic force, and slight adjustments in the posture of the confocal probe itself, relative slippage can easily occur between the probe and the mucosa. This causes the mucosal area actually observed by the confocal probe to deviate from the expected area calculated based solely on the overall peristalsis of the digestive tract, ultimately resulting in unexpected changes in the field of view in continuously captured cellular images.

[0032] Based on the above analysis, by analyzing the continuous microscopic images captured by the confocal probe, the relative sliding degree between the probe and the digestive tract wall mucosa can be calculated and quantified, providing a crucial basis for subsequent precise correction of the probe position. Therefore, this embodiment obtains the second displacement vector and sliding degree of the confocal probe at the current moment based on the differences in shape and position of cell regions in the microscopic images at the current moment and the previous adjacent moment, accurately reflecting the displacement and sliding of the confocal probe relative to the digestive tract in the microscopic images at the current moment.

[0033] Preferably, in one feasible embodiment, the second displacement vector is obtained as follows: Given the significant difference between the cell wall and cytoplasm in the microscopic image, each cell region in the microscopic image is directly obtained using an edge detection algorithm. The edge detection algorithm is a well-known technique and will not be elaborated further. To analyze the displacement of the confocal probe relative to the digestive tract in the microscopic image at the current moment, a cell region is randomly selected from both the current moment and the previous adjacent moment, forming a cell region pair. Then, based on the similarity in shape between the two cell regions in the cell region pair and the similarity in the distribution of other cell regions within the local area, the consistency of the cell region pair is obtained. The method for obtaining the degree of consistency is as follows: For any pair of cell regions, the centroids of the two cell regions in the pair are overlapped to obtain the overlap area as the first reference area of ​​the pair, and the sum of the non-overlapping areas of the two cell regions is obtained as the second reference area of ​​the pair. The larger the first reference area and the smaller the second reference area, the more similar the two cell regions in the pair are. Then, based on the negative correlation result of the second reference area and the first reference area, the first matching analysis value of the pair is obtained. Specifically, the negative of the ratio of the second reference area to the sum of the areas of the two cell regions in the pair is used as the power of an exponential function with the natural constant as the base. The output of the exponential function is the negative correlation result of the second reference area, which is used as the first analysis value. The ratio of the first reference area to the union area of ​​the two cell regions in the pair is used as the second analysis value. Then, the product of the first analysis value and the second analysis value is used as the first matching analysis value of the pair. The centroids of the cell regions are obtained using the irregular shape centroid calculation method, a well-known technique that will not be elaborated further. To more accurately analyze the similarity between the two cell regions in the cell region pair, the Euclidean distance between the centroids of the two cell regions in the cell region pair is used as the first analytical distance; the Euclidean distance between the centroids of the target neighboring cell regions of the two cell regions in the cell region pair is used as the second analytical distance. The closer the first analytical distance is to the second analytical distance, the more correlated the spatial relationship between the two cell regions in the cell region pair is, further indicating that the two cell regions in the cell region pair are more closely related. The more likely a cell region is to be imaged at different times, the more likely it is to be the same cell. The absolute values ​​of the differences between the first and second analysis distances are negatively correlated, and this is used as the second matching analysis value for the cell region pair. In this embodiment, the negative of the absolute value of the difference between the first and second analysis distances is used as the power of an exponential function with the natural constant as its base. The output of this exponential function is the result of negatively correlated differences between the first and second analysis distances. Finally, the product of the first and second matching analysis values ​​is used as the degree of consistency of the cell region pair. The method for obtaining the Euclidean distance is a well-known technique and will not be described further. The method for obtaining the target neighboring cell regions of a cell region is as follows: For any cell region in any microscopic image, the Euclidean distance between the centroids of this cell region and other cell regions in the same microscopic image is obtained, and all of them are used as the first distance; the other cell regions corresponding to the smallest first distance are all used as the reference neighboring cell regions of this cell region; considering that the cell sizes within the same region are similar, for any reference neighboring cell region, the absolute value of the difference between the area of ​​the reference neighboring cell region and the cell region is used as the morphological similarity analysis value between the reference neighboring cell region and the cell region; then the reference neighboring cell region corresponding to the smallest morphological similarity analysis value is used as the target neighboring cell region of this cell region. It should be noted that if there are multiple target neighboring cell regions for a certain cell region, the coordinates corresponding to the average of the coordinates of the centroids of all the target neighboring cell regions of this cell region are used as the coordinates uniquely corresponding to the centroid of the target neighboring cell region of this cell region. The greater the degree of consistency, the more likely the two cell regions in the corresponding cell region pair are to correspond to the same cell. Therefore, the cell region pair corresponding to the highest degree of consistency is taken as the target cell region pair. Then, the vector from the centroid of the cell region at the previous time step to the centroid of the cell region at the current time step in the target cell region pair is taken as the second displacement vector of the confocal probe at the current time step.

[0034] Preferably, in one feasible embodiment, the method for obtaining the degree of sliding is as follows: The positions corresponding to the centroids of all cell regions in the microscopic image at the current time and the previous adjacent time are all moved along the second displacement vector, and these positions are all taken as moving reference centroids; for any moving reference centroid, when the moving reference centroid is located in a cell region in the microscopic image at the current time, the cell region corresponding to the moving reference centroid in the microscopic image at the previous adjacent time and the cell region in the microscopic image at the current time are constructed as a reference cell pair; the ratio of the number of reference cell pairs to the total number of moving reference centroids is taken as a first feature value; the larger the first feature value, the higher the proportion of successfully matched cells in the microscopic image at the current time and the previous adjacent time, and the stronger the consistency of the overall cell displacement trend; the... The mean of the consistency of reference cell pairs is used as the second feature value. The larger the second feature value, the higher the overall matching consistency and the more stable the determination of the slippage trend. The larger the magnitude of the second displacement vector, the more significant the cell slippage. In order to quantitatively evaluate the degree of slippage of the confocal probe relative to the digestive tract, the product of the first feature value, the second feature value, and the magnitude of the second displacement vector is subjected to a minimum-maximum normalization (MPM) method, which is used as the degree of slippage of the confocal probe at the current moment. The dataset corresponding to the MMP method is the product of the first feature value, the second feature value, and the magnitude of the second displacement vector calculated at all moments within a preset time window (e.g., the first 10 seconds or the first 20 frames of images) with the current moment as the endpoint. It should be noted that if all products are equal within the preset time window, the degree of slippage is directly set to 0.

[0035] Step S4: Based on the first displacement vector, the degree of creep, the second displacement vector, and the degree of sliding, obtain the comprehensive relative displacement vector of the confocal probe at the current moment.

[0036] Specifically, the first displacement vector is a displacement vector initially calculated from the macroscopic images of a conventional endoscope. It directly reflects the overall displacement direction and distance of the mucosal region where the confocal probe is located, driven by gastrointestinal peristalsis. The degree of peristalsis is essentially a weighting coefficient of the first displacement vector; the greater the degree of peristalsis, the stronger the peristalsis at the current moment, and the more significant its dominant role in the displacement of the confocal probe, thus increasing the reference value of the first displacement vector. The second displacement vector is the relative sliding vector between the probe and the mucosa extracted from the microscopic images of the confocal probe. It records the local displacement of the confocal probe due to the mucosal smoothness not fully following the peristalsis. The degree of sliding is essentially a weighting coefficient of the second displacement vector; the greater the degree of sliding, the more intense the sliding between the confocal probe and the mucosa, and the more prominent the impact of the local displacement on the final position, thus increasing the reference value of the second displacement vector. Therefore, this embodiment obtains the comprehensive relative displacement vector of the confocal probe at the current moment based on the first displacement vector, the degree of peristalsis, the second displacement vector, and the degree of sliding. This is beneficial for accurately correcting the position of the confocal probe in the macroscopic images at the current moment.

[0037] Considering that the pixel scale of microscopic images differs significantly from that of macroscopic images in practice, this embodiment needs to obtain the magnification of the confocal probe relative to a conventional endoscope to quantify the scale correspondence between the microscopic and macroscopic dimensions. This provides a basis for unifying the dimensions of the subsequent superposition of displacement vectors of different dimensions. Then, the second displacement vector is divided by the aforementioned magnification to convert the second displacement vector into the same macroscopic pixel scale of a conventional endoscope as the first displacement vector, thus achieving unification of the dimensions of the two.

[0038] Preferably, in one feasible embodiment of this invention, the method for obtaining the comprehensive relative displacement vector is as follows: the product of the peristaltic degree and the first displacement vector is used as the corrected first displacement vector; the ratio of the second displacement vector to the magnification of the confocal probe relative to the conventional endoscope is used as the quantized second displacement vector; the product of the sliding degree and the quantized second displacement vector is used as the corrected second displacement vector; and the sum of the corrected first displacement vector and the corrected second displacement vector is used as the comprehensive relative displacement vector of the confocal probe at the current moment. This comprehensive relative displacement vector achieves an organic fusion of macroscopic and microscopic perspectives, capturing the overall movement trend of the digestive tract through macroscopic peristaltic components and compensating for local relative displacement errors with microscopic sliding components, ultimately obtaining a relative displacement vector that accurately reflects the actual positional changes of the confocal probe.

[0039] Step S5: Based on the reference position at the previous adjacent time and the comprehensive relative displacement vector, obtain the corrected reference position of the confocal probe in the macroscopic image at the current time.

[0040] Specifically, in this embodiment, the reference position at the previous adjacent time point is shifted according to the comprehensive relative displacement vector and used as the corrected reference position of the confocal probe in the macroscopic image at the current time point. This realizes the dynamic and accurate correlation between the confocal microscopic image and the conventional endoscopic macroscopic image, providing a reliable spatial coordinate basis for subsequent localization of early gastrointestinal cancer lesions and formulation of treatment plans.

[0041] It should be noted that, in practice, the reference position in the macroscopic image at each time point other than the initial time point may be deviated. Therefore, in this embodiment, the reference position of the confocal probe in the macroscopic image at the previous adjacent time point is essentially the corrected reference position of the confocal probe in the macroscopic image at the previous adjacent time point.

[0042] In summary, this embodiment acquires macroscopic and microscopic images of the digestive tract in real time, as well as the reference position of the confocal probe in the macroscopic image. Based on the differences in grayscale and reference position distribution between the macroscopic image at the current moment and the previous adjacent moment, and the differences in shape and position distribution of cell regions in the microscopic image at the current moment and the previous adjacent moment, the comprehensive relative displacement vector of the confocal probe at the current moment is obtained. Based on the reference position at the previous adjacent moment and the comprehensive relative displacement vector, the corrected reference position of the confocal probe in the macroscopic image at the current moment is obtained. This invention effectively improves the accuracy of locating early-stage digestive tract cancer lesions by accurately acquiring the corrected reference position of the confocal probe in the macroscopic image in real time.

[0043] Example 2: This invention also proposes an artificial intelligence-based confocal endoscopic risk assessment system for early gastrointestinal cancers. Please refer to [link to relevant documentation]. Figure 2 The diagram illustrates a structural diagram of an artificial intelligence-based confocal endoscopic gastrointestinal cancer risk assessment system according to an embodiment of the present invention. The system includes: an image acquisition module 10, a macroscopic image analysis module 20, a microscopic image analysis module 30, a comprehensive relative displacement vector acquisition module 40, and a position correction module 50.

[0044] The image acquisition module 10 is used to acquire macroscopic and microscopic images of the digestive tract in real time; and to acquire the reference position of the confocal probe in the macroscopic image in real time.

[0045] The macroscopic image analysis module 20 is used to obtain the first displacement vector of the confocal probe and the degree of peristalsis of the digestive tract at the current moment based on the difference in grayscale distribution and the difference in reference position distribution between the macroscopic images at the current moment and the previous adjacent moment.

[0046] The microscopic image analysis module 30 is used to obtain the second displacement vector and sliding degree of the confocal probe at the current moment based on the differences in shape and position of cell regions in the microscopic images at the current moment and the previous adjacent moment.

[0047] The comprehensive relative displacement vector acquisition module 40 is used to acquire the comprehensive relative displacement vector of the confocal probe at the current moment based on the first displacement vector, the degree of creep, the second displacement vector, and the degree of sliding.

[0048] The position correction module 50 is used to obtain the corrected reference position of the confocal probe in the macroscopic image at the current moment based on the reference position at the previous adjacent moment and the comprehensive relative displacement vector.

[0049] It should be noted that the system provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the artificial intelligence-based confocal endoscopic gastrointestinal cancer risk assessment system and the artificial intelligence-based confocal endoscopic gastrointestinal cancer risk assessment method provided in the above embodiments belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0050] Example 3: This invention also proposes an artificial intelligence-based confocal endoscopic gastrointestinal cancer risk assessment device. The device includes a memory and a processor. The memory stores executable program code, and the processor calls and executes the executable program code to perform the artificial intelligence-based confocal endoscopic gastrointestinal cancer risk assessment method provided in the embodiments of this application. Specifically, the device may be a chip, component, or module. The chip may include a connected processor and memory; the memory stores instructions, and when the processor calls and executes the instructions, the chip can perform the artificial intelligence-based confocal endoscopic gastrointestinal cancer risk assessment method provided in the above embodiments.

[0051] Furthermore, this application also protects a computer device; please refer to [link to relevant documentation]. Figure 3 The computer device includes a memory 401, a processor 402, and a computer program 403 stored in the memory 401 and running on the processor 402. When the processor 402 executes the computer program 403, the computer device can execute any of the aforementioned artificial intelligence-based confocal endoscopic early gastrointestinal cancer risk assessment methods.

[0052] Example 4: This embodiment also provides a computer-readable storage medium storing computer program code. When the computer program code is run on a computer, the computer executes the above-described related method steps to implement the artificial intelligence-based confocal endoscopic method for assessing the risk of early gastrointestinal cancer provided in the above embodiment.

[0053] Example 5: This embodiment also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned related steps to realize the artificial intelligence-based confocal endoscopic method for assessing the risk of early gastrointestinal cancer provided in the above embodiment.

[0054] In this embodiment, the device, computer-readable storage medium, computer program product, or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0055] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0056] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A method for risk assessment of early gastrointestinal cancer using confocal endoscopy based on artificial intelligence, characterized in that, The method includes the following steps: Real-time acquisition of macroscopic and microscopic images of the digestive tract; real-time acquisition of the reference position of the confocal probe in the macroscopic image; Based on the differences in grayscale distribution and reference position distribution between the macroscopic images at the current time and the previous adjacent time, the first displacement vector of the confocal probe at the current time and the degree of peristalsis of the digestive tract at the current time are obtained. Based on the differences in shape and position of cell regions in the microscopic images at the current time and the previous adjacent time, the second displacement vector and sliding degree of the confocal probe at the current time are obtained; Based on the first displacement vector, the degree of creep, the second displacement vector, and the degree of sliding, the comprehensive relative displacement vector of the confocal probe at the current moment is obtained; Based on the reference position at the previous adjacent time and the comprehensive relative displacement vector, the corrected reference position of the confocal probe in the macroscopic image at the current time is obtained. The method for obtaining the first displacement vector is as follows: The preset window region of the reference position in the macroscopic image at the previous adjacent time is taken as the first reference window region; the preset window regions of the reference position and its preset neighboring points in the macroscopic image at the current time are both taken as the second reference window region. For any second reference window region, the average difference in grayscale values ​​between pixels at the same position in the first reference window region and the second reference window region is taken as the degree of inconsistency between the first reference window region and the second reference window region. The second reference window region corresponding to the minimum degree of inconsistency is used as the matching window region of the first reference window region. The position within the matching window area that corresponds to the reference position within the first reference window area is taken as the suspected reference position at the current moment. The vector pointing from the reference position within the first reference window area to the suspected reference position is taken as the first displacement vector of the confocal probe at the current moment. The method for obtaining the degree of peristalsis is as follows: Overlay the macroscopic images at the current time and the previous adjacent time to obtain the gray value difference between pixels at the same position, and use them as the first difference. The proportion of first differences with a value of 0 among all first differences is used as the first degree of similarity. The pixels that overlap with the macroscopic image at the current moment after the macroscopic image at the previous adjacent moment is shifted according to the first displacement vector are all taken as the analysis pixels. The difference in grayscale value between each analyzed pixel and its overlapping pixels is used as the second difference. The proportion of second differences that are 0 among all second differences is taken as the second degree of similarity. The difference between the second degree of similarity and the first degree of similarity is used as the analytical difference; wherein, the second degree of similarity is greater than or equal to the first degree of similarity. The normalized product of the magnitude of the first displacement vector and the analysis difference is used as the degree of peristalsis of the lower digestive tract at the current moment. The method for obtaining the second displacement vector is as follows: Each cell region in the microscopic image is obtained by edge detection algorithm. One cell region is randomly selected from the microscopic image at the current time and the previous adjacent time and grouped into a cell region pair. The degree of consistency of the cell region pair is obtained based on the similarity in shape between the two cell regions in the pair and the similarity in the distribution of other cell regions within the local area. The cell region pair corresponding to the highest degree of consistency is taken as the target cell region pair; The vector pointing from the centroid of the target cell region at the previous time step to the centroid of the cell region at the current time step is used as the second displacement vector of the confocal probe at the current time step. The method for obtaining the degree of sliding is as follows: The positions corresponding to the centroids of all cell regions in the microscopic image of the current time and the previous adjacent time are all used as the reference centroids for movement. For any moving reference centroid, when the moving reference centroid is located in the cell region of the microscopic image at the current time, the cell region corresponding to the moving reference centroid in the microscopic image at the previous adjacent time and the cell region in the microscopic image at the current time are constructed as a reference cell pair. The ratio of the number of reference cell pairs to the total number of moving reference centroids is used as the first characteristic value; The mean of the consistency of all reference cell pairs is used as the second characteristic value; The normalized product of the first eigenvalue, the second eigenvalue, and the magnitude of the second displacement vector is used as the degree of sliding of the confocal probe at the current moment.

2. The method for assessing the risk of early gastrointestinal cancer based on artificial intelligence using confocal endoscopy as described in claim 1, characterized in that, The method for obtaining the degree of consistency is as follows: For any pair of cell regions, the two cell regions in the pair are overlapped with their centroids to obtain the overlap area as the first reference area of ​​the pair, and the sum of the non-overlapping areas of the two cell regions is used as the second reference area of ​​the pair. Based on the negative correlation results of the second reference area and the first reference area, the first matching analysis value of the cell region pair is obtained; The distance between the centroids of the two cell regions in the cell region pair is taken as the first analysis distance; the distance between the centroids of the target neighboring cell regions of the two cell regions in the cell region pair is taken as the second analysis distance; the result of negatively correlated between the first analysis distance and the second analysis distance is taken as the second matching analysis value of the cell region pair. The product of the first and second matching analysis values ​​is taken as the degree of consistency of the cell region pair.

3. The method for assessing the risk of early gastrointestinal cancer based on artificial intelligence using confocal endoscopy as described in claim 2, characterized in that, The method for obtaining the target neighboring cell region of the cell region is as follows: For any cell region in any microscopic image, the distance between the centroid of that cell region and the centroid of other cell regions in the same microscopic image is obtained and used as the first distance. All other cell regions corresponding to the smallest first distance are taken as the reference neighbor cell regions of that cell region; For any reference neighboring cell region, the area difference between the reference neighboring cell region and the cell region is used as the morphological similarity analysis value between the reference neighboring cell region and the cell region. The reference neighboring cell region corresponding to the smallest morphological similarity analysis value is taken as the target neighboring cell region of that cell region.

4. The method for risk assessment of early gastrointestinal cancer based on artificial intelligence using confocal endoscopy as described in claim 1, characterized in that, The method for obtaining the comprehensive relative displacement vector is as follows: The product of the degree of creep and the first displacement vector is used as the corrected first displacement vector; The ratio of the second displacement vector to the magnification of the confocal probe relative to the conventional endoscope is used as the quantized second displacement vector; The product of the degree of slip and the quantized second displacement vector is used as the corrected second displacement vector; The sum of the corrected first displacement vector and the corrected second displacement vector is used as the comprehensive relative displacement vector of the confocal probe at the current moment.

5. The method for risk assessment of early gastrointestinal cancer based on artificial intelligence using confocal endoscopy as described in claim 1, characterized in that, The method for obtaining the corrected reference position is as follows: The position of the reference position at the previous adjacent time step is shifted according to the comprehensive relative displacement vector and used as the corrected reference position of the confocal probe in the macroscopic image at the current time step.

6. The method for risk assessment of early gastrointestinal cancer based on artificial intelligence using confocal endoscopy as described in claim 1, characterized in that, The centroid of the cell region was obtained using the irregular shape centroid calculation method.

Images