Early gastrointestinal tumor double-spectrum imaging and biopsy navigation system under endoscope

Abstract

The invention discloses an endoscopic early-stage gastrointestinal tumor double-spectrum imaging and biopsy navigation system, which comprises an acquisition module used for acquiring source data in the process of probing target tissues by detection equipment; the source data comprises a dual-spectrum image acquired by an endoscope, CT spectrum data acquired by CT equipment, magnetic field coordinate data acquired by a magnetic field induction chip and a sampling state of a sampling instrument; the data processing module is used for determining a tissue area identifier based on the CT spectral data, the dual-spectral image and the magnetic field coordinate data in the source data, and generating a navigation control signal based on the instrument area identifier and the tissue area identifier corresponding to the sampling state; the biopsy navigation module is used for navigating the sampling instrument to move to the area where the risk part is located based on the navigation control signal and controlling the sampling instrument to execute sampling operation on the risk part to obtain sampling tissue. According to the scheme, the accuracy of gastrointestinal tumor diagnosis can be improved.

Description

Technical Field

[0001] This application relates to the field of digestive endoscopy diagnosis and treatment technology, specifically to, but not limited to, an endoscopic dual-spectral imaging and biopsy navigation system for early gastrointestinal tumors. Background Technology

[0002] Gastrointestinal tumors are a type of highly prevalent malignant tumor, and early, accurate biopsy is a key factor in improving the diagnosis and treatment outcomes of these tumors. In practice, before a biopsy, a computed tomography (CT) scan of the gastrointestinal tract is usually performed to obtain CT images. During the biopsy, the images acquired endoscopically and the CT images are combined to determine the navigation biopsy path for the specific biopsy sampling procedure. However, during the biopsy, the continuous peristalsis of the gastrointestinal tract causes dynamic changes in the location of the lesion, leading to deviations in the biopsy path and a decrease in the accuracy of biopsy sampling, resulting in missed diagnoses and misdiagnoses of gastrointestinal tumors. Summary of the Invention

[0003] Based on the above technical problems, this application provides an endoscopic dual-spectral imaging and biopsy navigation system for early gastrointestinal tumors, which can reduce the probability of biopsy path deviation and decreased sampling accuracy caused by peristalsis of gastrointestinal tissues, thereby improving the accuracy of gastrointestinal tumor diagnosis.

[0004] The technical solution provided in this application is as follows: This application provides an endoscopic dual-spectral imaging and biopsy navigation system for early gastrointestinal tumors, including: The acquisition module is used to acquire source data during the process of the detection device probing the target tissue; wherein, the detection device includes an endoscope, a CT device, a magnetic field sensing chip, and a sampling instrument; the target tissue includes at least a portion of the gastrointestinal tract; the source data includes bispectral images acquired by the endoscope, CT spectral data acquired by the CT device, magnetic field coordinate data acquired by the magnetic field sensing chip, and the sampling status of the sampling instrument; The data processing module is used to determine the tissue region identifier based on the CT spectral data, the bispectral image, and the magnetic field coordinate data in the source data, and to generate a navigation control signal based on the instrument region identifier corresponding to the sampling state and the tissue region identifier; wherein, the tissue region identifier includes the identifier of the area where the risk site contained in the target tissue is located. A biopsy navigation module is used to navigate the sampling instrument to the area where the risk site is located based on the navigation control signal, and to control the sampling instrument to perform a sampling operation on the risk site to obtain sampled tissue; wherein the sampled tissue is used to perform a biopsy.

[0005] The endoscopic dual-spectral imaging and biopsy navigation system for early gastrointestinal tumors provided in this application has at least the following beneficial effects: In the endoscopic early gastrointestinal tumor dual-spectrum imaging and biopsy navigation system provided in this application embodiment, the acquisition module is used to acquire source data during the exploration of target tissue by the detection device. The detection device includes an endoscope, a CT scanner, a magnetic field induction chip, and a sampling instrument. The target tissue includes at least a portion of the gastrointestinal tract. The source data includes dual-spectrum images acquired by the endoscope, CT spectral data acquired by the CT scanner, magnetic field coordinate data acquired by the magnetic field induction chip, and the sampling status of the sampling instrument. Thus, during the exploration of target tissue by the detection device, diverse source data can be continuously and tracked, thereby improving the diversity and comprehensiveness of the source data. Furthermore, through the CT spectral data, the magnetic field coordinate data, and the dual-spectrum images in the source data, the distribution pattern of risk sites in the target tissue can be comprehensively characterized from multiple dimensions. Simultaneously, the data processing module, based on the CT spectral data and dual-spectrum images in the source data... Spectral images and magnetic field coordinate data are used to identify tissue regions, including the regions containing risk sites within the target tissue. This enables high-precision identification of these risk sites. On the other hand, navigation control signals are generated based on the instrument region identifier corresponding to the sampling state. This establishes a correlation between the navigation control signals and the actual sampling state and instrument region identifier of the sampling instrument, improving the accuracy and real-time performance of the navigation control signals. Furthermore, the biopsy navigation module uses the navigation control signals to guide the sampling instrument to the region containing the risk site and controls the instrument to perform sampling operations on the risk site to obtain tissue for biopsy. This automated, real-time, and precise navigation of the sampling instrument reduces the probability of biopsy path deviation and decreased sampling accuracy due to peristalsis of the gastrointestinal tissue, thereby improving the accuracy of gastrointestinal tumor diagnosis. Attached Figure Description

[0006] Figure 1 This is a schematic diagram of the endoscopic dual-spectral imaging and biopsy navigation system for early gastrointestinal tumors provided in an embodiment of this application. Figure 2 This is another schematic diagram of the endoscopic dual-spectral imaging and biopsy navigation system for early gastrointestinal tumors provided in the embodiments of this application. Detailed Implementation

[0007] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0008] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0009] For gastrointestinal tumors, a common type of malignant tumor, early and accurate biopsy is a key factor in improving the diagnosis and treatment outcomes. In practice, a CT scan of the gastrointestinal tract is usually performed before a biopsy to obtain CT images. During the biopsy, the images acquired endoscopically and the CT images are combined to determine the navigation biopsy path for the specific biopsy sampling procedure. However, during the biopsy, continuous peristalsis of the gastrointestinal tract causes dynamic changes in the location of the lesion, leading to deviations in the biopsy path and decreased accuracy in biopsy sampling, resulting in missed diagnoses and misdiagnoses of gastrointestinal tumors.

[0010] Based on the above technical problems, this application provides an endoscopic dual-spectral imaging and biopsy navigation system for early gastrointestinal tumors. Figure 1 This is a schematic diagram of the endoscopic dual-spectrum imaging and biopsy navigation system for early gastrointestinal tumors provided in the embodiments of this application, as shown below. Figure 1 As shown, the endoscopic early gastrointestinal tumor dual-spectrum imaging and biopsy navigation system 100 may include: The acquisition module 101 is used to acquire source data during the process of the detection equipment probing the target tissue; wherein the detection equipment includes an endoscope, a CT device, a magnetic field sensing chip, and a sampling instrument; the target tissue includes at least a portion of the gastrointestinal tract; the source data includes dual-spectral images acquired by the endoscope, CT spectral data acquired by the CT device, magnetic field coordinate data acquired by the magnetic field sensing chip, and the sampling status of the sampling instrument. The data processing module 102 is used to determine the tissue region identifier based on the CT spectral data, bispectral image and magnetic field coordinate data in the source data, and to generate navigation control signals based on the instrument region identifier and tissue region identifier corresponding to the sampling state; wherein, the tissue region identifier includes the identifier of the area where the risk site contained in the target tissue is located. The biopsy navigation module 103 is used to navigate the sampling instrument to the area where the risk site is located based on the navigation control signal, and to control the sampling instrument to perform sampling operations on the risk site to obtain sampled tissue; wherein, the sampled tissue is used to perform biopsy.

[0011] In some embodiments, during the process of probing target tissue with a detection device, the various devices in the detection device can be controlled to perform data acquisition operations synchronously and continuously, thereby obtaining an image sequence including multiple bispectral images, a CT spectral sequence including CT spectral data at multiple times, a coordinate sequence including multiple magnetic field coordinate data, and a position sequence including multiple sampling states. Furthermore, the data in the above sequences can remain consistent in the time dimension.

[0012] In some embodiments, bispectral images can characterize the optical features of the mucosa on the surface of a target tissue; for example, bispectral images in an image sequence can be acquired in the following manner: By using sensors configured in conjunction with the endoscope, the reflected light from the target tissue surface in both the tumor-specific fluorescence band and the normal mucosa reference band is captured simultaneously. The reflected light is then converted from optical signals into electrical signals to generate a high-resolution bispectral image.

[0013] In some embodiments, the CT device may include a dual-energy CT device; accordingly, CT spectral data can be used to identify tumor tissue in a diseased state and normal tissue in a non-diseased state within a target tissue; wherein, the risk site may include tumor tissue in a diseased state; exemplarily, CT spectral data can be acquired in the following manner: The target tissue is scanned using a dual-energy CT scanner to generate full-spectrum data. Characteristic absorption peak parameters related to tumor tissue are extracted from the full-spectrum data to obtain CT spectral data.

[0014] In some embodiments, magnetic field coordinate data can characterize the spatial location of the sampling device; correspondingly, the magnetic field coordinate data can be obtained in the following ways: The magnetic field sensor chip at the front end of the endoscope receives magnetic field signals from magnetic markers located on or inside the target tissue. Combined with the endoscope position sensing signal, the magnetic field signal and the endoscope position sensing signal are denoised using an adaptive filtering algorithm to obtain the three-dimensional electromagnetic coordinates of the endoscope, and these three-dimensional electromagnetic coordinates are then determined as magnetic field coordinate data.

[0015] It should be noted that, in the embodiments of this application, the relative positional relationship between the geometric centers of the endoscope and the sampling instrument can be preset. Therefore, after obtaining the three-dimensional electromagnetic coordinates of the endoscope, the position of the sampling instrument can be determined based on the three-dimensional electromagnetic coordinates of the endoscope and the above-mentioned relative positional relationship.

[0016] It should be noted that the magnetic markers can be preset; for example, they can be preset by oral administration to the patient.

[0017] In some embodiments, the sampling state may include the position and / or orientation of the sampling component in a sampling instrument for performing the sampling operation; exemplarily, the sampling instrument may include biopsy forceps, and the sampling component may include the sampling jaws of the biopsy forceps; exemplarily, the sampling state may be obtained in the following manner: The sampling position of the sampling instrument is determined by tracking the infrared position of the infrared markers associated with the sampling instrument using an infrared sensor. The opening and closing angle of the sampling component is determined based on the infrared position of different infrared markers. At the same time, the contact force data between the sampling component and the mucosa of the target tissue is collected by a pressure sensor. The set of sampling position, opening and closing angle and contact force data is determined as the sampling state.

[0018] In some embodiments, the risk site may be located inside and / or on the surface of the target tissue, and the probability of lesion at the risk site is greater than or equal to the probability of tumor.

[0019] In some embodiments, the organization region identifier may include the spatial extent of the risk site; exemplarily, the spatial extent may include a set of coordinates formed by the edges of the risk site region, or it may include the geometric center of the risk site.

[0020] In some embodiments, the organization region identifier can be determined in the following ways: A first transformation matrix and a second transformation matrix are predetermined. Based on the absorption differences of risk sites and non-lesioned tissue parts in CT spectral data to multiple energy levels of radiation, a first coordinate set of the risk site in the first coordinate system is determined. Then, the first coordinate set is transformed using the first transformation matrix to obtain a second coordinate set of the risk site in the second coordinate system. The second coordinate set is then transformed using the second transformation matrix to obtain a third coordinate set of the risk site in the third coordinate system. The third coordinate set is then identified as the regional tissue identifier.

[0021] For example, the first coordinate system may include the coordinate system where the CT spectral data is located, and the first coordinate set may characterize the distribution of the risk site in the first coordinate system; the second coordinate system may include the coordinate system where the magnetic field coordinate data is located, and the second coordinate set may include the distribution of the risk site in the second coordinate system; the third coordinate system may include the image coordinates where the dual-spectral image is located, and the third coordinate set may characterize the distribution of the risk site in the third coordinate system.

[0022] For example, the first transformation matrix may include a mapping matrix that corresponds to the transformation between coordinates in the first coordinate system and coordinates in the second coordinate system; the second transformation matrix may include a mapping matrix that corresponds to the transformation between coordinates in the second coordinate system and coordinates in the third coordinate system.

[0023] For example, the first transformation matrix and the second transformation matrix can be established in advance; specifically, the first transformation matrix can be established in the following way: The transformation relationship between electromagnetic coordinates in the second coordinate system and voxel coordinates in the first coordinate system is established by using a rigid body transformation model. The rotation matrix and translation vector between the first and second coordinate systems are determined by the least squares method, forming the transformation matrix between electromagnetic coordinates and voxel coordinates. This transformation matrix is ​​then defined as the first transformation matrix.

[0024] For example, the second transformation matrix can be constructed in the following way: The transformation relationship between electromagnetic coordinates in the second coordinate system and pixel coordinates in the third coordinate system is established by using a perspective projection model. The transformation parameters used to characterize the transformation relationship are solved by the least squares method, and the transformation matrix between electromagnetic coordinates and pixel coordinates is obtained. This transformation matrix is ​​then determined as the second transformation matrix.

[0025] Specifically, after confirming the first and second transformation matrices, the calculation accuracy of the first and second transformation matrices can be verified using sample data. For example, the actual magnetic field coordinates in the sample data can be substituted into the first and second transformation matrices respectively for calculation, resulting in predicted voxel coordinates and predicted pixel coordinates respectively. If the prediction error between the predicted voxel coordinates and the actual voxel coordinates in the sample data is greater than or equal to the error threshold, the first transformation matrix needs to be calibrated. If the prediction error between the predicted pixel coordinates and the actual pixel coordinates in the sample data is greater than or equal to the error threshold, the second transformation matrix needs to be calibrated, until the prediction errors corresponding to the first and second transformation matrices are both less than the error threshold.

[0026] For example, the sample data may include pre-collected exploration data for exploring at least one target tissue; and the exploration data may include sample magnetic field data, sample CT spectral data, and sample sampling status; for example, a coordinate mapping relationship may be pre-established between different types of data in the sample data, and the coordinate mapping relationship can characterize the correspondence between actual pixel coordinates, actual voxel coordinates and actual magnetic field coordinates.

[0027] In some embodiments, the instrument area identifier may include the spatial range in which the sampling instrument is located.

[0028] In some embodiments, the navigation control signal may include at least one signal for indicating a change in the position and / or attitude of the sampling device; specifically, the navigation control signal may include at least one operation that the sampling device should perform when changing from its current position and / or current attitude to a suitable pose for performing sampling operations; for example, at least one operation may include position movement, angle rotation, and angle opening and closing adjustment of the sampling component, etc.

[0029] In some embodiments, the data processing module can generate navigation control signals in the following ways: If the tissue region represented by the tissue region identifier does not include the device region represented by the device region identifier, then navigation control signals can be generated based on the relative positional relationship between the tissue region identifier and the device region identifier; wherein, the relative positional relationship may include the distance and / or orientation between the tissue region and the device region.

[0030] In some embodiments, the biopsy navigation module can navigate the sampling instrument to the area of ​​the risk site in any of the following ways: The biopsy navigation module automatically controls the sampling instrument to move to the area where the risk site is located based on navigation control signals.

[0031] The biopsy navigation module outputs navigation control signals to provide control references for doctors or professionals to operate the sampling instruments. During the operation of the sampling instruments, the module continuously controls the sampling module to collect new source data. Based on the new source data, a new instrument area identifier is determined, and the navigation control signals are updated according to the new instrument area identifier. The module continues to output new navigation control signals so that doctors or professionals can adjust their operation of the sampling instruments in real time.

[0032] It should be noted that when the sampling instrument is located within the area of ​​the risk site, the data processing module can also output a sampling signal, which can be used by the biopsy navigation module to control the sampling instrument to automatically sample the risk site, or to perform a sampling operation in response to the operation of a doctor or professional technician.

[0033] As can be seen from the above, in the endoscopic early gastrointestinal tumor dual-spectrum imaging and biopsy navigation system provided in this application embodiment, the acquisition module is used to acquire source data during the process of the detection device exploring the target tissue. The detection device includes an endoscope, a CT device, a magnetic field induction chip, and a sampling instrument. The target tissue includes at least a portion of the gastrointestinal tract. The source data includes dual-spectrum images acquired by the endoscope, CT spectral data acquired by the CT device, magnetic field coordinate data acquired by the magnetic field induction chip, and the sampling status of the sampling instrument. Thus, during the process of the detection device exploring the target tissue, diverse source data can be acquired continuously and in a tracking manner, thereby improving the diversity and comprehensiveness of the source data. Furthermore, through the CT spectral data, the magnetic field coordinate data, and the dual-spectrum images in the source data, the distribution pattern of risk sites in the target tissue can be characterized from multiple dimensions and comprehensively. At the same time, the data processing module is based on the CT spectral data in the source data. The system uses dual-spectral images and magnetic field coordinate data to identify tissue regions, including the regions containing risk sites within the target tissue, enabling high-precision identification of these regions. Furthermore, it generates navigation control signals based on the instrument region identifier corresponding to the sampling state, establishing a correlation between the navigation control signals and the actual sampling state of the sampling instrument and the instrument region identifier. This improves the accuracy and real-time performance of the navigation control signals. Based on this, the biopsy navigation module guides the sampling instrument to the region containing the risk site based on the navigation control signals and controls the instrument to perform sampling operations on the risk site to obtain the tissue for biopsy. This automated, real-time, and precise navigation of the sampling instrument reduces the probability of biopsy path deviation and decreased sampling accuracy due to gastrointestinal peristalsis, thereby improving the accuracy of gastrointestinal tumor diagnosis.

[0034] Based on the foregoing embodiments, in the endoscopic early gastrointestinal tumor dual-spectrum imaging and biopsy navigation system provided in this application embodiment, the acquisition module is used to acquire the electromagnetic coordinates of the sampling instrument and the sampling force of the sampling instrument during the sampling operation. The data processing module is used to extract features from CT spectral data to obtain the sampling CT spectral features of the risk sites, integrate electromagnetic coordinates to obtain the sampling position features of the sampling instrument, extract features from the bispectral images corresponding to the risk sites to obtain sampling bispectral features, extract features from the sampling force to obtain sampling force features, and determine the sampling fingerprint data based on the sampling CT spectral features, sampling position features, sampling bispectral features, and sampling force features.

[0035] In some embodiments, electromagnetic coordinates may include the coordinates of the geometric center of the sampling component in a second coordinate system at the moment the sampling operation is performed; correspondingly, electromagnetic coordinates may be obtained in the following ways: When the sampling device performs the sampling operation, the infrared position of the infrared marker point set in the sampling component is obtained. The infrared position is calculated by the vector interpolation algorithm to obtain the center coordinates of the geometric center of the sampling angle in the third coordinate system. Then, the center coordinates are transformed by the inverse matrix of the second transformation matrix to obtain the electromagnetic coordinates. For example, in the above process, the center coordinates can be corrected based on the attitude and size of the sampling device to obtain the electromagnetic coordinates.

[0036] In some embodiments, the sampling force may include the force applied by the sampling component to the risk site and the changing state of the applied force; accordingly, the sampling force may be obtained in the following ways: The force sensor installed in the sampling component collects the force between the risk area and the sampling component to obtain the sampling force.

[0037] In some embodiments, the sampled CT spectral features may include data such as the sampling operation execution time, the intensity ratio of the dual-spectral bands, the standard deviation of the intensity of the specified band, the average tumor probability of the area covered by the opening of the sampling component, the maximum tumor probability, and the proportion of high-probability pixels; correspondingly, the sampled CT spectral features can be obtained in any of the following ways: The CT spectral data is feature extracted using a deep learning model to obtain the first extraction result. The first extraction result is then normalized to obtain the sampled CT spectral features.

[0038] In some embodiments, the sampling location feature may include the direction and / or position of the sampling component performing the sampling operation relative to a specified reference point; exemplarily, the specified reference point may include at least one coordinate point with a pre-set, known geometric location; accordingly, the sampling location feature may be obtained in the following ways: The distance between the electromagnetic coordinates and the magnetic marker points is verified, and outliers in the electromagnetic coordinates are removed based on the distance to obtain the sampling location features.

[0039] In some embodiments, the sampled dual-spectral features may include higher-order semantic features related to gastrointestinal tumors; exemplarily, higher-order semantic features may include tumor benign or malignant assessment results, tumor distribution status, and tumor texture features, etc.; correspondingly, the sampled dual-spectral features can be obtained in the following ways: A multi-scale texture analysis algorithm is used to extract features from the region of interest (ROI) of the bispectral image to obtain a second extraction result. The second extraction result is then standardized using Z-score to eliminate scale differences and obtain the sampled bispectral features.

[0040] In some embodiments, the sampling force characteristics may include the changes in the force state of the sampling component relative to the target tissue before, during, and after the sampling operation; correspondingly, the sampling force characteristics can be obtained in the following ways: Extract the temporal characteristics of the sampling intensity within a specified time period before and after the sampling operation to obtain the sampling intensity characteristics; for example, the specified time period can be preset or flexibly adjusted, and this application embodiment does not limit it.

[0041] In some embodiments, sampling fingerprint data may include sampling identifiers characterizing a complete sampling process of a sampling device targeting a risk site of the target tissue; exemplaryly, sampling fingerprint data may include a set of sampling CT spectral features, sampling location features, sampling dual-spectral features, and sampling force features; correspondingly, sampling fingerprint data may be obtained in the following ways: The sampling fingerprint data is obtained by stitching together the sampling CT spectral features, sampling location features, sampling dual spectral features, and sampling intensity features.

[0042] As can be seen from the above, in the endoscopic early gastrointestinal tumor dual-spectrum imaging and biopsy navigation system provided in this application embodiment, the acquisition module is used to acquire the electromagnetic coordinates and sampling force of the sampling instrument during the sampling operation. In this way, the electromagnetic coordinates and sampling force of the sampling instrument can be tracked and acquired. Furthermore, the data processing module is used to extract features from CT spectral data to obtain the sampling CT spectral features of the risk site, integrate the electromagnetic coordinates to obtain the sampling position features of the sampling instrument, extract features from the dual-spectral image corresponding to the risk site to obtain the sampling dual-spectral features, extract features from the sampling force to obtain the sampling force features, and determine the sampling fingerprint features based on the sampling CT spectral features, sampling position features, sampling dual-spectral features, and sampling force features. In this way, the comprehensiveness and completeness of the data in the sampling fingerprint features are improved. Through the sampling fingerprint features, the sampling process of the sampling instrument on the risk site can be accurately and comprehensively characterized from multiple dimensions.

[0043] Based on the foregoing embodiments, in the endoscopic early gastrointestinal tumor dual-spectrum imaging and biopsy navigation system provided in this application, the biopsy navigation module is used to generate a re-examination control signal based on the sampling fingerprint data.

[0044] The re-inspection control signal is used to control at least some of the equipment in the testing equipment to move to the area where the risk location is located in order to perform a re-inspection operation; the re-inspection operation is performed after the target time period of the sampling operation.

[0045] In some embodiments, the target time period can be preset or adjusted; however, this application does not limit this.

[0046] In some embodiments, the re-examination control signal can control the endoscope and sampling instruments in the re-examination device to move to the area where the risk site is located, and can also control the CT device to adjust the data acquisition posture so that the radiation emitted by the CT device can be relative to the area where the risk site is located.

[0047] It should be noted that, during the re-examination operation, if it is not necessary to control the CT equipment to collect CT spectral data again, the CT spectral data at the time of sampling operation can be determined as the re-examination CT spectrum; and if it is not necessary to perform the sampling operation again, the instrument position and sampling force at the time of sampling operation can be determined as the re-examination position and re-examination force.

[0048] In some embodiments, the re-inspection control signal can be generated in the following manner: The sampling location features are obtained from the sampling fingerprint data, and a first instruction for the sampling instrument is generated based on the sampling location corresponding to the sampling location features. If a resampling operation is required, the sampling force features are obtained from the sampling fingerprint data, and the sampling force corresponding to the sampling force features is adjusted to obtain a second instruction for the sampling component. If CT spectral data needs to be acquired again, the radiation posture of the CT device is obtained from the sampling fingerprint data, and a third instruction for the CT device is generated based on the radiation posture. If dual-spectral images need to be acquired again, the sampling pose of the endoscope is obtained from the sampling fingerprint data, and a fourth instruction for the endoscope is generated based on the sampling pose. Then, the set of the first to fourth instructions (if all of them exist) is determined as the re-examination control signal.

[0049] The first instruction may include control instructions for controlling the position and / or posture changes of the sampling instrument; the second instruction may include control instructions for controlling the sampling force of the sampling component; the third instruction may include control instructions for controlling the direction and angle of the radiation rays of the CT equipment; the fourth instruction may include control instructions for controlling the position and posture of the endoscope; and the sampling fingerprint data may include the equipment posture and the sampling posture. For example, the radiation posture may include the incident direction and angle of the rays when the CT equipment irradiates the risk area, and the sampling posture may include the position and / or posture of the endoscope when the sampling operation is performed.

[0050] As described above, in the endoscopic early gastrointestinal tumor dual-spectrum imaging and biopsy navigation system provided in this application embodiment, the biopsy navigation module is used to generate a re-examination control signal based on the sampling fingerprint data. This re-examination control signal controls at least a portion of the detection equipment to move to the area where the risk site is located to perform a re-examination operation. The re-examination operation is performed after the target time period during which the sampling instrument performs the sampling operation. Thus, through the above operations, precise control of the movement state of the detection equipment can be achieved during the re-examination operation, thereby improving the accuracy of the detection equipment in re-examining risk sites.

[0051] Based on the foregoing embodiments, the endoscopic dual-spectral imaging and biopsy navigation system for early gastrointestinal tumors provided in this application includes: The data acquisition module is used to collect re-inspection data during the re-inspection operation of the target tissue by the detection equipment; The data processing module is used to process the re-inspection data to obtain re-inspection fingerprint data.

[0052] The re-examination data includes re-examination images acquired by endoscopy, re-examination CT spectra acquired by CT equipment, re-examination magnetic field data acquired by magnetic field induction chip, and re-examination position of sampling instrument; the re-examination operation is performed after the target time period of the sampling operation.

[0053] In some embodiments, the re-examination image may include a dual-spectral image acquired by an endoscope during the re-examination operation; the re-examination CT spectrum may include CT spectrum data acquired by a CT device during the re-examination operation; the re-examination magnetic field data may include magnetic field coordinate data acquired by a magnetic field sensing chip during the re-examination operation; and the re-examination position may include the instrument position of the sampling instrument obtained during the re-examination operation.

[0054] In some embodiments, re-examination fingerprint data can uniquely characterize the re-examination process for a sampling operation; for example, re-examination fingerprint data can be obtained in the following ways: Feature extraction is performed on the re-examination CT spectrum to obtain re-examination CT spectral features. The re-examination location is integrated to obtain re-examination location features. Feature extraction is performed on the re-examination image to obtain re-examination dual-spectral features. If the re-examination operation requires sampling, the sampling intensity corresponding to the re-examination operation is obtained and its features are extracted to obtain re-examination intensity features. Thus, the re-examination CT spectral features, re-examination location features, and re-examination dual-spectral features are spliced ​​together to obtain re-examination fingerprint data. Alternatively, the re-examination CT spectral features, re-examination location features, re-examination dual-spectral features, and re-examination intensity features are spliced ​​together to obtain re-examination fingerprint data.

[0055] As can be seen from the above, in the endoscopic early gastrointestinal tumor dual-spectrum imaging and biopsy navigation system provided in this application embodiment, the acquisition module is used to acquire re-examination data during the re-examination operation of the target tissue through the detection device. The re-examination data includes re-examination images acquired by the endoscope, re-examination CT spectra acquired by the CT device, re-examination magnetic field data acquired by the magnetic field induction chip, and the re-examination position of the sampling instrument. This improves the comprehensiveness and completeness of the re-examination data. Furthermore, the data processing module is used to process the re-examination data to obtain re-examination fingerprint data. The re-examination operation is performed after the target time period when the sampling instrument performs the sampling operation. Thus, the execution status of the re-examination operation can be comprehensively and accurately characterized through the re-examination fingerprint data.

[0056] Based on the foregoing embodiments, the endoscopic dual-spectral imaging and biopsy navigation system for early gastrointestinal tumors provided in this application includes: The data processing module is also used to generate a re-examination control signal based on the fingerprint matching degree between the re-examination fingerprint data and the sampled fingerprint data; The biopsy navigation module is used to control the pose of the endoscope based on the re-examination control signal.

[0057] Among them, the sampled fingerprint data is associated with the source data; the re-examination fingerprint data is associated with the re-examination data; the re-examination data is collected by the acquisition module during the re-examination operation of the detection equipment on the target tissue; the re-examination data includes the re-examination images acquired by the endoscope, the re-examination CT spectrum acquired by the CT equipment, the re-examination position of the sampling instrument, and the re-examination force of the sampling instrument.

[0058] In some embodiments, if it is necessary to perform sampling again and / or acquire CT spectral data during a re-examination, the sampling instrument and / or CT equipment can be synchronously controlled based on the re-examination control signal.

[0059] In some embodiments, fingerprint matching degree can characterize the similarity between data in the re-examined fingerprint data and data in the sampled fingerprint data.

[0060] In some embodiments, the re-inspection control signal can be determined in the following ways: If the fingerprint matching degree is greater than or equal to a preset target threshold, the re-inspection control signal may include: determining a first step based on the fingerprint matching degree, and gradually adjusting at least one of the position, angle, and attitude of the target device during the re-inspection operation based on the first step; if the fingerprint matching degree is less than the target threshold, the re-inspection control signal may include: determining a second step based on the fingerprint matching degree, and gradually adjusting at least one of the position, angle, and attitude of the target device during the re-inspection operation based on the second step; wherein, the first step may be less than the second step, and the target device may include at least some of the detection devices corresponding to the target data to be collected during the re-inspection process.

[0061] As can be seen from the above, in the endoscopic early gastrointestinal tumor dual-spectrum imaging and biopsy navigation system provided in this application embodiment, the data processing module is also used to generate a re-examination control signal based on the fingerprint matching degree between the re-examination fingerprint data and the sampling fingerprint data. The sampling fingerprint data is associated with the source data, and the re-examination fingerprint data is associated with the re-examination data. The re-examination data is acquired by the acquisition module during the re-examination operation of the detection device on the target tissue. The re-examination data includes the re-examination image acquired by the endoscope, the re-examination CT spectrum acquired by the CT device, the re-examination position of the sampling instrument, and the re-examination force of the sampling instrument. In this way, the re-examination control signal is associated with both the source data and the re-examination data, thereby improving the accuracy of the re-examination control signal. On this basis, the biopsy navigation module is used to control at least the pose of the endoscope based on the re-examination control signal, which can realize targeted adjustment of the pose of the endoscope, thereby improving the efficiency and accuracy of the re-examination operation.

[0062] Based on the foregoing embodiments, in the endoscopic early gastrointestinal tumor dual-spectrum imaging and biopsy navigation system provided in this application, the data processing module is used to determine the m-th matching data based on the degree of matching between the m-th re-examination feature in the re-examination fingerprint data and the m-th sampling feature in the sampling fingerprint data, to perform statistics on the first matching data to the M-th matching data to obtain the fingerprint matching degree, and to generate a re-examination control signal based on the fingerprint matching degree.

[0063] Where m is an integer greater than or equal to 1 and less than or equal to M, and M is an integer greater than 1, used to characterize the number of features contained in the re-examination fingerprint data.

[0064] In some embodiments, if the re-examination CT spectrum and / or re-examination image are not acquired during the re-examination operation, the CT spectrum data in the source data can be identified as the re-examination image, and / or the bispectral image in the source data can be identified as the re-examination image. In this case, in the process of determining the degree of matching, the matching data between the CT spectrum data and the re-examination CT spectrum can be omitted, and / or the matching data between the bispectral data and the re-examination image can be omitted.

[0065] In some embodiments, the matching data may include the similarity between the m-th re-examination feature in the re-examination fingerprint data and the m-th sampling feature in the sampling fingerprint data.

[0066] In some embodiments, if the re-inspection operation does not require re-sampling, then the matching data between the sampling intensity and the re-inspection intensity may not need to be calculated.

[0067] In some embodiments, if the sampling operation is performed again during the re-inspection operation, the location matching data between the sampling location and the re-inspection location can be calculated in the following manner: Obtain the first coordinate corresponding to the sampling position, obtain the second coordinate corresponding to the re-inspection position, and then determine the position matching data based on the spatial distance between the first coordinate and the second coordinate; specifically, it can be shown in equation (1): (1) in, For location matching data, As the first coordinate, The second coordinate, The spatial extent of the risk area before the sampling operation is performed.

[0068] For example, the CT matching data between the re-examination CT spectrum and the sampled CT spectrum data, the image matching data between the re-examination image and the bispectral image, and the intensity matching data between the sampling intensity and the re-examination intensity can all be calculated using equation (2): (2) in, This refers to any one of the following: CT matching data, image matching data, and force matching data. These refer to any one of the following: re-examination CT spectrum, re-examination image, and re-examination intensity. This can be any of the following: CT spectral data, bispectral images, or sampling intensity.

[0069] For example, fingerprint matching accuracy can be obtained in the following way: The fingerprint matching degree is obtained by weighting the first matching data to the Mth matching data based on the weights in the weight set; specifically, it can be achieved by equation (3): (3) in, For fingerprint matching accuracy, ( , , , ) is the weight set, Matching data to images, Matching data for CT scans, For force matching data, the first matching data to the Mth matching data may include position matching data, image matching data, force matching data and CT matching data.

[0070] It should be noted that the weights in the weight set can be predetermined or adjusted according to at least one of the following: the distribution of the target tissue, the risk site, the patient's age, and physical condition.

[0071] In some embodiments, the re-inspection control signal can be generated in any of the following ways: Based on the relationship between the matching score corresponding to the fingerprint matching degree and the pre-set threshold set, a re-inspection control signal is generated.

[0072] Specifically, if the matching score is greater than or equal to the first threshold, a re-examination control signal can be generated to guide the displacement of the sampling instrument and / or endoscope based on the degree of difference between the sampling position and the re-examination position. For example, if the first threshold is 0.8, and the matching score is greater than or equal to 0.8, the following re-examination control signal can be output to guide the endoscope and / or sampling instrument: The sampling position is at the first position. Please move to the left by a first distance and move upward by a second distance to reach the second position, starting from the current position. For example, the re-examination control signal can also be output through the human-computer interaction interface, and the second position may include the location of the risk site.

[0073] Specifically, if the matching score is less than the first threshold but greater than or equal to the second threshold, the following re-examination control signal can be output: control the endoscope and / or sampling instrument to continue moving and re-collect the sampling position; wherein, the second threshold can be 0.6.

[0074] For example, if the matching score is less than the second threshold but greater than or equal to the third threshold, the following re-examination control signal can be output: redeploy the magnetic markers and acquire the re-examination CT spectrum; wherein, the third threshold can be 0.4.

[0075] As can be seen from the above, in the endoscopic early gastrointestinal tumor dual-spectrum imaging and biopsy navigation system provided in this application embodiment, the data processing module is used to determine the m-th matching data based on the matching degree between the m-th re-examination feature in the re-examination fingerprint data and the m-th sampling feature in the sampling fingerprint data, and to perform statistics on the first matching data to the M-th matching data to obtain the fingerprint matching degree between the re-examination fingerprint data and the sampling fingerprint data. In this way, by associating the fingerprint matching degree with all features in the re-examination fingerprint data and the sampling fingerprint data through the above operations, the comprehensiveness and accuracy of the matching degree can be improved. On this basis, the re-examination control signal can be generated based on the fingerprint matching degree, which can improve the accuracy of the re-examination control signal.

[0076] Based on the foregoing embodiments, in the endoscopic early gastrointestinal tumor dual-spectrum imaging and biopsy navigation system provided in this application, the data processing module is used to determine a weight set based on the time interval between the re-examination operation and the sampling operation, and to perform statistics on the first matching data to the Mth matching data based on the weight set to obtain the fingerprint matching degree.

[0077] In some embodiments, the time interval between the re-inspection operation and the sampling operation can be the target time period in the foregoing embodiments. In practical applications, as the target time period extends, the degree of morphological difference in the sampling operation site in the target tissue will increase. Therefore, determining or adjusting the weight set according to the time interval or the length of the target time period can reduce the probability of insufficient matching accuracy due to excessively long time intervals.

[0078] In some embodiments, the weight set can be determined in the following way: The weights corresponding to the re-examination images, the re-examination CT spectra, and the re-examination intensity increase with the increase of the time interval, while the weights corresponding to the re-examination positions decrease with the increase of the time interval. The weights corresponding to the re-examination images, the re-examination CT spectra, the re-examination intensity, and the re-examination positions are defined as a weight set; and the sum of each weight can be 1.

[0079] As can be seen from the above, in the endoscopic early gastrointestinal tumor dual-spectrum imaging and biopsy navigation system provided in this application embodiment, a weight set is determined based on the time interval between the re-examination operation and the sampling operation, so that the weights in the weight set can reflect the change in the time interval between the re-examination operation and the sampling operation; and, based on the weight set, the fingerprint matching degree is obtained by statistically analyzing the first matching data to the Mth matching data, which can reduce the probability of insufficient fingerprint matching degree accuracy due to excessive time interval, thereby improving the accuracy of fingerprint matching degree.

[0080] Based on the foregoing embodiments, in the endoscopic early gastrointestinal tumor dual-spectrum imaging and biopsy navigation system provided in this application, the data processing module is used to determine the regional matching degree between the tissue region identifier and the instrument region identifier. If the regional matching degree is less than or equal to the matching degree threshold, a navigation control signal is generated based on the regional matching degree.

[0081] Correspondingly, if the regional matching degree is greater than the target threshold, the sampling device can be controlled to perform a sampling operation.

[0082] In some embodiments, tissue region identification can be determined by a biopsy probability feature map; for example, the biopsy probability feature map can be obtained by processing bispectral images and CT spectral data, wherein the biopsy probability feature map can quantify the tumor probability of risk sites and display the tumor probability quantification results intuitively; for example, the area where the risk site indicated by the tissue region identification in the biopsy probability feature map is located can also be marked with the tumor probability quantification results, and the area where the risk site is located can be marked in the form of a closed curve in the biopsy probability feature map.

[0083] In some embodiments, the region matching degree can characterize the degree of area overlap between the area where the risk site is located and the area where the sampling device is located; for example, the region matching degree can be calculated by equation (4): (4) in, For regional matching degree, The area of ​​the region where the sampling device is located. This refers to the area where the risky location is situated.

[0084] In some embodiments, navigation control signals can be generated in the following ways: Based on the relative regional relationship between the area where the sampling device is located and the area where the risk site is located, and combined with the regional matching degree, a navigation control signal is generated. The relative regional relationship may include the distance and / or spatial orientation between the area where the sampling device is located and the area where the risk site is located. Accordingly, the navigation control signal may include: gradually increasing the regional matching degree by adjusting at least one of the position, attitude and angle of the sampling device until the regional matching degree is greater than the matching degree threshold.

[0085] For example, the matching threshold can be 0.8.

[0086] For example, if the area matching degree is less than or equal to the matching degree threshold, vibration parameters can be determined, and the vibration state of at least one handle associated with the detection device can be controlled by the vibration parameters to provide intuitive prompts for navigation control signals; for example, the vibration parameters may include vibration frequency and vibration intensity, and the vibration intensity and vibration frequency corresponding to the vibration state may increase as the area matching degree decreases.

[0087] As described above, in the endoscopic early gastrointestinal tumor dual-spectrum imaging and biopsy navigation system provided in this application embodiment, the data processing module is used to determine the regional matching degree between the tissue region identifier and the instrument region identifier. If the regional matching degree is less than or equal to the matching degree threshold, a navigation control signal is generated based on the regional matching degree. This not only achieves tracking-based judgment of the regional matching degree between the tissue representation and the instrument region identifier, but also realizes conditional control of the navigation control signal generation operation, thereby improving the correlation between the navigation control signal and the tissue region identifier and the instrument region identifier, and improving the dynamics and accuracy of the navigation control signal.

[0088] Based on the foregoing embodiments, in the endoscopic early gastrointestinal tumor dual-spectrum imaging and biopsy navigation system provided in this application embodiment, the data processing module is used to control the sampling intensity of the acquisition module if the regional matching degree is greater than the matching degree threshold, and to generate a sampling control signal if the sampling intensity is greater than or equal to the intensity threshold.

[0089] The sampling control signal is used to control the sampling instrument to perform the sampling operation.

[0090] Accordingly, if the regional matching degree is less than the matching degree threshold, navigation control signals can be generated using the method provided in the aforementioned embodiments.

[0091] For example, if the sampling force is less than the force threshold, a prompt message can be output to prompt the sampling component to increase the sampling force.

[0092] In some embodiments, the sampling control signal can be generated in the following manner: If the sampling force is greater than or equal to the force threshold, and the duration for which the sampling force remains greater than or equal to the force threshold is greater than or equal to the time period threshold, then a sampling control signal is generated.

[0093] For example, if the sampling force is greater than or equal to the force limit during the sampling operation, the sampling can be interrupted to reduce the probability of damage to the target tissue caused by excessive sampling force; wherein, the force limit may include a force value in which the probability of damage to the target tissue is greater than or equal to a probability threshold.

[0094] As can be seen from the above, in the endoscopic early gastrointestinal tumor dual-spectrum imaging and biopsy navigation system provided in this application embodiment, the data processing module controls the acquisition module to acquire sampling force if the region matching degree is greater than the matching degree threshold, and generates a sampling control signal to control the sampling instrument to perform sampling operations if the sampling force is greater than or equal to the force threshold. Thus, through the above operations, precise control of the sampling operation is achieved. Furthermore, by simultaneously associating the region matching degree and the sampling force to the sampling control signal, the probability of erroneous execution of the sampling operation can be reduced, and the accuracy of the sampling operation can be improved.

[0095] Based on the foregoing embodiments, in the endoscopic early gastrointestinal tumor dual-spectral imaging and biopsy navigation system provided in this application embodiment, the data processing module is used to generate a spectral feature map that includes at least risk sites based on CT spectral data and dual-spectral images, and to process the spectral feature map to obtain a biopsy probability feature map.

[0096] The biopsy probability feature map includes tissue region identifiers.

[0097] In some embodiments, the spectral feature map may include the risk sites of the target tissue and the degree of absorption and / or reflection of light of different wavelengths by normal tissue; for example, the spectral feature map may be obtained in the following ways: The CT spectral data is analyzed to determine the voxel coordinate set corresponding to the tumor characteristic absorption peaks in the CT spectral data. Then, the voxel coordinate set is converted into an electromagnetic coordinate set in a second coordinate system using a first transformation matrix, and then converted into a pixel coordinate set in a third coordinate system using a second transformation matrix. Next, the set of pixels corresponding to the dual-spectral image and the pixel coordinate set is weighted and corrected to obtain a spectral feature map. For example, if the CT spectral data shows that the risk area has an absorption peak in the 532nm band, then the corresponding pixel set in the dual-spectral image is weighted and enhanced based on the pixel coordinate set corresponding to the absorption peak to highlight the difference between the risk area and normal tissue.

[0098] For example, the pixel enhancement result obtained by weighting the pixel at position (x,y) in the corresponding pixel set in the bispectral image. It can be calculated using equation (5): (5) in, and As weight, The values ​​are for the absorption peak in the 532 nm band in the CT spectral data. The values ​​are for the absorption peak in the 650 nm band in the CT spectral data. The average absorption coefficients of normal tissues, such as normal mucosa, for the two aforementioned wavelength bands in CT spectral data. The light intensity value of the dual-spectral image in the 532nm band. The light intensity value of the dual-spectral image in the 650nm band.

[0099] In some embodiments, the biopsy probability feature map can be obtained in the following ways: An initial tumor feature score is determined for each pixel in the spectral feature map. Then, based on the continuity between mucosal tissues in the target tissue, the initial tumor feature score is corrected to obtain a corrected score. The corrected score is then divided into intervals to obtain the final tumor probability for each pixel. The set of pixels whose final tumor probability is greater than or equal to the probability threshold is marked in the spectral feature map to obtain the tissue region identifier. The spectral feature map marked with the tissue region identifier is determined as the biopsy probability feature map. For example, the tissue region identifier can be marked in the form of a closed curve.

[0100] As described above, in the endoscopic early gastrointestinal tumor dual-spectrum imaging and biopsy navigation system provided in this application embodiment, the data processing module is used to generate a spectral feature map that includes at least the risk sites based on CT spectral data and dual-spectrum images. The spectral feature map is then processed to obtain a biopsy probability feature map, which includes tissue region identification. Thus, through the spectral feature map and the biopsy probability feature map, a gradual and progressive, intuitive and visual representation of the risk sites can be achieved, thereby improving the intuitiveness and accuracy of tissue region identification.

[0101] Figure 2 This is another structural schematic diagram of the endoscopic early gastrointestinal tumor dual-spectrum imaging and biopsy navigation system provided in the embodiments of this application, as shown below. Figure 2 As shown, the endoscopic early gastrointestinal tumor dual-spectrum imaging and biopsy navigation system 100 may include a multi-source data acquisition module, a multi-source modal data fusion calibration module, a biopsy instrument intelligent navigation control module, and a fingerprint generation module.

[0102] For example, the multi-source data acquisition module can be the acquisition module in the foregoing embodiments, used to acquire bispectral data, electromagnetic navigation data, CT spectral data and biopsy instrument status; wherein, the electromagnetic navigation data can be the magnetic field coordinate data in the foregoing embodiments, and the biopsy instrument status can be the sampling status in the foregoing embodiments.

[0103] For example, the multi-source modal data fusion calibration module and the fingerprint generation module can be the data processing modules in the foregoing embodiments.

[0104] For example, the multi-source modal data fusion calibration module can perform dynamic anchor point coordinate calibration, dynamic bispectral-CT spectral feature fusion, and optical biopsy probability map generation operations; wherein, dynamic anchor point coordinate calibration may include calibrating the coordinates of the endoscope and sampling instruments by using the coordinates of pre-set degradable magnetic marker points, and determining the first transformation matrix and the second transformation matrix by dynamic bispectral-CT spectral feature fusion, and processing the CT spectral data and bispectral data based on the first transformation matrix and the second transformation matrix to generate an optical biopsy probability map; for example, the optical biopsy probability map may be the biopsy probability feature map in the aforementioned embodiments.

[0105] For example, the biopsy instrument intelligent navigation control module can be the biopsy navigation module in the foregoing embodiments; wherein, the biopsy instrument intelligent navigation control module generates navigation control signals through the method provided in the foregoing embodiments, and realizes intelligent navigation of the sampling instrument based on the navigation control signals.

[0106] For example, the fingerprint generation module can generate sampled fingerprint data when the sampling operation is performed, and can also process the re-examination data using the method provided in the foregoing embodiments to generate re-examination fingerprint data when the re-examination operation is performed.

[0107] The above structure enables the organic integration of multi-source data acquisition, fusion calibration, intelligent navigation, and fingerprint generation, thereby automating and intelligently planning the biopsy path, improving the accuracy of biopsy sampling, and reducing the probability of missed and misdiagnosed gastrointestinal tumors.

[0108] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0109] The methods disclosed in the various method embodiments provided in this application can be arbitrarily combined to obtain new method embodiments without conflict.

[0110] The features disclosed in the various product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0111] The features disclosed in the various method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0112] It should be noted that the aforementioned computer-readable storage media can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM), etc.; or it can be various electronic devices including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0113] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0114] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0115] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware nodes. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0116] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0117] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0118] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0119] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. An endoscopic dual-spectral imaging and biopsy navigation system for early gastrointestinal tumors, characterized in that, include: The acquisition module is used to acquire source data during the process of the detection device probing the target tissue; wherein, the detection device includes an endoscope, a CT device, a magnetic field sensing chip, and a sampling instrument; the target tissue includes at least a portion of the gastrointestinal tract; the source data includes bispectral images acquired by the endoscope, CT spectral data acquired by the CT device, magnetic field coordinate data acquired by the magnetic field sensing chip, and the sampling status of the sampling instrument; The data processing module is used to determine the tissue region identifier based on the CT spectral data, the bispectral image, and the magnetic field coordinate data in the source data, and to generate a navigation control signal based on the instrument region identifier corresponding to the sampling state and the tissue region identifier; wherein, the tissue region identifier includes the identifier of the area where the risk site contained in the target tissue is located. A biopsy navigation module is used to navigate the sampling instrument to the area where the risk site is located based on the navigation control signal, and to control the sampling instrument to perform a sampling operation on the risk site to obtain sampled tissue; wherein the sampled tissue is used to perform a biopsy.

2. The system according to claim 1, characterized in that, The acquisition module is used to acquire the electromagnetic coordinates of the sampling device and the sampling force of the sampling device during the sampling operation performed by the sampling device. The data processing module is used to extract features from the CT spectral data to obtain the sampling CT spectral features of the risk site, integrate the electromagnetic coordinates to obtain the sampling position features of the sampling instrument, extract features from the bispectral image corresponding to the risk site to obtain sampling bispectral features, extract features from the sampling force to obtain sampling force features, and determine sampling fingerprint data based on the sampling CT spectral features, the sampling position features, the sampling bispectral features, and the sampling force features.

3. The system according to claim 2, characterized in that, The biopsy navigation module is used to generate a re-examination control signal based on the sampled fingerprint data; wherein, the re-examination control signal is used to control at least a portion of the detection equipment to move to the area where the risk site is located in order to perform a re-examination operation; the re-examination operation is performed after the target time period of the sampling operation.

4. The system according to claim 1, characterized in that, The acquisition module is used to acquire re-examination data during the re-examination operation of the target tissue by the detection device; wherein, the re-examination data includes the re-examination image acquired by the endoscope, the re-examination CT spectrum acquired by the CT device, the re-examination position of the sampling instrument, and the re-examination force of the sampling instrument; The data processing module is used to process the re-inspection data to obtain re-inspection fingerprint data; wherein the re-inspection operation is performed after the target time period of the sampling operation.

5. The system according to claim 1, characterized in that, The data processing module is further configured to generate a re-examination control signal based on the fingerprint matching degree between the re-examination fingerprint data and the sampled fingerprint data; wherein, the sampled fingerprint data is associated with the source data; the re-examination fingerprint data is associated with the re-examination data; the re-examination data is acquired by the acquisition module during the re-examination operation performed by the detection device on the target tissue; the re-examination data includes the re-examination image acquired by the endoscope, the re-examination CT spectrum acquired by the CT device, the re-examination position of the sampling instrument, and the re-examination force of the sampling instrument; The biopsy navigation module is used to control at least the pose of the endoscope based on the re-examination control signal.

6. The system according to claim 5, characterized in that, The data processing module is used to determine the m-th matching data based on the matching degree between the m-th re-examination feature in the re-examination fingerprint data and the m-th sampling feature in the sampling fingerprint data, to perform statistics on the first matching data to the M-th matching data to obtain the fingerprint matching degree, and to generate the re-examination control signal based on the fingerprint matching degree; m is an integer greater than or equal to 1 and less than or equal to M, M is an integer greater than 1, and is used to characterize the number of features contained in the re-examination fingerprint data.

7. The system according to claim 6, characterized in that, The data processing module is used to determine a weight set based on the time interval between the re-inspection operation and the sampling operation, and to perform statistics on the first matching data to the Mth matching data based on the weight set to obtain the fingerprint matching degree.

8. The system according to claim 1, characterized in that, The data processing module is used to determine the regional matching degree between the tissue region identifier and the device region identifier. If the regional matching degree is less than or equal to the matching degree threshold, the navigation control signal is generated based on the regional matching degree.

9. The system according to claim 8, characterized in that, The data processing module is used to control the acquisition module to acquire sampling force if the region matching degree is greater than the matching degree threshold, and to generate a sampling control signal if the sampling force is greater than or equal to the force threshold; wherein the sampling control signal is used to control the sampling device to perform the sampling operation.

10. The system according to any one of claims 1 to 9, characterized in that, The data processing module is used to generate a spectral feature map that includes at least the risk site based on the CT spectral data and the bispectral image, and to process the spectral feature map to obtain a biopsy probability feature map; wherein the biopsy probability feature map includes the tissue region identifier.

Images