Matching of luminescence image segmentation limited to the analysis region
By segmenting luminescent images into analysis regions and using matching indicators, the method addresses the challenge of accurate lesion recognition in fluorescence imaging, improving surgical and diagnostic precision.
Patent Information
- Application Number
- JP2022562575
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2020-04-14
- Filing Date
- 2021-03-25
- Publication Date
- 2025-10-07
- Estimated Expiration
- 2041-03-25
Smart Images

Figure 0007750861000001 
Figure 0007750861000002 
Figure 0007750861000003
Abstract
Description
[Technical Field]
[0001] The present disclosure relates to imaging applications, and more particularly to luminescence imaging. [Background technology]
[0002] The background of the present disclosure is introduced below with a discussion of technology relevant to its context. However, this discussion's reference to documents, acts, artifacts, or the like does not imply or represent that the technology discussed is part of the prior art or is common general knowledge in the field relevant to the present disclosure.
[0003] Luminescence imaging, and in particular fluorescence imaging, are specific imaging techniques used to obtain images that provide a visual representation of objects even when they are not directly visible. Luminescence imaging is based on the phenomenon of luminescence, which consists of the emission of light by luminescent substances when subjected to excitation, distinct from heating, and in particular the phenomenon of fluorescence, which occurs in fluorescent substances (called fluorophores). These emit light (fluorescence) when illuminated. For this purpose, fluorescence images are generally displayed to represent different locations of an object and the fluorescent light emitted by the fluorophores present there.
[0004] Fluorescence imaging is routinely used in medical devices to examine (internal) body parts of patients. In this case, a fluorescent agent (possibly configured to reach a specific molecule of a desired target, e.g., a lesion such as a tumor, and remain fixed there in a fluorescence molecular imaging (FMI) application) is typically administered to the patient. The representation of the (fixed) fluorescent agent in the fluorescence image facilitates the identification (and quantification) of the target. This information can be used in several medical applications, e.g., in surgical applications to recognize the margins of a lesion to be excised, in diagnostic applications to locate or monitor a lesion, and in therapeutic applications to delineate a lesion to be treated.
[0005] However, accurate recognition of lesions remains quite challenging because it is adversely affected by several interfering factors. For example, lesions can emit variable amounts of fluorescent light due to differences in the biology and perfusion of their tissues. Furthermore, other parts of the field of view that are different from the lesions can emit interfering fluorescent light that varies according to their type. For example, interfering light can come from surgical instruments, hands, surgical tools, surrounding body parts (e.g., the skin around the surgical cavity or unrelated organs within it), and background materials.
[0006] In particular, fluorescence images are often divided into segments with approximately homogeneous properties to distinguish fluorophores (and corresponding targets) from the rest of the field of view. With particular reference to medical applications, segmentation of fluorescence images can be used to distinguish lesions from healthy tissue. For this purpose, body part locations are classified into lesion segments or healthy tissue segments by comparing the corresponding (fluorescence) values of each fluorescence image with a segmentation threshold. The segmentation threshold is generally calculated automatically according to the statistical distribution of fluorescence values. However, interfering light biases the statistical distribution of fluorescence values and thus the segmentation threshold (increasing or decreasing it). This carries the risk of misclassifying body part locations.
[0007] For example, in surgical applications, this leads to uncertainty regarding the accurate recognition of lesion margins (with the risk of incomplete resection of the lesion or excessive removal of healthy tissue). In diagnostic applications, this negatively impacts the identification and / or quantification of the lesion, which may lead to erroneous interpretations (risk of false positives / negatives and incorrect follow-up). In therapeutic applications, this negatively impacts the delineation of the lesion to be treated (with the risk of reduced efficacy of the treatment or damage to healthy tissue).
[0008] U.S. Publication No. 2019 / 030371 discloses an approach for segmenting 3D medical images by using a first neural network operating in 2D or 2.5D to generate a quick estimate of smaller regions of interest within a larger 3D structure, and then using a second neural network operating in 3D to obtain a more accurate segmentation of those regions of interest. Summary of the Invention [Means for solving the problem]
[0009] A simplified summary of the disclosure is presented here to provide a basic understanding of the disclosure. However, its sole purpose is to introduce some concepts of the disclosure in a simplified form as a prelude to the more detailed description below. It is not intended to identify key elements or delineate the scope of the disclosure.
[0010] In general terms, the present disclosure is based on the idea of matching segmentations when limited to an analysis domain.
[0011] In particular, one aspect provides a method for imaging a field of view that includes a target that includes a luminescent material, the method including establishing an analysis region in a portion of the luminescent image that surrounds a suspect representation of the target, and dividing the analysis region into a luminescent agent detection segment and a non-luminescent agent detection segment. The luminescent image is then displayed with the detected segments highlighted according to a matching indicator based on the quality of the segmentation.
[0012] A further aspect provides a computer program for implementing this method.
[0013] A further aspect provides a corresponding computer program product.
[0014] A further aspect provides a system for implementing this method.
[0015] A further aspect provides a corresponding surgical method.
[0016] A further aspect provides a corresponding diagnostic method.
[0017] A further aspect provides a corresponding method of treatment.
[0018] More particularly, one or more aspects of the present disclosure are set forth in independent claims, and advantageous features thereof are set forth in dependent claims, the language of all claims being incorporated herein by reference in their entirety (with any advantageous feature provided with reference to any particular aspect applying mutatis mutandis to all other aspects). [Brief explanation of the drawings]
[0019] The techniques of the present disclosure, and additional features and advantages thereof, will be best understood by reference to the following detailed description, given solely by way of non-limiting indication when read in connection with the accompanying drawings, in which, for simplicity, corresponding elements are given equal or similar reference numerals, their descriptions will not be repeated, and the name of each entity will generally be used to indicate both its type and its attributes (e.g., value, content, and representation) in which:
[0020] [Figure 1] FIG. 1 shows a schematic block diagram of an imaging system that can be used to practice a technique according to an embodiment of the present disclosure. [Figure 2A] 1 illustrates various application examples of a technique according to an embodiment of the present disclosure. [Figure 2B] 1 illustrates various application examples of a technique according to an embodiment of the present disclosure. [Figure 2C] 1 illustrates various application examples of a technique according to an embodiment of the present disclosure. [Figure 3A] 1 illustrates various application examples of a technique according to an embodiment of the present disclosure. [Figure 3B] 1 illustrates various application examples of a technique according to an embodiment of the present disclosure. [Figure 3C] 1 illustrates various application examples of a technique according to an embodiment of the present disclosure. [Figure 3D] 1 illustrates various application examples of a technique according to an embodiment of the present disclosure. [Figure 4] 1 illustrates major software components that can be used to implement an approach according to one embodiment of the present disclosure. [Figure 5A] 1 shows an operational diagram describing the flow of operations associated with implementing a technique according to one embodiment of the present disclosure. [Figure 5B] 1 shows an operational diagram describing the flow of operations associated with implementing a technique according to one embodiment of the present disclosure. [Figure 5C] 1 shows an operational diagram describing the flow of operations associated with implementing a technique according to one embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION
[0021] Referring specifically to FIG. 1, there is shown a schematic block diagram of an imaging system 100 that can be used to practice techniques according to embodiments of the present disclosure.
[0022] The imaging system 100 allows imaging of a scene in a corresponding field of view 103 (defined by the portion of the world within a solid angle to which the imaging system 100 is sensitive). For example, the imaging system 100 is used to assist a surgeon in a surgical application commonly known as fluorescence-guided surgery (FGS), and particularly in relation to tumors, fluorescence-guided resection (FGR). In this particular case, the field of view 103 pertains to a patient 106 undergoing a surgical procedure in which a fluorescent agent has been administered (e.g., adapted to accumulate in the tumor). The field of view 103 includes a body part 109 of the patient 106, and a surgical cavity 112 (e.g., a small skin incision in minimally invasive surgery) has been opened to expose a tumor 115 to be resected. The field of view 103 may also include one or more foreign objects distinct from the surgical cavity 112 (not shown), such as surgical instruments, hands, surgical tools, surrounding body parts, background material, etc. (surrounding or overlapping the surgical cavity 112).
[0023] The imaging system 100 comprises an imaging probe 127 for acquiring images of the field of view 103 and a central unit 121 for controlling its operation.
[0024] Starting from the imaging probe 118, it has an illumination unit (for illuminating the field of view 103) and an acquisition unit (for acquiring an image of the field of view 103) comprising the following components: In the illumination unit, an excitation light source 124 and a white light source 127 generate excitation light and white light, respectively. The excitation light has a wavelength and energy (e.g., of the near-infrared, or NIR, type) suitable for exciting the fluorophores of the fluorescent agent, while the white light appears substantially colorless to the human eye (e.g., contains all wavelengths of the spectrum visible to the human eye at equal intensity).
[0025] The corresponding transmission optics 130 and transmission optics 133 transmit the excitation light and the white light, respectively, to the (same) field of view 103. In the acquisition unit, collection optics 136 collect the light from the field of view 103 (in an epi-illumination configuration). The collected light comprises the fluorescence emitted by any fluorophores (illuminated by the excitation light) present in the field of view. Indeed, fluorophores enter an excited (electronic) state when they absorb the excitation light; the excited state is unstable, so that the fluorophores decay from there to the ground (electronic) state in a very short time, thereby emitting fluorescent light (at a characteristic wavelength longer than that of the excitation light due to the energy dissipated as heat in the excited state) with an intensity that depends on the amount of illuminated fluorophores (and other factors include the fluorophore position within the field of view 103 and the body-part 109).
[0026] Additionally, the collected light includes visible light, or reflected light in the visible spectrum reflected by any objects present in the field of view (illuminated by white light). A beam splitter 139 splits the collected light into two channels. For example, the beam splitter 139 may be a dichroic mirror that transmits and reflects the collected light at wavelengths above and below a threshold wavelength between the visible light spectrum and the fluorescent light spectrum (or vice versa). In the (transmitted) channel of the beam splitter 139, fluorescent light is defined by the portion of the collected light in that spectrum. An emission filter 142 filters the fluorescent light to remove excitation light (which may be reflected by the field of view) and ambient light (which may be generated by intrinsic fluorescence).
[0027] A fluorescence camera 145 (e.g., of the EMCCD type) receives the fluorescent light from the emission filter 142 and generates a corresponding fluorescence (digital) image representing the distribution of fluorophores in the field of view 103. In the other (reflected) channel of the beam splitter 139, with visible light defined by the portion of the collected light in its spectrum, a reflectance or photographic camera 148 (e.g., of the CCD type) receives the visible light and generates a corresponding reflectance or photographic (digital) image representing what is visible to the human eye in the field of view 103.
[0028] Moving to the central unit 121, it comprises several units connected to each other via a bus structure 151. In particular, one or more microprocessors (μP) 154 provide the logic capabilities of the central unit 121. A non-volatile memory (ROM) 157 stores basic code for bootstrapping the central unit 121, and a volatile memory (RAM) 160 is used by the microprocessors 154 as working memory. The central unit 121 is provided with a mass memory 163 (e.g., a solid-state disk or SSD) for storing programs and data.
[0029] Furthermore, the central unit 121 comprises a number of controllers 166 for peripherals or input / output (I / O) units. In particular, the controller 166 controls the excitation light source 124, the white light source 127, the fluorescence camera 145 and the photo camera 148 of the imaging probe 118. Furthermore, the controller 166 controls further peripherals (indicated generally by the reference numeral 169), such as one or more monitors for displaying images, a keyboard for entering commands, a trackball for moving a pointer on the monitor, a drive for reading and writing to a removable storage unit (e.g. a USB key), a network interface card (NIC) for connecting to a communication network (e.g. a LAN).
[0030] 2A-2C, various application examples of techniques according to an embodiment of the present disclosure are shown, in particular, Figures 2A-2C relate to a (basic) unassisted mode of operation, and Figures 3A-3D relate to a (advanced) assisted mode of operation of the imaging system.
[0031] 2A (unassisted mode of operation), a fluorescent image 205 is acquired of the field of view 103. The fluorescent image 205 provides a representation of the surgical cavity 112 (containing the tumor) and various foreign bodies surrounding the surgical cavity 112, including skin 210, tissue retractors 215, and clamps 220.
[0032] 2B, in an approach according to one embodiment of the present disclosure, an analysis region 225 is set in a portion of the fluorescence image 205 that surrounds the (suspected) representation of the tumor. In particular, in this particular implementation, the analysis region 225 has a predefined shape and size (a small square equal to a predefined portion of the fluorescence image 205) and is located in the center of the fluorescence image 205 (adjusted to include representations of the tumor and some adjacent healthy tissue, if possible).
[0033] 2C, the analysis region 225 is divided into a detection segment 230d and a non-detection segment 230n according to the (fluorescence) values of the fluorescence image 205 of only the analysis region 225. The detection segment 230d represents the detection of the fluorescent agent, and the non-detection segment 230n represents the non-detection of the fluorescent agent, i.e., in this case, the (suspected) tumor and the healthy tissue that defines its background, respectively.
[0034] For example, a segmentation threshold is calculated according to the statistical distribution of the fluorescence values of the analysis region 225. Corresponding locations of the field of view 103 are assigned to the detected segment 230d or the non-detected segment 230n according to a comparison of the fluorescence values with the segmentation threshold. Then, a matching indicator is determined based on the quality indicator of the segmentation of the analysis region 225.
[0035] For example, the average fluorescence values of the detected segment 230d (tumor intensity (TI) equals 6123 for the tumor in the figure) and the average fluorescence values of the non-detected segment 230n (background intensity (BI) equals 4386 for the healthy tissue background in the figure) are calculated. A matching indicator is set to a quality indicator given by the average value of the detected segment 230d divided by the average value of the non-detected segment (230n) (tumor-to-background ratio (TBR) equals 1.4 in the figure). The fluorescence image 205 is then displayed with the detected segment 230d highlighted according to the matching indicator.
[0036] For example, the matching indicator is compared to a predefined matching threshold (TH=1 in the figure) for the type of surgical procedure. If the matching indicator is higher than the matching threshold (in this case 1.4>1), the detection segment 230d is colored, while the rest remain black and white (not shown in the figure).
[0037] The above approach significantly facilitates the recognition of the tumor: in fact, the restriction of the segmentation to only the (small) analysis region 225 makes it possible to eliminate further variations in the fluorescence light due to interfering fluorescence light from other parts of the field of view 103 different from the tumor.
[0038] In particular, the statistical distribution of fluorescence values within the analysis region 225 is now unbiased (because it is not affected by interfering light that affects the rest of the fluorescence image 205). As a result, the segmentation threshold is more accurate, thereby reducing the risk of misclassification of locations in the field of view 103.
[0039] At the same time, visualization of the fluorescence image 205 (representing the entire field of view 103) provides a contextualized representation of the tumor within the body part, which in turn allows moving the analysis region 225 to further parts of the field of view 103 as needed.
[0040] In either case, the (dynamic) highlighting of the detected segment 230d according to the matching indicator makes it possible to recognize a tumor only if it really should be present. Indeed, if the quality of the segmentation is sufficiently good, it is very likely that the detected segment 230d represents a tumor. In this case, the highlighted visualization of the detected segment 230d is immediately obvious to the surgeon, so that the corresponding part of the body part can be excised with high confidence, which is related to a true positive. Conversely, if the quality of the segmentation is relatively poor, it is doubtful whether the detected segment 230d actually represents a tumor. In this case, the missing (or reduced) highlighting of the detected segment prevents the surgeon from acting on the corresponding part of the body part, since there is a risk of a false positive.
[0041] The above procedure significantly reduces the risk of incomplete tumor removal or excessive removal of healthy tissue, all of which has beneficial effects on the patient's health.
[0042] 3A (assisted mode of operation), a pair of corresponding reflected 205R and fluorescent 205F images have been acquired of the same field of view 103. The reflected 205R and fluorescent 205F images provide a simultaneous representation (in both visible and fluorescent light) of the surgical cavity 112 (with the tumor) and various foreign objects, including the skin 235, tissue retractors 240 surrounding the surgical cavity 112, and the surgeon's hand 245 (partially) overlapping the surgical cavity 112.
[0043] 3B , in one embodiment of the present disclosure, informational (reflection) regions 250Ri are identified in the reflected image 205R according to their content. The informational regions 250R represent the surgical cavity free of foreign objects (in this case, skin, tissue retractors, and the surgeon's hands) and represent the informational portions of the field of view 103 that are actually of interest for the surgical procedure (i.e., the region of interest or ROI). The remainder of the reflected image 205R then defines non-informational (reflection) regions 250Rn that represent foreign objects and represent the non-informational portions of the field of view 103 that are not of interest for the surgical procedure. For example, as described in more detail below, this result is achieved using a semantic segmentation technique (e.g., based on the use of neural networks).
[0044] The identification of informational regions 250Ri (and also non-information regions 250Rn) in the reflection image 205F is transferred to the fluorescence image 205F. In particular, (fluorescent) informational regions 250Fi are identified in the fluorescence image 205F that correspond to the informational regions 250Ri. As a result, the remainder of the fluorescence image 205F defines (fluorescent) non-informational regions 250Fn that correspond to the non-information regions 250Rn.
[0045] 3C, a similar analysis region 260 is set in a portion of the information region 250Fi (within the fluorescence image 205F) that surrounds the (suspected) representation of the tumor. In particular, in this particular implementation, the analysis region 260 has a predetermined defined shape and size (a small square equal to a predefined portion of the information region 250Fi) and is located in the center of the information region 250Fi (adjusted to include representations of the tumor and some adjacent healthy tissue, if possible).
[0046] Moving to FIG. 3D, the analysis region 260 is processed in the same manner as described above. Accordingly, the analysis region 260 is divided into a detected segment 265d and a non-detected segment 265n according to the fluorescence values of the fluorescence image 205F in the analysis region 260 alone (using a segmentation threshold calculated according to the statistical distribution of the fluorescence values). A matching indicator is determined based on the quality indicator of the segmentation of the analysis region 260 (e.g., set to the corresponding TBR). As shown, in this case, the fluorescence image (205F) is displayed with the detected segment 265d highlighted according to the matching indicator, e.g., colored if the matching indicator is higher than the matching threshold (otherwise remaining black and white, not shown).
[0047] The above implementation further improves the reliability of tumor recognition. Indeed, in this case, only the (informative) representation of the surgical cavity is taken into account, instead automatically ignoring the (non-informative) representation of foreign objects (those surrounding and / or overlapping it). This avoids (or at least substantially reduces) the negative influence of foreign objects on the segmentation of the analysis region 260. In particular, the statistical distribution of fluorescence values on which the segmentation of the analysis region 260 was based is now unbiased (because fluorescence values in the non-informative region 250Fn do not participate in it).
[0048] Referring now to FIG. 4, there is shown major software components that may be used to implement techniques according to embodiments of the present disclosure.
[0049] All software components (programs and data) are generally designated by the reference 400. The software components 400 are typically stored in mass memory and, when the programs are executed, are loaded (at least partially) into the working memory of a central unit of the imaging system, along with the operating system and other application programs not directly related to the disclosed techniques (omitted from the figure for simplicity). The programs may be initially installed into the mass memory, for example, from a removable storage unit or from a communications network. In this regard, each program may be a module, segment, or portion of code, which comprises one or more executable instructions for implementing specific logical functions.
[0050] A fluorescence manager 405 manages the fluorescence unit (including the excitation light source and the fluorescence camera) of a dedicated imaging system to acquire fluorescence images of a field of view appropriately illuminated for this purpose. The fluorescence manager 405 accesses (in write mode) a fluorescence image repository 410, which stores a series of fluorescence images acquired successively during an ongoing imaging process.
[0051] Each fluorescence image is defined by a bitmap containing a matrix of cells (e.g., 512 rows and 512 columns), each storing a pixel (fluorescence) value, i.e., the (fluorescence) value of an elementary pixel corresponding to a (fluorescence) location in the field of view. Each pixel value defines the brightness of the pixel as a function of the intensity of the fluorescent light emitted by that location and also as a function of the amount of fluorescent agent present there (e.g., from black to white as the amount of fluorescent agent increases).
[0052] Similarly, the reflectance manager 415 manages the reflectance unit (including a white light source and a photographic camera) of a dedicated imaging system to acquire reflectance images of a field of view appropriately illuminated for this purpose. The reflectance manager 415 accesses (in light mode) a reflectance image repository 420, which stores a series of reflectance images acquired consecutively during the same imaging process (synchronized with the corresponding fluorescence images in repository 410).
[0053] Each reflectance image is defined by a bitmap containing a matrix of cells (with the same or different size as for the fluorescence image), each storing a pixel (reflectance) value corresponding to a (reflectance) location in the field of view, where each pixel value defines the visible light (e.g., its RGB components) reflected by that location.
[0054] In the supported implementation, a reducer 425 reduces a (selected) fluorescence image to its information domain by semantically segmenting the corresponding reflectance image into its information and non-information domains (according to its content, possibly according to the content of the fluorescence image) and transferring the reflectance image segmentation to the fluorescence image. The reducer 425 accesses (in read mode) the reflectance image repository 420 and (optionally) the fluorescence image repository 410, and it accesses (in write mode) the reduction mask repository 430.
[0055] The reduction mask repository 430 stores (fluorescence) reduction masks that define information / non-information regions of the fluorescence image. The reduction mask is formed by a matrix of cells with the same dimensions as the fluorescence image, each of which stores a reduction flag that indicates the classification of the corresponding pixel into an information region or a non-information region. For example, if the pixel belongs to an information region, the reduction flag is asserted (activated) (e.g., logic value 1), and if the pixel belongs to a non-information region, the reduction flag is deasserted (deactivated) (e.g., logic value 0).
[0056] The reducer 425 also uses a variable (not shown) that stores a (reflection) reduction mask that similarly defines the information / non-information regions of the reflection image (i.e., a matrix of cells with the same size as the reflection image, each storing a reduction flag indicating the classification of the corresponding pixel into an information or non-information region). In either case, a setter 435 is used to set the analysis region (within the entire fluorescence image in unaided implementations, or within its information region in aided implementations). The setter 435 has access (in read mode) to the fluorescence image repository 410 and (possibly) the reduction mask repository 430.
[0057] Furthermore, the setter 435 exposes a user interface for manually adjusting the analysis region. The setter 435 accesses (in lite mode) the analysis mask repository 440. The analysis mask repository 440 stores analysis masks that define the (common) analysis region of the fluorescence image. The analysis mask is formed by a matrix of cells with the same dimensions as the fluorescence image, each storing an analysis flag that is asserted (e.g., logical 1) if the pixel belongs to the analysis region and deasserted (e.g., logical 0) otherwise.
[0058] A segmenter 445 uses a thresholding technique to divide the analysis regions of the fluorescence image into their detected and non-detected segments. The segmenter 445 accesses (in read mode) the fluorescence image repository 410, the analysis mask repository 440, and (possibly) the reduced mask repository 430, and it accesses (in write mode) the segmentation mask repository 450. The segmentation mask repository 450 has an entry for each fluorescence image in the corresponding repository 410, storing the segmentation mask that defines the detected / non-detected segments of the analysis regions of the fluorescence image.
[0059] The segmentation mask is formed by a matrix of cells having the same dimensions as the fluorescence image, each storing a segmentation flag that is asserted (e.g., logical value 1) if the pixel belongs to a detection segment of the analysis region, and deasserted (e.g., logical value 0) otherwise, i.e., if the pixel belongs to a non-detection segment of the analysis region or if it is outside the analysis region and possibly the information region.
[0060] The verifier 455 verifies the segmentations of the fluorescence images (by calculating corresponding match indicators and, if possible, comparing them to match thresholds). The verifier 455 has access (in read mode) to the fluorescence image repository 410, the segmentation mask repository 450, the analysis mask repository 440, and (if possible) the reduction mask repository 430. The verifier 455 also has access (in read mode) to the match threshold repository 460.
[0061] The matching threshold repository 460 stores one or more matching thresholds for corresponding targets (e.g., various types of tumors in various organs). These matching thresholds are predetermined from clinical evidence and / or research. For example, for each target, the same operations described above are applied during several surgical procedures, corresponding matching indicators (for their segmentation) are recorded, and the subsequent pathology report of the corresponding biopsy determines whether the body part of the detected segment is actually the target (positive) or not (negative).
[0062] The match threshold for the target is calculated to optimize its recognition accuracy, which is given by a match indicator that is higher than the match threshold for positive pathology reports and lower than the match threshold for negative pathology reports (e.g., to minimize false negatives where the match indicator is lower than the match threshold for positive pathology reports). The verifier 455 also exposes a user interface for manually selecting the target type for each imaging procedure. The verifier 455 accesses the match results repository 465 (in write mode, and also in read mode if possible).
[0063] The matching results repository 465 includes an entry for each fluorescence image in the corresponding repository 410. The entry stores the (matching) result of matching the segmentation of the fluorescence image. For example, the matching result is given by a matching indicator and / or a (matching) flag that is asserted (e.g., logical value 1) if the match is positive (the matching indicator is higher than the matching threshold) and deasserted (e.g., logical value 0) if the match is negative (the matching indicator is lower than the matching threshold), possibly accompanied by additional information (described below) that was used to match the segmentation.
[0064] A visualizer unit 470 visualizes the fluorescence images with (possibly) highlighting of their detected segments on the monitor of the imaging system. The visualizer unit 470 has access (in read mode) to the fluorescence image repository 410, the segmentation mask repository 450 and the matching result repository 465.
[0065] 5A-5C, various operational diagrams are shown describing the flow of operations associated with implementing techniques according to embodiments of the present disclosure.
[0066] In particular, the operational diagram depicts an exemplary process that may be used to image a patient during a surgical procedure using method 500. In this regard, each block may correspond to one or more executable instructions for implementing the specified logical function on a central unit of the imaging system.
[0067] Prior to a surgical procedure, a healthcare operator (e.g., a nurse) administers a fluorescent agent to a patient. The fluorescent agent (e.g., indocyanine green, methylene blue, etc.) is adapted to reach a specific (biological) target, such as a tumor to be resected, and remain substantially immobilized there. This result can be achieved by using either a non-targeted fluorescent agent (e.g., adapted to accumulate in the target body without specific interaction, e.g., passive accumulation) or a targeted fluorescent agent (adapted to attach to the target using specific interactions, e.g., by incorporating target-specific ligands into the fluorescent agent formulation based on chemical binding properties and / or physical structures adapted to interact with various tissues, vascular properties, metabolic properties, etc.).
[0068] The fluorescent agent is administered to the patient intravenously as a bolus (using a syringe), so that the fluorescent agent circulates within the patient's vasculature until it reaches the tumor and binds to it. Instead, the remaining (unbound) fluorescent agent is cleared from the blood pool (according to its corresponding half-life). After a waiting period (e.g., from a few minutes to 24-72 hours) during which the fluorescent agent accumulates in the tumor and is washed out from the rest of the patient's body, the surgical procedure can begin. Therefore, the operator switches the imaging system.
[0069] In response, the process begins by proceeding from black start circle 502 to block 504. At this point, if desired, the operator selects the type of target (i.e., organ and tumor) for the surgical procedure in the verifier's user interface. The verifier then retrieves the matching threshold for the (selected) target from the corresponding repository. The operator then positions the imaging probe in proximity to the area of the patient where the surgical cavity is being opened by the surgeon, and the operator enters a start command into the imaging system (e.g., using its keyboard).
[0070] In response, the fluorescence manager and reflectance manager turn on the excitation light source and white light source, respectively, to illuminate the field of view in block 506. Proceeding to block 508, the fluorescence manager and reflectance manager simultaneously acquire (new) fluorescence images and (new) reflectance images, respectively, and add them to their corresponding repositories. The fluorescence and reflectance images are thus acquired substantially simultaneously, providing different representations (in the fluorescence and visible light sense, respectively) of the same field of view that are spatially coherent (i.e., a predictable correlation exists between their pixels, even reaching perfect identity). The fluorescence / reflectance images are continuously displayed on the monitor in real time, allowing the operator to correct the position of the imaging probe and / or the patient.
[0071] These operations continue until the surgical cavity is properly imaged, and then the operator enters an initialization command into the imaging system (e.g., using its keyboard) in block 510 to set the analysis region in the (current) fluorescence image, possibly utilizing the corresponding (current) reflectance image. The flow of operations branches in block 512 according to the imaging system's operating mode (e.g., manually set during its configuration, defined by default or only available). In particular, in unassisted (operating) mode, block 514 is executed, while in assisted (operating) mode, blocks 516-520 are executed. In both cases, the flow of operations rejoins at block 522.
[0072] Now, referring to block 514 (unassisted mode), the setter sets the context space (for setting the analysis region) to the entire fluorescence image (extracted from the corresponding repository), and the process descends in block 522.
[0073] Referring instead to block 516 (assisted mode), the reducer segments the fluorescence image according to the semantic segmentation of the reflectance image (extracted from the corresponding repository) (as described in co-pending international application PCT / EP2021 / 054162). In essence, the reducer can apply one or more filters to improve the quality of the reflectance image. In particular, histogram equalization can be applied when the reflectance image is suitable for identifying information regions but is not very bright (its pixel value average falls between corresponding thresholds).
[0074] Additionally or alternatively, the reducer can shrink the reflectance image to reduce computational complexity, group its pixels into substantially homogeneous groups (each represented by a group value based on the corresponding pixel value) to simplify semantic segmentation, and apply a motion compensation algorithm (to align the reflectance image with the fluorescence image) and / or a warping algorithm (to correct distortion of the reflectance image relative to the fluorescence image). The reducer then semantically segments the (possibly preprocessed) reflectance image by applying a classification algorithm or deep learning techniques.
[0075] In the case of a classification algorithm, the reducer extracts one or more features from the reflectance image and possibly the fluorescence image by applying a filter that generates a corresponding feature map. The reducer computes a reflectance reduction mask by applying a specific classification algorithm, such as a conditional random field (CRF) algorithm, to the feature map. The reducer can refine the resulting reflectance reduction mask as needed (e.g., by assigning any truncated portion of a non-information region completely surrounded by an information region and / or by removing isolated misclassified pixels). Alternatively, in the case of a deep learning approach, the reducer applies the reflectance image (and possibly the fluorescence image) to a neural network (e.g., U-net), which directly generates the reflectance reduction mask.
[0076] The reducer divides the fluorescence image into its information and non-information regions by transferring the corresponding segmentation of the reflectance image thereto, in block 518. To this end, the reducer adapts the reflectance reduction mask to the fluorescence image (by shrinking / expanding it if it has a different size), if necessary. The reducer then sets the fluorescence reduction mask equal to the (adapted) reflectance reduction mask, if possible, and saves it in the corresponding repository. The setter then sets the context space (for setting the analysis region) to the information region of the fluorescence image defined by the (fluorescence) reduction mask, in block 520. The process then descends in block 522.
[0077] Referring now to block 522, the setter initializes the analysis region to have an initial shape (e.g., square) and an initial size. The initial size is equal to a default percentage (e.g., 1-10%, preferably 2-8%, more preferably 3-7%, e.g., 5%) of the size of the context space. For example, the analysis region is sized to include a number of pixels equal to a default percentage of the number of pixels in the context space (given by all cells of the fluorescence image in unaided mode, or by cells of the reduction mask whose reduction flag is asserted in assisted mode). The operational flow again branches in block 524 according to the (further) operational mode of the imaging system (e.g., manually set during its configuration, defined by default or the only available one). In particular, in manual (operational) mode, block 526 is executed, while in automatic (operational) mode, blocks 528-544 are executed. In both cases, the operational flow rejoins at block 546.
[0078] Referring now to block 526 (manual mode), the setter initializes the analysis region to have an initial position that is at the center of the context space. For example, a working frame is defined as the fluorescence image (unaided mode) or as the smallest rectangular area of the fluorescence image that encloses the information region (aided mode). The setter determines the center of the working frame and aligns the center of the analysis region with it. The process then descends to block 546.
[0079] Instead, referring to block 528 (automatic mode), the analysis region is initialized to the initial position of a best candidate region selected from among multiple candidate regions (which are candidates for initializing the analysis region). The candidate region is defined by moving a window with the initial shape and size of the analysis region across the same working frame as described above (i.e., the fluorescence image in unaided mode, or the smallest rectangular region of the fluorescence image that encloses the information region in aided mode).
[0080] For this purpose, first, a (current) candidate region is considered at the start of the working frame (e.g., its upper left corner). Then, a loop is entered in block 530, where the operational flow branches according to the selection criterion for the candidate region (e.g., manually set during its configuration, defined by default or the only available one). In particular, in the case of a selection criterion based on a quality indicator, blocks 532 to 534 are executed, while in the case of a selection criterion based on an intensity indicator, block 536 is executed. In both cases, the operational flow rejoins in block 538.
[0081] Now, with reference to block 532 (quality indicator), the setter instructs the segmenter to divide the candidate region into its (candidate) detected segments and (candidate) non-detected segments (as described below). The setter instructs the verifier to calculate a (candidate) quality indicator of the segmentation of the candidate region (as described below) in block 534. Alternatively, with reference to block 536 (intensity indicator), the setter calculates an intensity indicator of the candidate region according to its pixel values in the context space (i.e., all of them in unaided mode, or only those belonging to the information region in aided mode), e.g., equal to their average.
[0082] Now, moving to block 538, the setter compares the quality / strength indicator of the candidate region with a running value (initialized to a null value) consisting of the quality / strength indicator of the candidate region tentatively selected as the analysis region. If the quality / strength indicator is higher (possibly strictly higher) than the running value, this means that the candidate region is better than the (tentative) analysis region. Therefore, the setter tentatively sets the analysis region to the candidate region in block 540 and replaces its quality / strength indicator with the running value.
[0083] The process then continues to block 542. If the quality / strength indicator is lower (possibly strictly) than the running value (meaning the provisional analysis region is still so), the same point is reached directly from block 538. At this point, the setter checks whether the last candidate region has been processed. This occurs when the candidate region reaches the end of the working frame (opposite its start, e.g., its bottom right corner in the example problem).
[0084] If not, the setter advances to the next candidate region by moving along the scan path of the working frame in block 544. For example, the candidate region is moved horizontally away from the starting edge by one or more pixels (to the right) until the opposing boundary of the working frame is reached, then moved vertically within the working frame by one or more pixels (downward), then moved horizontally in the opposite direction (to the left) by one or more pixels until the opposing boundary of the working frame is reached, then moved vertically within the working frame by one or more pixels (downward).
[0085] The process then returns to block 530 to repeat the same operations on this next candidate region. The same operations can be performed (at least partially) simultaneously to reduce computation time. Conversely, once all candidate regions have been processed, the loop is exited by descending to block 546. Thus, at the exit of the loop, the analysis region is automatically set to the best candidate region.
[0086] Referring now to block 546, the setter adds the representation of the analysis region thus initialized to the fluorescence images continuously displayed on the monitor (e.g., by coloring the outline of the analysis region (e.g., red) and the fluorescence images in black and white). The surgeon may request the operator to manually adjust the analysis region via the setter's user interface in block 548. For example, a dial may be provided to change the size of the analysis region and / or four buttons may be provided to move the analysis region along corresponding directions (leftward, rightward, upward, and downward).
[0087] This allows for the inclusion of a suspected tumor representation within the analysis region even in abnormal situations (e.g., when the tumor is very large and / or close to the borders of the surgical cavity). Additionally or alternatively, the surgeon can request the operator to move the imaging probe or directly move the surgical cavity in block 550 (to obtain a counter-movement of the analysis region on the corresponding fluorescence / reflection image). In many practical situations, this allows for the inclusion of a suspected tumor representation within the analysis region without changing its size and position via the setter's user interface. This is particularly advantageous when only the surgical cavity is moved, as it allows the surgeon to adjust the analysis region themselves without contact with the imaging system (and without concerns about sterility).
[0088] Once the surgeon confirms that the analysis region is correctly positioned, the operator enters a confirmation command into the imaging system (e.g., using its keyboard) in block 552, and in response, the setter saves the analysis mask defining the analysis region thus determined in the corresponding repository.
[0089] From here, the analysis region in each (new) fluorescence image acquired is divided into its detected and non-detected segments. To this end, the segmenter determines in block 554 a segmentation threshold for the analysis region according to the statistical distribution of its pixel values that may lie within the information region, for example by applying Otsu's algorithm (so as to minimize the intra-class variance of the detected and non-detected segments).
[0090] For this purpose, the segmenter considers pixel values of the fluorescence image for which the analysis flag in the analysis mask is asserted, and furthermore, the reduction flag in the reduction mask is asserted in assisted mode (retrieved from the corresponding repository). In assisted mode, it is thus possible to position the analysis region across the contour of an information region (for example, when a tumor approaches it) without being adversely affected by the content of non-information regions of the fluorescence image.
[0091] The segmenter generates a segmentation mask in block 556 by comparing pixel values in the analysis region, and possibly in the information region, with a segmentation threshold. In particular, for each pixel for which the analysis flag in the analysis mask is asserted and, in assisted mode, the reduction flag in the reduction mask is asserted, the corresponding segmentation flag is asserted if the corresponding pixel value in the fluorescence image is higher (possibly strictly) than the segmentation threshold, and deasserted otherwise. Meanwhile, for each other pixel, the corresponding segmentation flag is always deasserted. The segmenter then stores the resulting segmentation mask in the corresponding repository.
[0092] The operational flow then branches in block 558 according to the evaluation mode of the matching indicator of the analysis region segmentation (e.g., manually selected at the beginning of the imaging procedure, defined by default or the only available one). In particular, in dual (evaluation) mode, blocks 560 to 564 are executed, while in single (evaluation) mode, block 566 is executed. In both cases, the operational flow merges in block 568.
[0093] Referring now to block 560 (dual mode), a quality indicator is determined according to a comparison between detected and non-detected segments. In particular, the segmenter calculates a (detection) statistical parameter for the detected segments, for example equal to the average of its pixel values (TI). For this purpose, the verifier considers pixel values of the fluorescence image for which the detection flag is asserted in the segmentation mask (retrieved from the corresponding repository).
[0094] Similarly, the segmenter calculates (non-detection) statistical parameters for non-detection segments in block 562, which are again equal to the average of their pixel values (BI). For this purpose, the verifier considers (retrieved from the corresponding repository) pixel values of the fluorescence image for which the detection flag in the segmentation mask is deasserted, and at the same time the analysis flag in the analysis mask is asserted, and the reduction flag in the reduction mask is asserted for the assisted mode. The verifier calculates a quality indicator in block 564 according to a comparison between the detection statistical parameters and the non-detection statistical parameters (e.g., their ratio (TBR)).
[0095] The process then continues to block 568. Referring instead to block 566 (single mode), a quality indicator is determined according to the detected segment alone. In particular, the segmenter sets the quality indicator equal to the continuity indicator of the detected segment, e.g., a maximum value reduced by an amount proportional to the number of cuts of the detected segment from its main portion (possibly weighted according to their size). The process then continues to block 568.
[0096] Referring now to block 568, the flow of operations branches according to the verification mode of the imaging system (e.g., manually selected at the beginning of the imaging procedure, defined by default or the only available one). In particular, in dynamic (verification) mode, all fluorescence images are verified individually. This is well suited for situations where the field of view is continuously changing (e.g., surgical applications), even if that application is not precluded in any other situation. In this case, the verifier sets the verification indicator of the fluorescence image to its quality indicator in block 570.
[0097] Conversely, in static (matching) mode, multiple fluorescence images are considered for matching. This is well suited to situations where the field of view remains substantially the same over time (e.g., in diagnostic / therapeutic applications), even if the application is not precluded in any other situation (e.g., by considering multiple fluorescence images). In this case, the verifier, in block 572, sets the matching indicator for the fluorescence image to the average of the quality indicators of a set of fluorescence images including the current one and one or more previous ones (retrieved from the corresponding repository), e.g., a predetermined number, e.g., 5-10.
[0098] This improves the reliability of the match due to the corresponding smoothing of transient changes in the field of view. In both cases, the verifier can compare the match indicator to a match threshold (searched for above for the target) in block 574. The verifier then stores the match results thus obtained in a corresponding repository. In particular, the match results include a match indicator and / or a match flag that is asserted if the match indicator is higher (possibly strictly) than the match threshold (positive confirmation) and deasserted otherwise (negative confirmation), along with additional information, if possible, including statistical detection parameters, statistical non-detection parameters, and the match threshold.
[0099] The operation flow then branches according to the highlighting mode of the detected segments (e.g., manually selected at the beginning of the imaging procedure, defined by default or the only available one) in block 576. In particular, in the selection (highlighting) mode, the operation flow further branches according to the match result (retrieved from the corresponding repository) in block 578. If the match result is positive (the match flag is asserted), the visualization device checks whether the selection mode is standalone in block 580.
[0100] If so, the visualization device updates the fluorescence image by highlighting the pixels of the detection segment as shown in the segmentation mask (retrieved from the corresponding repository) in block 582. For example, the pixels of the detection segment are colored (e.g., red) and the intensity increases with the corresponding pixel values across the entire display range of the monitor (the remainder of the fluorescence image is black and white).
[0101] Referring again to block 576, in progressive (highlighting) mode, the process descends to block 584. The same point is reached from block 580 if the selection mode is combined with the progressive mode. In both cases, the visualization device updates the fluorescence image by highlighting the pixels of the detection segment, for example, as indicated in the segmentation mask (retrieved from the corresponding repository), with the highlighting intensity depending, for example, on the matching indicator. The pixels of the detection segment are colored, with the intensity increasing with the corresponding pixel value (across the entire display range of the monitor), and the wavelength increasing with the matching indicator (e.g., from blue to red if the progressive mode is standalone, or from yellow to red if the progressive mode is combined with the selection mode), while the rest of the fluorescence image is black and white, or vice versa.
[0102] The process then continues to block 582 or from block 582 to block 586. If the match result is negative (the match flag is deasserted), the same point is reached directly from block 578 in selection mode. In this case, the fluorescence image remains unchanged, for example, completely black and white (including its detected segment). The visualization device now displays the fluorescence image on the monitor together with a representation of the analysis area and, if possible, a highlighted detected segment (and corresponding possible additional information of the match). The detected segment and / or its higher intensity coloring thus provides the surgeon with an immediate indication of its likelihood of actually representing the tumor to be resected (thanks to utilization of the full display range of the monitor).
[0103] The setter checks in block 588 whether the operator has entered a change command into the imaging system (e.g., using its keyboard) to change the analysis region, for example, if the surgeon wishes to move to a different part of the surgical cavity. If so, the process returns to block 512 to set the (new) analysis region as described above. Conversely, the segmenter checks in block 590 whether the operator has entered an end command into the imaging system (e.g., using its keyboard) to end the imaging procedure. If not, the process returns to block 554 to repeat the same operations on the next fluorescence image. Conversely, the process ends at concentric black / white stop circles 592 (after the fluorescence manager and reflectance manager turn off the excitation light source and white light source, respectively).
[0104] (Variation) Naturally, those skilled in the art may apply many logical and / or physical modifications and alterations to the present disclosure to meet local and specific requirements. More particularly, while the present disclosure has been described in some detail with reference to one or more embodiments thereof, it should be understood that various omissions, substitutions, and changes in form, details, and other embodiments are possible. In particular, various embodiments of the present disclosure may be practiced without the specific details (e.g., numerical values) set forth in the foregoing description to provide a more thorough understanding, or conversely, well-known features may be omitted or simplified so as not to obscure the description in unnecessary detail. Furthermore, it is expressly intended that specific elements and / or method steps described in connection with any embodiment of the present disclosure may be incorporated in any other embodiment as a matter of general design choice. Furthermore, items presented in the same group and various embodiments, examples, or alternatives should not be construed as being de facto equivalents to each other (but are separate and autonomous entities). In all cases, each numerical value should be read as modified according to the applicable tolerance, and unless otherwise indicated, the terms "substantially," "about," and "approximately" should be understood to mean within a 10%, preferably within a 5%, and even more preferably within a 1% range. Moreover, each range of numerical values should be intended as explicitly specifying any possible number along a continuum within that range (including its endpoints). Ordinal numbers or other qualifiers are merely used as labels to distinguish between elements with the same name and do not, in themselves, imply a priority, preference, or sequence.The terms "include," "comprise," "have," "contain," "involve," etc. are intended to have an open, non-exhaustive meaning (i.e., not limited to the listed items), the terms "based on," "dependent on," "according to," "function of," etc. are intended to have a non-exhaustive relationship (i.e., possibly including additional variables), the term "a / an" is intended to mean one or more items (unless expressly stated otherwise), and the term "means for" (or any means-plus-function form) is intended to mean any structure adapted or configured to perform the associated function.
[0105] For example, one embodiment provides a method for imaging a field of view, however, the method can be used to image any field of view for any purpose (e.g., medical applications, forensic analysis, defect / crack inspection, etc.).
[0106] In one embodiment, the field of view includes a target, however the target may be of any type (e.g., a body part, a fingerprint, a mechanical part, etc.).
[0107] In one embodiment, the target comprises a luminescent material, however the luminescent material may be of any extrinsic / intrinsic type (e.g., any luminescent agent based on any luminescence phenomenon, e.g., fluorescence, phosphorescence, chemiluminescence, bioluminescence, stimulated Raman emission, etc., any naturally occurring luminescent component, etc.).
[0108] In one embodiment, the method comprises the following steps under the control of a computer device: However, the computer device may be of any type (see below).
[0109] In one embodiment, the method includes providing (to a computing device) a luminescent image of the field of view, although the luminescent image may be provided in any manner (e.g., captured directly, transferred on a removable storage unit, uploaded over a network, etc.).
[0110] In one embodiment, the luminescence image includes a plurality of luminescence values representing luminescence light emitted by the luminescent material from corresponding luminescence locations in the field of view. However, the luminescence image may have any size and shape, and may include any type of luminescence values and any luminescence locations (e.g., grayscale or color values in RBG, YcBcr, HSL, CIE-L*a*b, Lab color, or similar representations for pixels, voxels). The luminescence light may be of any type (e.g., NIR, infrared (IR), visible light, etc.) and may be emitted in any manner (e.g., in response to corresponding excitation light, or more generally, any other excitation other than heating).
[0111] In one embodiment, the method includes setting (by a computer device) an analysis region in a portion of the luminescent image surrounding the suspect representation of the target. However, the analysis region may be set in any manner (e.g., the entire luminescent image or only within its information region, predefined, or determined manually, semi-automatically, or fully automatically on-the-fly).
[0112] In one embodiment, the method includes dividing (by a computer device) the analysis region into a detection segment representing the detection of the luminescent agent and a non-detection segment representing the non-detection of the luminescent agent according to the luminescence value of only the analysis region. However, the analysis region may be divided by any method (e.g., by a corresponding mask or defined directly in the fluorescence image, a thresholding algorithm, a classification algorithm, a machine learning technique, etc.).
[0113] In one embodiment, the method includes determining (by a computer device) a matching indicator based on the quality indicators dividing the analysis region. However, the matching indicator may be determined in any manner (e.g., based only on the quality indicator of the subject luminescent image, and also based on the quality indicators of one or more other luminescent images), according to any quality indicator (e.g., based on a comparison of detected and non-detected segments, based only on detected segments, etc.).
[0114] In one embodiment, the method includes displaying (by a computing device) the fluorescence image with the detection segments highlighted according to the matching indicator. However, the fluorescence image may be displayed in any manner (e.g., on any display unit (e.g., monitor, virtual reality glasses, etc.), printed, locally or remotely, in real time or offline, alone, combined / overlaid with the corresponding reflectance image, etc.). The detection segments may be highlighted in any manner (e.g., colored relative to the rest of the image being black and white, or with high intensity relative to the rest of the image being low intensity) and according to the matching indicator (e.g., selectively highlighted or not highlighted, with gradual highlighting intensity depending on the matching indicator, combinations thereof, with or without any further information).
[0115] Further embodiments provide additional advantageous features that may be omitted altogether in the basic implementation.
[0116] In particular, in one embodiment, the method includes displaying (by a computing device) the luminescent image with the detection segments selectively highlighted according to the matching indicator, although the detection segments may be selectively highlighted in any manner (e.g., simply according to a comparison of the matching indicator to a threshold, with hysteresis, etc.).
[0117] In one embodiment, the method includes displaying (by a computing device) the luminescent image with the detected segments highlighted or unhighlighted according to a comparison of the match indicator to a match threshold, although the match threshold may have any value (e.g., varying with the target, fixed, etc.).
[0118] In one embodiment, the method includes displaying (by a computer device) the luminescence image in black and white, with the detection segments highlighted or not highlighted by being displayed in color or black and white, respectively. However, the detection segments may be displayed in any color (e.g., a single color whose intensity varies with its luminescence value, a range of colors that depend on its luminescence value, a single color at a constant brightness, etc.).
[0119] In one embodiment, the method includes displaying (by a computing device) the luminescent image with the detection segment progressively highlighted with a highlighting intensity according to the matching indicator. However, the highlighting intensity may be of any type (e.g., different colors, brightness, etc.) and it may depend on the matching indicator in any way (e.g., any linear or non-linear function, e.g., proportional, exponential, logarithmic, continuous or discrete, always, or when the detection segment needs to be highlighted as described above, etc.).
[0120] In one embodiment, the method includes displaying (by a computing device) the luminescent image with the detected segment highlighted with a highlighting intensity proportional to the matching indicator, although this result may be achieved in any manner (e.g., with a highlighting intensity proportional to the entire matching indicator, or proportional to only the portion above the matching threshold, etc.).
[0121] In one embodiment, the method includes displaying (by a computer device) the luminescence image in black and white, with the detected segment highlighted by being displayed in color along with a visual cue that is dependent on the matching indicator, although this result may be achieved in any manner (e.g., alone or in combination with other visual cues that are dependent on the luminescence value, e.g., single color or varying colors that are dependent on the luminescence value, brightness that is dependent on the matching indicator, fixed brightness or brightness that is dependent on the luminescence value, varying colors that are dependent on the matching indicator, etc.).
[0122] In one embodiment, the method includes determining (by a computer device) a quality indicator according to a comparison of the content of the detected segment with the content of the non-detected segment, although this result may be achieved in any manner (e.g., by calculating the quality indicator as a similarity index, interclass variance, by prior or direct aggregation of the luminescence values of the two segments).
[0123] In one embodiment, the method includes calculating (by a computer device) a detection value according to the luminescence values of the detection segment and a non-detection value according to the luminescence values of the non-detection segment. However, the detection / non-detection values may be of any type (e.g., corresponding statistical parameters such as mean, median, mode, variance, etc., maximum / minimum values, etc.).
[0124] In one embodiment, the method includes calculating (by a computer device) a quality indicator according to a comparison of the detected value and the non-detected value. However, the quality indicator may be calculated in any manner according to the detected value and the non-detected value (e.g., a ratio, a difference, etc.)
[0125] In one embodiment, the method includes determining (by a computer device) the quality indicator according to the content of the detection segment only. However, the quality indicator may be determined according to the content of the detection segment in any way (e.g., by setting the quality indicator to any statistical parameter of its luminescence values, such as its mean, median, mode, variance, etc.; by setting the quality indicator according to one or more qualitative properties of the detection segment, such as its continuity, shape, regular / irregular boundaries, etc.).
[0126] In one embodiment, the method includes calculating (by a computer device) a quality indicator according to the continuity of the detection segment, although the continuity may be defined in any manner (e.g., according to the number of disconnected portions of the detection segment, by weighting the disconnected portions according to their size and / or distance, etc.).
[0127] In one embodiment, the method includes initializing (by a computing device) an analysis region to a portion of a context space equal to the luminescent image. However, the context space may be set exactly to the luminescent image, or more generally, may correspond to it (e.g., equal to a predefined interior portion thereof), and the analysis region may be initialized accordingly in any way (e.g., at a predefined or run-time determined location, with or without the possibility of updating it later, with or without requiring manual confirmation, etc.).
[0128] In one embodiment, the field of view includes a region of interest for imaging and one or more foreign objects distinct from the region of interest. However, the region of interest may be of any type (e.g., a surgical cavity, an internal cavity of an endoscopic procedure, either an open type accessed through a hollow, or a closed type accessed through an incision, etc.). The foreign objects may be of any number and type (e.g., instruments, hands, tools, body parts, background material, etc.), and may be located in any position (e.g., overlapping, surrounding, spaced apart from the region of interest to any extent, any combination thereof, etc.).
[0129] In one embodiment, the method includes providing (by a computing device) an auxiliary image of the field of view, however the auxiliary image may be provided in any manner (either the same or different from the luminescent image).
[0130] In one embodiment, the auxiliary image includes a plurality of auxiliary values representing auxiliary light (different from the emissive light) received from corresponding auxiliary locations in the field of view. However, the auxiliary image may have any size and shape and may include any type of auxiliary value and any auxiliary location (the same or different from the emissive image). The auxiliary light may be of any type different from the emissive light of the emissive image (e.g., visible light, IR light, ultraviolet (UV) light, other emissive light of a different wavelength, etc.).
[0131] In one embodiment, the method includes identifying (by a computer device) auxiliary information regions of the auxiliary image that represent foreign object-free regions of interest according to the content of the auxiliary image. However, the auxiliary information regions may be of any type (e.g., a single region, one or more disjoint regions, a corresponding mask or one defined directly in the auxiliary image, etc.), and they may be identified in any manner (e.g., by semantically / non-semantically dividing the auxiliary image into auxiliary information regions and auxiliary non-information regions, by searching for auxiliary information regions in the auxiliary image, etc.).
[0132] In one embodiment, the method includes identifying (by a computer device) luminous information regions of the luminous image that correspond to auxiliary information regions. However, the luminous information regions may be identified in any manner (e.g., by transferring the identification of the auxiliary information regions directly or with any adaptation, with any post-processing, etc.). Furthermore, this operation may be performed indiscriminately or may be conditioned on the quality of the identification of the auxiliary information regions. For example, a quality metric may be calculated, and all luminous locations may be assigned to luminous information regions if the quality metric does not reach a corresponding threshold.
[0133] In one embodiment, the method includes initializing (by a computer device) an analysis region to a portion of a context space defined according to the luminescence information region. However, the context space may be defined in any manner according to the luminescence information region (e.g., equal to the largest rectangular region of the fluorescent image enclosed therein, equal to the smallest rectangular region of the fluorescent image enclosed therein, etc.). The analysis region may be initialized accordingly in any manner (either the same or different from the context space corresponding to the luminescence image).
[0134] In one embodiment, the auxiliary image is a reflected image, the auxiliary light is visible light, and the auxiliary values represent visible light reflected at corresponding auxiliary locations in a field of view illuminated by white light, although the white light (and corresponding visible light) may be of any type (e.g., non-luminescent light that does not produce significant luminescence in luminescent materials).
[0135] In one embodiment, identifying the auxiliary information regions includes semantically segmenting (by a computer device) the auxiliary image, although the auxiliary image may be semantically segmented in any manner (e.g., using any type of classification algorithm based on any number and type of features, using any neural network-based deep learning technique, according to the auxiliary image alone or additionally according to the fluorescence image, etc.).
[0136] In one embodiment, the auxiliary image is semantically divided into auxiliary information regions corresponding to at least one region of interest class of the region of interest and auxiliary non-information regions corresponding to one or more foreign object classes of the foreign objects, however, the region of interest classes and foreign object classes may be of any number and type (e.g., a single region of interest class for the entire region of interest, a single foreign object class for all foreign objects, a single foreign object class for corresponding portions or groups of the region of interest, multiple foreign object classes for corresponding foreign object types or groups, etc.).
[0137] In one embodiment, the method includes initializing (by a computer device) the analysis domain to a portion of the context space having a predefined initial shape, although the initial shape may be of any type (e.g., square, circle, rectangle, etc.), and in any case does not exclude the possibility of manually selecting the initial shape at runtime.
[0138] In one embodiment, the method includes initializing (by a computer device) the analysis region to a portion of the context space having a predefined initial size. However, the initial size may be of any type (e.g., a predetermined percentage of the context space, a predetermined value, etc.), and in any case does not exclude the possibility of manually selecting the initial size at runtime (either in an absolute or relative sense).
[0139] In one embodiment, the method includes initializing (by a computer device) the analysis domain to a portion of the context space with a predetermined initial position, although the initial position may be of any type (e.g., center, edge, etc.), and in any case does not exclude the possibility of manually selecting the initial position at run time.
[0140] In one embodiment, the method includes setting (by a computer device) an initial size of the analysis region according to at least one predetermined percentage of the size of the context space, although this percentage may be of any type and value (e.g., a single percentage for the region, two or more percentages corresponding to dimensions, e.g., width and height, etc.).
[0141] In one embodiment, the method includes setting (by a computer device) an initial location of the analysis domain at the center of the context space. However, the analysis domain may be centered in the context space in any manner (e.g., at its geometric center, its center of gravity, etc.).
[0142] In one embodiment, the method includes initializing (by a computer device) the analysis region to a best candidate region of the context space selected from among multiple candidate regions that are candidates for initializing the analysis region according to their corresponding content. However, the candidate regions may be of any number and type (e.g., predetermined shapes / sizes that move at any pitch throughout the context space, shapes and / or sizes that vary throughout the context space, etc.). The best candidate region may be selected in any manner (e.g., according to corresponding quality indicators, strength indicators, combinations thereof, etc.).
[0143] In one embodiment, the method includes dividing (by a computer device) each of the candidate regions into candidate detection segments and candidate non-detection segments representing detection of a luminescent agent and non-detection of a luminescent agent according to the luminescence value of the candidate region only, although each candidate region may be divided in any manner (either the same or different from the analysis region).
[0144] In one embodiment, the method includes calculating (by a computer device) corresponding candidate quality indicators for said division of the candidate region, although each candidate quality indicator may be determined in any manner (either the same or different for the quality indicator).
[0145] In one embodiment, the method includes selecting (by a computer device) the best candidate region having the best of the candidate quality indicators, although the best candidate quality indicator may be determined in any manner (e.g., the highest or lowest, if increasing or decreasing with quality, etc.).
[0146] In one embodiment, the method includes calculating (by a computing device) corresponding intensity indicators of the candidate regions, each intensity indicator indicating the intensity of luminescence light emitted from the location of the corresponding candidate region only. However, each intensity indicator may be calculated in any manner (e.g., according to any statistical parameter of the corresponding luminescence values, such as their mean, median, mode, variance, etc.).
[0147] In one embodiment, the method includes selecting (by a computer device) the best candidate region having the best one of the intensity indicators, although the best intensity indicator may be determined in any manner (e.g., the highest or lowest, if it increases or decreases with the concentration of the luminescent substance, etc.).
[0148] In one embodiment, the method includes displaying (by a computing device) the luminescent image together with a representation of the analysis region. However, the luminescent image may be displayed together with any representation of the analysis region (e.g., any color, any line, its outline, etc.).
[0149] In one embodiment, the method includes receiving (by a computing device) a manual adjustment of the analysis region. However, the analysis region may be adjusted in any manner (e.g., by changing its shape, size, and / or position using any input unit, e.g., a keyboard, trackball, mouse, keypad, etc.).
[0150] In one embodiment, the method includes receiving (by a computing device) confirmation of the analysis region after manual movement of the contents of the field of view. However, confirmation may also be provided by any input unit (e.g., keyboard, trackball, mouse, dedicated button, etc.) after any movement of the field of view and / or imaging probe.
[0151] In one embodiment, the method includes providing (by a computing device) one or more additional luminescence images of the field of view, each additional luminescence image including a corresponding additional luminescence value representing luminescence light emitted by the luminescent material from a location in the field of view. However, the additional luminescence images may be any number (e.g., fixed or up to all available ones before and / or after the luminescence image), and they may be of any type and provided in any way (either the same or different from the luminescence image).
[0152] In one embodiment, the method includes dividing (by a computer device) a corresponding further analysis region of the further luminescence image corresponding to the analysis region into a further detection segment and a further non-detection segment representing detection of a luminescent agent and non-detection of a luminescent agent, respectively, according to a further luminescence value of only the further analysis region. However, the further analysis region may be divided in any manner (either the same or different from the analysis region).
[0153] In one embodiment, the method includes determining (by a computer device) a corresponding further quality indicator for dividing the further analysis region, although the further quality indicator may be determined in any manner (either the same or different from the quality indicator).
[0154] In one embodiment, the method includes determining (by a computer device) the matching indicator further based on the further quality indicator. However, the further quality indicator may be used in any way to determine the matching indicator (e.g., by calculating a global quality indicator (equal to any statistical parameter of the quality indicators and the further quality indicators, e.g., their mean, median, mode, etc.) and using it to determine corresponding partial matching indicators according to such quality indicators and the further quality indicators, and then using these to determine the matching indicator, e.g., considering the matching result positive if at least a predetermined percentage of these, up to all, etc.).
[0155] In one embodiment, the method includes determining (by a computer device) a segmentation threshold according to a statistical distribution of luminescence values within the analysis region, although the segmentation threshold may be determined in any manner (e.g., based on statistical distribution, entropy, clustering, or object attributes, etc.).
[0156] In one embodiment, the method includes dividing (by a computer device) the analysis region into detection and non-detection segments according to a comparison of the luminescence value of the analysis region with a segmentation threshold. However, the locations may be assigned to detection / non-detection segments in any manner according to the segmentation threshold (e.g., to detection segments if they are (possibly strictly) higher or lower than it).
[0157] In one embodiment, the method is for imaging a body part of a patient in a medical application. However, the medical application may be of any type (e.g., surgical, diagnostic, therapeutic, etc.). In any case, the method can facilitate the doctor's task, not only by providing intermediate results that can assist them, but also strictly speaking, always involving a medical procedure performed by the doctor himself. Furthermore, the body part may be of any type (e.g., an organ, e.g., a region of the liver, prostate, or heart, tissue, etc.), in any state (e.g., in a living being, in a cadaver, extracted from the body, e.g., a biopsy sample, etc.), and of any patient (e.g., human, animal, etc.).
[0158] In one embodiment, the target is defined by a target state of the body part, however the target state may be of any type (e.g., pathological tissue, e.g., tumor, inflammation, etc., healthy tissue, etc.).
[0159] In one embodiment, the luminescent agent is pre-administered to the patient before the method is performed. However, the luminescent agent may be of any type (e.g., any targeted luminescent agent based on specific or non-specific interactions, non-targeted luminescent agent, etc.), and it may be pre-administered by any method (e.g., using a syringe, an infusion pump, etc.) and at any time (e.g., before, just before, continuously during, etc.) before the method is performed. In either case, this is a data processing method that can be performed independently of interaction with the patient. Furthermore, the luminescent agent may be administered to the patient non-invasively (e.g., orally to image the gastrointestinal tract, via a nebulizer into the respiratory tract, via topical spray application or local introduction during a surgical procedure), or in any case without substantial physical intervention (e.g., intramuscular injection) for the patient, which requires specialized medical expertise or involves risks to his / her health.
[0160] In one embodiment, the field of view includes a surgical cavity that exposes a body part during a surgical procedure, however, the surgical cavity may be of any type for any surgical procedure (e.g., minimally invasive surgery, open body in standard surgery, etc.).
[0161] In one embodiment, the luminescent material is a fluorescent material (the luminescence image is a fluorescence image, and the luminescence values represent the fluorescent light emitted by the fluorescent material from the corresponding luminescence locations illuminated by its excitation light), however, the fluorescent material may be of any type (e.g., exogenous / endogenous, etc.).
[0162] In general, similar considerations apply when the same technique is implemented in an equivalent manner (by using more steps or similar steps with the same functionality of some of them, by eliminating some non-essential steps, or by adding further optional steps). Furthermore, steps may be performed (at least partially) in different orders, simultaneously, or in an interleaved manner.
[0163] One embodiment provides a computer program configured to cause a computing device to perform the above-described method when the computer program is executed on the computing device. One embodiment provides a computer program product, including a computer-readable storage medium embodying the computer program, which is loadable into the working memory of the computing device, thereby configuring the computing device to perform the same method. However, the computer program may also be implemented as a stand-alone module, as a plug-in to an existing software program (e.g., an imaging system manager), or directly in the latter. In either case, similar considerations apply if the computer program is configured differently or if additional modules or functions are provided. Similarly, memory structures may be of other types or may be replaced by equivalent entities (not necessarily consisting of physical storage media). The computer program may take any form suitable for use by any computing device (see below) and thereby configure the computing device to perform the desired operations. In particular, the computer program may be in the form of external or resident software, firmware, or microcode (either object code or source code, e.g., compiled or interpreted). Furthermore, the computer program may be provided on any computer-readable storage medium. A storage medium is any tangible medium (as distinct from a transitory signal itself) that can hold and store instructions for use by a computing device. For example, the storage medium may be of an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor type. Examples of such storage media are fixed disks (which may be preloaded with a program), removable disks, memory keys (e.g., USB type), etc.The computer program may be downloaded to the computing device from a storage medium or via a network (e.g., the Internet, a wide area network, and / or a local area network, including transmission cables, optical fibers, wireless connections, and network devices). One or more network adapters in the computing device receive the computer program from the network and transfer it to one or more storage devices of the computing device for storage. In either case, an approach according to an embodiment of the present disclosure lends itself to implementation in a hardware structure (e.g., by electronic circuits integrated in one or more chips of semiconductor material, such as a field programmable gate array (FPGA) or application specific integrated circuit (ASIC)), or in a combination of appropriately programmed or otherwise configured software and hardware.
[0164] An embodiment provides a system, comprising means configured to perform the steps of the above-described method. An embodiment provides a system including circuitry (i.e., any suitably configured hardware, e.g., by software) for performing each step of the same method. However, the system may be of any type (e.g., any computing device, e.g., a central unit of an imaging system, a separate computer, etc.).
[0165] In one embodiment, the system is an imaging system, however, the imaging system may be of any type (e.g., guided surgical instrument, endoscope, laparoscope, etc.).
[0166] In one embodiment, the imaging system includes an illumination unit for applying excitation light and possibly white light to the field of view, although the illumination unit may be of any type (e.g., laser, LED, UV / halogen / xenon lamp, etc.).
[0167] In one embodiment, the imaging system comprises an acquisition unit for acquiring the fluorescence image and possibly auxiliary images, however the acquisition unit may be of any type (e.g., any number and type of lenses, waveguides, mirrors, CCD, ICCD, EMCCD, CMOS, InGaAs, PMT sensors, etc.).
[0168] In general, similar considerations apply when a system has different structures or comprises equivalent components, or when it has other operational characteristics. In either case, all of its components may be separated into more elements, or two or more components may be integrated into a single element. Furthermore, each component may be replicated to support the execution of corresponding operations in parallel. Furthermore, unless otherwise specified, any interaction between different components generally need not be sequential, but may be direct or indirect via one or more intermediaries.
[0169] One embodiment provides a surgical method comprising the following steps: according to said method, a body part of a patient is imaged, thereby displaying a luminescent image, and during a surgical procedure on the body part, detected segments are highlighted according to a matching indicator; the body part is operated on according to said display of the luminescent image; however, the proposed method may find application in any kind of surgical method in the broadest sense of the term (e.g. for therapeutic purposes, for preventive purposes, for aesthetic purposes, etc.) and for operating on any kind of body part of any patient (see above).
[0170] One embodiment provides a diagnostic method comprising the following steps: according to said method, a body part of a patient is imaged, thereby displaying a luminescence image, and during a diagnostic procedure on the body part, detected segments are highlighted according to a matching indicator; the body part is analyzed according to said display of the luminescence image; however, the proposed method may find use in any kind of diagnostic application in the broadest sense of the term (e.g., helping to assess health status, discover new lesions, monitor known lesions, etc.), and for analyzing any kind of body part of any patient (see above).
[0171] One embodiment provides a therapeutic method comprising the following steps: according to said method, a body part of a patient is imaged, thereby displaying a luminescent image, and during a therapeutic treatment of the body part, detected segments are highlighted according to a matching indicator; the body part is treated according to said display of the luminescent image. However, the proposed method may find application in any kind of therapeutic method in the broadest sense of the term (e.g., treating a pathological condition, avoiding its progression, preventing the occurrence of a pathological condition, or simply helping to improve the patient's comfort), and for treating any kind of body part of any patient (see above).
[0172] In one embodiment, the surgical, diagnostic and / or therapeutic methods each include administering a luminescent agent to the patient, although the luminescent agent may be administered in any manner (see above), or this step may be omitted altogether (if the luminescent agent is endogenous).
Claims
1. A method (500) for imaging a field of view (103) with a target (115) comprising a luminescent material, comprising: providing (508) to the computer device (100) a luminescence image (205; 205F) of the field of view (103), the luminescence image (205; 205F) including a plurality of luminescence values representing luminescence light emitted by a luminescent material from corresponding locations of the field of view (103); setting (512-552) an analysis region (225; 260) in a portion of the luminescent image (205; 205F) surrounding a suspect representation of the target (115) by the computer device (100); Dividing (554-556) the analysis area (225; 260) according to the luminescence values of only the analysis area (225; 260) by the computer device (100) into detection segments (230d; 265d) and non-detection segments (230n; 265n) representing the detection of a luminescent agent and the non-detection of a luminescent agent, respectively; - steps (558-574) of determining, by the computer device (100), a matching indicator based on the quality indicator of the division of the analysis area (225; 260), said quality indicator being determined according to a comparison (560-564) of the content of the detected segment (230d; 265d) with the content of the non-detected segment (230n; 265n) or according to the content of the detected segment (230d; 265d) only (566); and displaying (576-586) by the computer device (100) the luminescent image (205; 205F) with the detected segment (230d; 265d) highlighted according to the matching indicator.
2. 10. The method (500) of claim 1, comprising the step of displaying (578-582, 586) by a computer device (100) the luminescent image (205; 205F) with the detected segments (230d; 265d) selectively highlighted in accordance with the matching indicator.
3. The method (500) of claim 2, comprising a step (578-582, 586) of displaying, by a computer device (100), the luminescent image (205; 205F) together with the detected segments (230d; 265d) that are highlighted or not highlighted according to a comparison of the matching indicator with a matching threshold.
4. The method (500) according to claim 2 or 3, comprising a step (578-582, 586) of displaying the luminescent image (205; 205F) in black and white by the computer device (100), together with the detected segments (230d; 265d) that are highlighted or not highlighted by displaying in color or black and white.
5. A method (500) according to any one of claims 1 to 4, comprising a step (584-586) of displaying by a computer device (100) the luminescent image (205; 205F) with the detected segments (230d; 265d) progressively highlighted with a highlighting intensity that depends on the matching indicator.
6. 6. The method (500) of claim 5, comprising the step of displaying (584-586) by a computer device (100) the luminescent image (205; 205F) with the detected segment (230d; 265d) highlighted with a highlighting intensity proportional to the matching indicator.
7. The method (500) according to claim 5 or 6, comprising steps (584-586) of displaying by the computer device (100) the luminescent image (205; 205F) in black and white, with the detected segments (230d; 265d) highlighted by being displayed in color together with a visual stimulus dependent on the matching indicator.
8. Calculating (560-562) by the computer device (100) detection values according to the luminescence values of the detection segments (230d; 265d) and non-detection values according to the luminescence values of the non-detection segments (230n; 265n); A method (500) according to any of the preceding claims, comprising a step (564) of calculating, by the computer device (100), a quality indicator according to a comparison between the detected value and the non-detected value.
9. The method (500) according to any of the preceding claims, comprising a step (566) of calculating, by the computer device (100), a quality indicator according to the continuity of the detected segments (230d; 265d).
10. A method (500) according to any of the preceding claims, comprising a step (514) of initializing, by the computer device (100), the analysis domain (225; 260) to a portion of the context space equal to the luminescence image (205; 205F).
11. The field of view (103) includes a region of interest (112) for imaging and one or more foreign objects (210-220; 235-245) different from the region of interest (1123); providing (508) to the computer device (100) an auxiliary image (205R) of the field of view (103), the auxiliary image (205R) including a plurality of auxiliary values representing auxiliary light distinct from the emitted light, the auxiliary values being received from corresponding auxiliary positions in the field of view; Identifying (516) by the computer device (100) auxiliary information regions (250Ri) of the auxiliary image (205R) representing the region of interest (112) free of foreign objects (210-220; 235-245) according to the content of the auxiliary image (205R); Identifying (518) by the computer device (100) luminescent information areas (250Fi) of the luminescent image (205F) corresponding to the auxiliary information areas (250Ri); A method (500) according to any one of claims 1 to 9, comprising a step (520) of initializing, by the computer device (100), the analysis region (225; 260) to a part of the context space defined according to the light emitting information region (250Fi).
12. The auxiliary image is a reflected image (205R), The fill light is visible light, 12. The method (500) of claim 11, wherein the auxiliary values represent visible light reflected by corresponding auxiliary locations of the field of view (103) illuminated by white light.
13. 13. The method (500) according to claim 11 or 12, wherein the step of identifying (516) auxiliary information regions comprises a step (516) of semantically dividing (by the computer device (100) the auxiliary image (205R) into auxiliary information regions (250Ri) corresponding to at least one region of interest class of the regions of interest (112) and auxiliary non-information regions (250Rn) corresponding to one or more foreign object classes of the foreign objects (210-220; 235-245).
14. A method (500) according to any one of claims 9 to 13, comprising a step (522-526) of initializing, by a computer device (100), an analysis domain (225; 260) to a portion of a context space (205; 250Fi) having a predefined initial shape, initial size and / or initial position.
15. 15. The method (500) of claim 14, comprising a step (522) of setting, by the computer device (100), an initial size of the analysis domain (225; 260) according to at least one predefined percentage of the size of the context space (205; 250Fi).
16. 16. The method (500) of claim 14 or 15, comprising a step (526) of setting, by the computer device (100), an initial position of the analysis domain (225; 260) at the center of the context space (205; 250Fi).
17. A method (500) according to any of claims 10 to 13, comprising a step (528-544) of initializing, by the computer device (100), the analysis region (225; 260) to a best candidate region selected from among a plurality of candidate regions of the context space (205; 250Fi), which is a candidate for initializing the analysis region (225; 260) according to its corresponding content.
18. Dividing (532) each of the candidate regions into candidate detection segments and candidate non-detection segments representing detection of a luminescent agent and non-detection of a luminescent agent, respectively, according to the luminescence values of the candidate region only, by the computer device (100); calculating (532) by the computer device (100) corresponding candidate quality indicators of said step (532) of dividing the analysis domain; and selecting (538-540) by the computer device (100) the best candidate region having the best of the candidate quality indicators.
19. calculating (536), by the computer device (100), corresponding intensity indicators for the candidate regions, each of the intensity indicators indicating an intensity of luminescent light emitted from the location of the corresponding candidate region only; and selecting (538-540) by the computer device (100) the best candidate region having the best one of the intensity indicators.
20. displaying (546) the luminescence image (205; 205f) together with a representation of the analysis region (225; 260) by the computer device (100); A method (500) according to any of the preceding claims, comprising receiving (548) by the computer device (100) a manual adjustment of the analysis region (225; 260).
21. A method (500) according to any of the preceding claims, comprising receiving (550) by the computer device (100) a confirmation of the analysis region (225; 260) after manual movement of the contents of the field of view (103).
22. providing (508), by the computer device (100), one or more further luminescence images (205; 205F) of the field of view (103), each further luminescence image (205; 205F) including a corresponding further luminescence value representing luminescence light emitted by a luminescent material from a location in the field of view (103); - dividing (554-556) by the computer device (100) the corresponding further analysis region (225; 260) of the further luminescence image (205, 205p) corresponding to said analysis region (225; 260) into further detection segments (230d; 265d) and further non-detection segments (230n; 265n) representing the detection of a luminescent agent and the non-detection of a luminescent agent, respectively, according to further luminescence values of only said further analysis region (225; 260); determining (558-564) by the computer device (100) corresponding further quality indicators of said steps (554-556) of dividing further analysis regions; A method (500) according to any of the preceding claims, comprising the step of determining (572) by the computer device (100) a matching indicator based on the further quality indicator.
23. determining (554) by the computer device (100) a segmentation threshold according to the statistical distribution of the luminescence values in the analysis region (225; 260); A method (500) according to any of claims 1 to 22, comprising a step (556) of dividing the analysis region (225; 260) into detection segments (230d; 265d) and non-detection segments (230n; 265n) according to a comparison of the luminescence values of the analysis region (225; 260) with a segmentation threshold, by the computer device (100).
24. The method (500) is for imaging a body part (109) of a patient (106) in a medical application, The method (500) according to any of the preceding claims, wherein the target is defined by a target state (115) of the body part (109).
25. 25. The method (500) of claim 24, wherein the luminescent agent is pre-administered to the patient (106) prior to performing the method (500).
26. 26. The method (500) of claim 24 or 25, wherein the field of view (103) includes a surgical cavity (112) exposing a body part (106) during a surgical procedure.
27. the luminescent material is a fluorescent material, The luminescence image (205; 205F) is a fluorescence image, The method (400) according to any of the preceding claims, wherein the luminescence values represent the fluorescence emitted by the fluorescent material from the corresponding emission sites illuminated by the excitation light of the fluorescent material.
28. A computer program (400) configured to, when the computer program (400) is executed on a computer device (100), cause the computer device (100) to perform a method (500) according to any of claims 1 to 27.
29. A computer program product comprising a computer readable storage medium embodying a computer program, the program instructions being loadable into a working memory of a computing device, thereby configuring the computing device to perform the method according to any of claims 1 to 27.
30. A system (100) comprising means (121) configured to perform the steps of the method (500) according to any of the claims 1 to 27.
31. 30. The system (100) of claim 30, wherein the system (100) is an imaging system configured to perform the steps of the method (500) of claim 27, an illumination unit (124, 130) for applying excitation light to the field of view (103); and an acquisition unit (136-145) for acquiring a luminescence image (205, 205F).
Citation Information
Patent Citations
Neoplasm image detecting method, and device therefor
JP2005195379A
Apparatus and method for analyzing tissue class along tubular structure
JP2006110341A
Fluorescence observation device
JP2011177419A
Image processing device, image processing system, image processing program and image processing method
JP2016118428A
Photosensitizing antibody-fluorophore conjugates
JP2017071654A