Calculation of calcium arcs to the center of the lumen

Abstract

A method, system, and apparatus including a computer readable storage medium for calculating calcium arcs of a lumen center is provided. The method includes one or more processors receiving image frames and an identification of a plaque region within the image frames, where the image frames are taken while an imaging device is within a lumen depicted in the image frames. The one or more processors identify a lumen center of the lumen within the image frames and generate an arc around the lumen center using at least the lumen center having a coverage angle centered on the lumen center.

Description

[Technical field]

[0001] [CROSS REFERENCE TO RELATED APPLICATIONS] This application claims the benefit of the filing date of U.S. Provisional Patent Application No. 63 / 178,388, entitled “CALCIUM ARC COMPUTATION RELATIVE TO LUMEN CENTER,” filed April 22, 2021, the disclosure of which is incorporated herein by reference. [Background technology]

[0002] Optical coherence tomography (OCT) is an imaging technique that has widespread applications in ophthalmology, cardiology, gastroenterology, and other fields of medical and scientific research. OCT can be used in conjunction with a variety of other imaging techniques, such as intravascular ultrasound (IVUS), angiography, fluoroscopy, and x-ray-based imaging.

[0003] To perform imaging, an imaging probe can be mounted on a catheter and can be maneuvered through a point of interest, such as a blood vessel of a patient. The imaging probe can transmit multiple image frames of the point of interest, which can be further processed or analyzed, for example, to diagnose a patient with a medical condition or as part of a scientific study. A normal artery has a layered structure that includes an intima, a media, and an adventitia. As a result of some medical conditions, such as atherosclerosis, the intima or other portions of an artery may contain plaque, which may be formed from different types of fibers, proteoglycans, lipids, or calcium. Summary of the Invention

[0004] Some aspects of the present disclosure are to calculate a coverage angle of an arc corresponding to a plaque area in the tissue surrounding the lumen. A system configured as described herein can receive image data including identification of plaque, such as calcium, in the tissue surrounding the lumen. The system can calculate a coverage angle of the device-centered arc relative to the position of the device in the lumen at the time the device captured the image. The system can calculate a center point of the lumen and use this lumen center to generate an arc referenced to the center of the lumen (lumen-centered arc) and a coverage angle corresponding to this arc. The processed image can be annotated with the lumen-centered arc, which can be further used as part of analyzing the image, for example, as part of evaluating plaque in the image under the principle of calcium scoring.

[0005] In addition to providing image frames with lumen-centered arcs for calcium scoring, the lumen-centered arcs can be displayed through a display viewport with less frame-to-frame variation compared to the device-centered arcs. The arcs referenced to the device center can vary significantly between frames of the same lumen during pullback, at least because the position of the imaging probe of the imaging device can vary as the imaging device is manipulated within the lumen. In addition, depending on the field-of-view (FOV) set for viewing the image frames, the device-centered arcs may appear larger or smaller than the corresponding detection area of ​​the plaque corresponding to the arc. The lumen-centered arcs calculated as described herein can be displayed without the aforementioned random variations caused by different FOVs and / or changes in the position of the imaging device between frames.

[0006] An aspect of the present disclosure provides a method comprising: one or more processors receiving an image frame and an identification of a plaque region within the image frame, the image frame being captured by an imaging device while the imaging device was within a lumen depicted in the image frame; the one or more processors identifying a lumen center of the lumen depicted in the image frame; and the one or more processors using at least the lumen center to generate an arc of the lumen center having a coverage angle centered on the lumen center.

[0007] The identification of the plaque region may include a mask, where a region of pixels in the mask corresponds to the plaque region depicted in the image frame.

[0008] The method may further include a step of identifying an endpoint of the plaque region, where a line tangent to the plaque region passes through the endpoint and the lumen center, the line tangent to the plaque region defining a coverage angle of an arc referenced to the center of the lumen (arc of the lumen center).

[0009] The step of generating an arc of the lumen center may further include storing a representation of the arc of the lumen center in memory, which may include position data defining the location of the endpoints and the lumen center.

[0010] The identification information of the plaque region may include a mask, where a region of pixels in the mask corresponds to the plaque region in the image frame, and the identification information may include data defining a location of the pixel in the mask corresponding to the plaque region, expressed as a polar coordinate relative to a device center of the imaging device while the imaging device was within the lumen, and the method may further include a step of converting the location of the pixel from the polar coordinate relative to the device center to a polar coordinate relative to the lumen center.

[0011] Transforming the pixel location may include transforming the pixel location from polar coordinates relative to the device center to Cartesian coordinates, and transforming the pixel location from Cartesian coordinates to polar coordinates relative to the lumen center.

[0012] The method may further include generating an identification of the plaque region from the image frames, which may include receiving the image frames and processing the image frames through one or more machine learning models trained to identify one or more predictive regions of plaque within the image frames.

[0013] The step of identifying the lumen center may include calculating a lumen contour of the lumen, generating a spline estimate of the lumen contour, and identifying the lumen center as an estimate of the center of gravity from a plurality of samples of the spline estimate.

[0014] The image frame may be a first image frame, one of a plurality of image frames in a sequence, and the lumen center is a first lumen center, and the method may further include identifying a respective lumen center for each of the plurality of image frames in the sequence; applying a low-pass smoothing filter to the respective lumen center for each of the plurality of image frames including the first image frame and the first lumen center; and generating an arc of a second lumen center having a coverage angle centered on the smoothed lumen center using at least the lumen center after applying the low-pass smoothing filter and an endpoint of the arc of the device center.

[0015] The step of applying a low pass smoothing filter may further include applying a low pass smoothing filter on an array of lumen centers of the multiple image frames that are symmetric around the lumen center of the image frame in the middle of the sequence, and the filter may further include one or more filter coefficients applied to the first image frame, the one or more filter coefficients depending at least on a relative frame index between the image frame in the middle of the sequence and the first image frame.

[0016] The step of identifying a respective lumen center for each of a plurality of image frames in the sequence may further include identifying one or more image frames depicting a respective side branch from the lumen, and, for each of the one or more image frames, interpolating the respective lumen center from lumen centers of adjacent image frames in the sequence that do not depict the respective side branch.

[0017] The method may further include displaying the image frames annotated with the lumen center arc on a display coupled to the one or more processors.

[0018] The step of displaying the image frame annotated with an arc of the lumen center may further include displaying the image frame within a field of view of a display viewport having a boundary, including displaying the arc of the lumen center along a boundary of the display viewport.

[0019] The imaging device may be an optical coherence tomography (OCT) imaging device or an intravascular ultrasound (IVUS) imaging device.

[0020] The method may further include generating, by the one or more processors, one or more calcium scoring metric values ​​corresponding to the plaque region using, at least in part, the coverage angle of the arc of the lumen center.

[0021] The plaque region may be a first plaque region of a plurality of plaque regions in the image frame, and the method may further include generating, for each of the plurality of plaque regions, an arc of a respective lumen center having a respective coverage angle centered on the lumen center.

[0022] The plaque region may be calcium.

[0023] Some aspects of the present disclosure also provide a system comprising one or more processors configured to receive image frames and an identification of a plaque region within the image frames, where the image frames are captured by an imaging device while the imaging device is within a lumen depicted in the image frames, identify a lumen center of the lumen depicted in the image frames, and generate an arc of the lumen center using at least the lumen center having a coverage angle centered on the lumen center.

[0024] The identification of the plaque region may include a mask, where a region of pixels in the mask corresponds to the plaque region depicted in the image frame.

[0025] The one or more processors can be further configured to identify an endpoint of the plaque region, where a line tangent to the plaque region passes through the endpoint and the lumen center, and where the line tangent to the plaque region defines a coverage angle of an arc of the lumen center.

[0026] To generate the arc of the lumen center, the one or more processors may be further configured to store a representation of the arc of the lumen center in memory, which may include position data defining the endpoints and the position of the lumen center.

[0027] The identification information of the plaque region may include a mask, where a region of pixels in the mask corresponds to the plaque region in the image frame, and the identification information may include data defining a location of the pixel in the mask corresponding to the plaque region, expressed as a polar coordinate relative to a device center of the imaging device while the imaging device was within the lumen, and the one or more processors are further configured to convert the location of the pixel from the polar coordinate relative to the device center to a polar coordinate relative to the lumen center.

[0028] Transforming the pixel location may further include transforming the pixel location from polar coordinates relative to the device center to Cartesian coordinates, and transforming the pixel location from Cartesian coordinates to polar coordinates relative to the lumen center.

[0029] The one or more processors may be further configured to receive the image frames and generate identification information of plaque regions from the image frames, including processing the image frames through one or more machine learning models trained to identify one or more predicted regions of plaque within the image frames.

[0030] To identify the lumen center, the one or more processors may be further configured to calculate a lumen contour of the lumen, generate a spline estimate of the lumen contour, and identify the lumen center as an estimate of the center of gravity from a plurality of samples of the spline estimate.

[0031] The image frame may be a first image frame, one of a plurality of image frames in the sequence, and the lumen center is a first lumen center, and the one or more processors are further configured to: identify a respective lumen center for each of the plurality of image frames in the sequence; apply a low-pass smoothing filter to the respective lumen center for each of the plurality of image frames including the first image frame and the first lumen center; and generate an arc of a second lumen center having a coverage angle centered on the smoothed lumen center using at least the lumen center after applying the low-pass smoothing filter and an endpoint of the arc of the device center.

[0032] To apply the low-pass smoothing filter, the one or more processors may be further configured to apply a low-pass smoothing filter on an array of lumen centers of the multiple images that are symmetric around the lumen center of the image frame in the middle of the sequence, and the filter may further include one or more filter coefficients applied to the first image frame, the one or more filter coefficients depending at least on a relative frame index between the image frame in the middle of the sequence and the first image frame.

[0033] To identify a respective lumen center for each of a plurality of image frames in the sequence, the one or more processors may be further configured to identify one or more image frames depicting a respective side branch from the lumen, and for each of the one or more image frames, interpolate the respective lumen center from lumen centers of adjacent image frames in the sequence that do not depict the respective side branch.

[0034] The method may further include displaying the image frame annotated with the lumen center arc on a display coupled to the one or more processors.

[0035] To display the image frame annotated with an arc of the lumen center, the one or more processors may be further configured to display the image frame within a field of view of a display viewport having a boundary, including displaying the arc of the lumen center along a boundary of the display viewport.

[0036] The imaging device may be an optical coherence tomography (OCT) imaging device or an intravascular ultrasound (IVUS) imaging device.

[0037] The one or more processors may be further configured to generate one or more calcium scoring metric values ​​corresponding to plaque regions using, at least in part, the coverage angle of the arc of the lumen center.

[0038] The plaque region is a first plaque region of a plurality of plaque regions in the image frame, and the one or more processors are further configured to generate, for each of the plurality of plaque regions, an arc of a respective lumen center having a respective coverage angle centered on the lumen center.

[0039] Aspects of the present disclosure also provide one or more non-transitory computer-readable storage media storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations including receiving, by the one or more processors, an image frame and an identification of a plaque region within the image frame, where the image frame was taken by an imaging device while the imaging device was within a lumen depicted in the image frame; identifying, by the one or more processors, a lumen center of the lumen depicted in the image frame; and generating, by the one or more processors, an arc of the lumen center having a coverage angle centered on the lumen center using at least the lumen center.

[0040] The identification of the plaque region may include a mask, where regions of pixels in the mask correspond to the plaque regions in the image frame.

[0041] The operations may further include identifying an endpoint of the plaque region, where a line tangent to the plaque region passes through the endpoint and the lumen center, where the line tangent to the plaque region defines a coverage angle of an arc of the lumen center.

[0042] Generating the arc of the lumen center may further include storing a representation of the arc of the lumen center in memory, which may include position data defining the location of the endpoints and the lumen center.

[0043] The identification information of the plaque region may include a mask, where a region of pixels in the mask corresponds to the plaque region in the image frame, and the identification information may include data defining a location of the pixel in the mask corresponding to the plaque region, expressed as a polar coordinate relative to a device center of the imaging device while the imaging device was within the lumen, and the operation may further include converting the location of the pixel from the polar coordinate relative to the device center to a polar coordinate relative to the lumen center.

[0044] Transforming the pixel location may further include transforming the pixel location from polar coordinates relative to the device center to Cartesian coordinates, and transforming the pixel location from Cartesian coordinates to polar coordinates relative to the lumen center.

[0045] The operations may further include generating an identification of plaque regions from the image frames, which may include receiving the image frames and processing the image frames through one or more machine learning models trained to identify one or more predicted regions of plaque within the image frames.

[0046] Identifying the center point of the lumen may include calculating a lumen contour of the lumen, generating a spline estimate of the lumen contour, and identifying the lumen center as an estimate of the center of gravity from a plurality of samples of the spline estimate.

[0047] The image frame is a first image frame and is one of a plurality of image frames in the sequence, and the lumen center is a first lumen center, and the operations may further include identifying a respective lumen center for each of the plurality of image frames in the sequence; applying a low-pass smoothing filter to the respective lumen center for each of the plurality of image frames including the first image frame and the first lumen center; and generating an arc of a second lumen center having a coverage angle centered on the smoothed lumen center using at least the lumen center after applying the low-pass smoothing filter and an endpoint of the arc of the device center.

[0048] Applying the low pass smoothing filter may further include applying a low pass smoothing filter on an array of lumen centers of the multiple images that is symmetric around the lumen center of the image frame in the middle of the sequence, and the filter may further include one or more filter coefficients applied to the first image frame, the one or more filter coefficients depending at least on a relative frame index between the image frame in the middle of the sequence and the first image frame.

[0049] Identifying a respective lumen center for each of a plurality of image frames in the sequence may further include identifying one or more image frames depicting a respective side branch of the lumen, and, for each of the one or more image frames, interpolating the respective lumen center from lumen centers of adjacent image frames in the sequence that do not depict the respective side branch.

[0050] The operations may further include displaying the image frame annotated with the lumen center arc on a display coupled to the one or more processors.

[0051] Displaying the image frame annotated with an arc of the lumen center may further include displaying the image frame within a field of view of a display viewport having a boundary, including displaying the arc of the lumen center along a boundary of the display viewport.

[0052] The method may include generating a display angle that corresponds to the coverage angle based on the coverage angle and a circumference of a display port of a display device configured to display the image frame.

[0053] Generating the viewing angle may include generating the viewing angle in response to receiving an indication that a field of view value of a display device displaying the image frame has changed.

[0054] The one or more processors may be further configured to generate a display angle corresponding to the coverage angle based on the coverage angle and a circumference of a display port of a display device configured to display the image frame.

[0055] To generate the viewing angle, the one or more processors are further configured to generate the viewing angle in response to receiving an indication that a field of view value of a display device displaying the image frame has changed.

[0056] The operations may further include generating a display angle corresponding to the coverage angle based on the coverage angle and a circumference of a display port of a display device configured to display the image frame.

[0057] Generating the viewing angle may include generating the viewing angle in response to receiving an indication that a field of view value of a display device displaying the image frame has changed. [Brief description of the drawings]

[0058] [Figure 1]FIG. 13 illustrates an image frame annotated with calcium arcs referenced to the center of the lumen, according to aspects of the present disclosure. [Diagram 2] FIG. 1 is a block diagram of an exemplary image processing system according to aspects of the present disclosure. [Diagram 3] 13 is a flowchart of an exemplary process for calculating a calcium arc referenced to the center of a lumen, according to aspects of the present disclosure. [Figure 4A] FIG. 1 shows a calcium arc referenced to the center of the device. [Figure 4B] FIG. 4B shows a calcium mask of the calcium region shown in FIG. 4A. [Figure 5A] FIG. 13 shows the coverage angle from both the calcium arc referenced to the center of the device and the calcium arc referenced to the center of the lumen in the image frame. [Figure 5B] FIG. 5B shows a calcium mask identifying the calcium region between the endpoints as shown in FIG. 5A. [Figure 6] 13 is a flowchart of an exemplary process for calculating a calcium arc referenced to the center of a lumen in an image frame, according to aspects of the present disclosure. [Figure 7] FIG. 13 shows a calcium mask spanning the top and bottom of an image frame. [Figure 8] 11 is a flowchart of an exemplary process for identifying a lumen center of an image frame, according to an aspect of the present disclosure. [Figure 9] 13 is a flowchart of an exemplary process for smoothing a lumen center of a sequence of image frames, according to an aspect of the present disclosure. [Figure 10] 1 is a chart illustrating the relationship between relative frame index and filter weights. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0059] (overview) Aspects of the present disclosure provide for using at least the lumen center to calculate a calcium arc referenced to the lumen center of a calcium region in a luminal image frame (a lumen center calcium arc).

[0060] A calcium arc is a measure of the angle that calcium forms in tissue surrounding a lumen. For example, calcium in tissue around half the lumen will form a calcium arc with a coverage angle of 180 degrees, while calcium in tissue around three-quarters of the lumen will form a calcium arc with a coverage angle of 270 degrees. Although reference is made herein to calcium arcs corresponding to calcium in tissue, aspects of the present disclosure also include applying the described techniques to plaque and other identified areas of interest in tissue surrounding a lumen. Other examples of plaque areas include fibrous or lipid.

[0061] An imaging device, such as a catheter having an imaging probe, may be configured to be manipulated through the lumen and capture images as a sequence of image frames. Raw image frames collected from the device are initially centered relative to the position of the probe within the lumen at the time the image is captured. As used herein, the device center in an image frame refers to the position of the imaging probe within the lumen at the time the imaging probe captured the image frame. In examples where the imaging device is a catheter, the device center may also be referred to as the catheter center measured from the center of the catheter as shown in the image frame captured by the imaging probe. A system configured to process the images may identify from the image frames the presence of areas of plaque, such as calcium, found in the tissue surrounding the lumen.

[0062] Some diagnostic techniques, such as calcium scoring, use at least in part information about calcium arcs formed by calcium regions identified in image frames of blood vessels and surrounding tissue. Calcium scoring refers to a technique that quantifies calcium features in plaque detected in biomedical images, such as images of blood vessels. Calcium scoring has been used to assess a patient's risk of heart attack or other cardiovascular disease. The biomedical images used for calcium scoring can be obtained from a device, such as a catheter, having an imaging probe configured to take images of the blood vessels while passing through the vessels.

[0063] The principle of calcium scoring often considers the calcium arc of a calcium region relative to the center point of the lumen in the image frame, rather than the center point of the device capturing the image. The calcium arc relative to the center point of the lumen is referred to herein as the calcium arc of the lumen center, and the center point of the lumen or lumen center of the image frame refers to the location of the center of the lumen depicted in the image frame.

[0064] One problem is that techniques to identify calcium regions often require the use of raw image data that does not identify the luminal center of each of the images in the data. Even after identifying the luminal center of the image, another problem is to accurately identify the coverage angle of the calcium arc of the luminal center.

[0065] Aspects of the present disclosure provide techniques for accurately calculating a luminal center calcium arc from raw image data identifying plaque regions. By calculating a luminal center calcium arc for a calcium region, the coverage angle for the luminal center calcium arc can be more accurately identified compared to the coverage angle for a device center calcium arc on the same calcium region. Furthermore, the calculated luminal center calcium arc can be displayed more consistently for image frame recordings captured during pullback compared to displaying a device center arc. The luminal center arc as described herein can be displayed according to a viewing angle that is undistorted relative to the field of view in which the image frames are viewed through a display viewport. In this manner, the luminal center arc can be displayed in a manner that is undistorted relative to image frames viewed using different fields of view, as described in more detail herein.

[0066] FIG. 1 illustrates an image frame 100 annotated with lumen-centered calcium arcs 105A and 100B according to an embodiment of the present disclosure. The image frame 100 also illustrates the lumen 102, the device center 110 of the device (not shown) that captured the image frame 100, and the lumen center 115. The image frame 100 may include data identifying calcium regions 106A and 106B, for example, as a mask, such as the mask illustrated and described with reference to FIG. 4B. The image frame 100 may be displayed through a display viewport that includes a viewport boundary 101. The viewport boundary 101 surrounds a portion of the image frame 100 depicting the portion of the lumen and surrounding tissue captured by the imaging device. As an example, the image frame 100 may be displayed depicting a two-dimensional cross-section of the imaged lumen. Calcium arcs 105A and 105B around the center of the lumen are shown on the viewport boundary 101. The image frame 100 may be viewed according to different values ​​of the field of view (FOV) of the image frame. Image frame 100 may be displayed at different FOV values, which generally correspond to how much of the lumen and surrounding tissue is visible within viewport boundary 101 at one time.

[0067] The lumen-centered calcium arcs can be annotated on the image frames to provide a more accurate comparison between image frames taken from the same sequence by an intraluminal imaging device relative to the device-centered calcium arcs. This is because, at least, the lumen-centered calcium arcs are based on the location of the lumen center and change less across the sequence of images compared to maneuvering the imaging device through the lumen. In addition, the system is configured to calculate a display angle corresponding to the coverage angle of the lumen-centered arcs, which can be displayed without distortion between image frames. For example, as shown in FIG. 1, the lumen-centered arcs 105A and 105B can be consistently shown on the viewport boundary 101 regardless of the current FOV in which the image frame 100 is displayed. The system can detect whether the FOV of the image has been updated and, in response, update the display angle representing the coverage angle of the lumen-centered arcs, which would otherwise appear distorted. The distortion can occur at least because the image frames are displayed relative to the center of the device responsible for capturing the images. While the device-centered arc can be displayed at the same angle regardless of FOV, the lumen-centered arc cannot. Therefore, the system is configured to separately calculate a coverage angle for the lumen-centered arc that can be used for downstream processes such as calcium scoring, and a display angle that corresponds to the coverage angle and does not appear distorted across different FOVs.

[0068] The lumen-centered calcium arc can also be provided as an input to downstream processes for calcium scoring. A lumen-centered calcium arc can yield a more accurate score, at least because the coverage angle of the arc is more consistently identified relative to the lumen center.

[0069] Aspects of the present disclosure also provide techniques for identifying a lumen center of a lumen depicted over a sequence of image frames, such as a video recording captured by an imaging device. The location of the lumen center can be smoothed over multiple image frames to reduce jitter and lumen center inconsistency between frames. Furthermore, a system configured according to aspects of the present disclosure can calculate and provide calcium arcs of multiple lumen centers based on both the identified lumen centers for the image frames and / or the smoothed lumen centers. In addition, aspects of the present disclosure can be more easily integrated into existing processing pipelines because at least only raw image frames (generally showing the device center for each frame) are required. In other words, the described techniques can receive raw image data having identified plaque regions from any of a variety of sources without requiring additional pre-processing.

[0070] (Example System) FIG. 2 is a block diagram of an exemplary imaging system 200 according to an aspect of the disclosure. The system 200 may include an imaging device 205 with an imaging probe 204 that may be used to image a lumen 202, such as the lumen of a blood vessel. The imaging device 205 may be, for example, a catheter. The imaging probe 204 may be an OCT probe and / or an IVUS catheter. Although the examples provided herein refer to an OCT probe, the use of an OCT probe is not intended to be limiting. An IVUS catheter may be used in conjunction with or in place of an OCT probe. A guidewire (not shown) may be used to introduce the probe 204 into the lumen 202. The probe 204 may be introduced and pulled back along the length of the lumen while collecting data, for example, as a sequence of image frames. According to some examples, the probe 204 may be held stationary during pullback such that multiple scans of OCT and / or IVUS data sets may be collected. The data sets or frames of image data can be used to identify features such as calcium arcs relative to the device or lumen center, as described herein.

[0071] The probe 204 can be connected to an image processing subsystem 208 via an optical fiber 206. The image processing subsystem 208 can include a light source such as a laser, an interferometer having a sample arm and a reference arm, various optical paths, a clock generator, photodiodes, and other OCT and / or IVUS components.

[0072] The probe 204 can be connected to an optical receiver 210. According to some examples, the optical receiver 210 can be a balanced photodiode-based system. The optical receiver 210 can be configured to receive light collected by the probe 204.

[0073] The subsystem 208 may include a computing device 212. The computing device 212 may include one or more processors 213, memory 214, instructions 215, and data 216. The computing device 212 may also implement a calcium mask engine 220, a luminal center engine 225, and / or a calcium arc engine 230.

[0074] The one or more processors 213 may be any combination of a variety of different processors, such as commercially available microprocessors. Alternatively, the one or more processors may be one or more devices, such as a graphics processing unit (GPU), a field programmable gate array (FPGA), and / or an application specific integrated circuit (ASIC). Although FIG. 2 functionally depicts the processor, memory, and other elements of device 212 as being in the same block, it will be understood by those skilled in the art that a processor, computing device, or memory may actually include multiple processors, computing devices, or memories that may or may not be stored in the same physical enclosure. Similarly, the memory may be a hard drive or other storage medium located in a different enclosure than the enclosure of computing device 212. Thus, it will be understood that reference to a processor or computing device includes reference to a collection of processors or computing devices or memories that may or may not operate in parallel.

[0075] The memory 214 can store information accessible by the processor, including instructions 215 executable by the processor 213, and data 216. The memory 214 can be any type of memory operable to store information accessible by the processor 213, including non-transitory computer-readable media, such as hard drives, memory cards, read-only memory ("ROM"), random access memory ("RAM"), optical disks, and other writable and read-only memory, or other media that stores data that can be read with the aid of an electronic device. The subject matter disclosed herein can include different combinations of the foregoing, where different portions of the instructions 215 and data 216 are stored in different types of media.

[0076] Memory 214 may be retrieved, stored, or modified by processor 213 according to instructions 215. For example, data 216 may be stored in computer registers, in a relational database as a table with multiple different fields and records, in an XML document, or in a flat file, although this disclosure is not limited by a particular data structure. Data 216 may also be formatted in a computer-readable format, such as, but not limited to, binary values, ASCII, or Unicode. Simply by way of further example, data 216 may be stored compressed or uncompressed, or in various image formats (e.g., JPEG), vector-based formats (e.g., Scalable Vector Graphics (SVG)), or as a bitmap that includes pixels stored with computer instructions that render graphics. Additionally, data 216 may include sufficient information to identify the relevant information, such as numbers, descriptive text, proprietary codes, pointers, references to data stored in other memories (including other network locations), or information used by a function to calculate the relevant data.

[0077] The instructions 215 may be any set of instructions executed by the processor 213, either directly, such as machine code, or indirectly, such as a script. In that regard, the terms "instructions," "application," "steps," and "program" may be used interchangeably herein. The instructions may be stored in object code format for direct processing by the processor, or in any other computing device language, including a script or collection of separate source code modules that are interpreted on demand or pre-compiled. The function, method, and routines of the instructions are described in more detail below, including descriptions of the calcium mask engine 220, the lumen-centered engine 225, and the calcium arc engine 230.

[0078] The computing device 212 may receive one or more image frames collected by the imaging probe 204 while being maneuvered through the lumen 202. The computing device 212 may be configured to process the input raw image frames through a calcium mask engine 220. The calcium mask engine 220 is configured to receive the image frames as input and identify one or more plaque regions (or "blobs") within the images. To do so, the calcium mask engine 220 may implement any of a variety of image processing techniques, including one or more machine learning models trained to perform image segmentation on the input image data. Although the subsystem 208 is described as implementing an engine 220 that identifies calcium, it should be understood that the subsystem 208 may be generally configured for any image segmentation task, such as for identifying one or more plaque regions within an input image frame.

[0079] Although the subsystem 208 has been described as processing raw image frames received from the imaging probe 204 through the optical receiver 210, in some implementations the subsystem 208 receives image frames with one or more identified plaque regions. In other words, instead of processing the image frames to identify plaque regions, the subsystem 208 may be configured to receive processed images from another device, such as an image processing device, a storage device, etc.

[0080] The identity of the plaque regions of the image can be represented in a number of ways. For example, the plaque regions can be annotated in the image frame with some visible indicator, such as pixel outlines, perimeters, or shaded areas. The annotations can be included in an image overlay, added directly to the image, or included in any other manner. Additionally or alternatively, the image frame can be accompanied by a calcium mask. The calcium mask can correspond to pixels in the image frame that are predicted to correspond to calcium regions. Figures 4B and 5B, described below, show examples of calcium masks over predicted regions of calcium in their respective image frames.

[0081] In some examples, the plaque region may not be annotated in the image frame, and instead engine 225 is configured to identify the perimeter of the region instead. In examples where the plaque region is not annotated, engine 225 can identify the perimeter of the plaque region. In some examples, engine 225 stores data indicating the location of pixels representing an identified region of plaque or the perimeter of the identified region. The data can be, for example, a mask, a list of pixel locations, or the like. In some examples, engine 225 can generate a visual annotation of the region after identification. In examples where engine 225 does not receive a mask or other data indicating the location of various different plaque regions, engine 225 can instead perform its own identification and generate data as described herein identifying the plaque region.

[0082] In one example, engine 225 can process scanlines or rows of pixels in a received image and identify the intensity of backscatter from each pixel and / or edge based on attenuation calculated from pixel intensity gradients along the scanline. Engine 225 can identify which type of tissue, lesion, or plaque a pixel corresponds to from the measured intensity and / or attenuation. For example, higher intensity, e.g., brighter reflection of backscatter, and lower attenuation, e.g., lower intensity change from pixel to pixel, can correspond to fibrous tissue. Lower intensity, lower attenuation, and sharper edges between pixels measured at different intensity / attenuation can correspond to calcium regions. Lower intensity, higher attenuation, and diffuse edges between pixels measured at different intensity / attenuation can correspond to lipid regions, which motivates the determination of lipid offset using edges, as described above. Engine 225 can be configured with various predefined threshold detection algorithms or adaptive gradient detection algorithms of intensity and / or attenuation for use in comparing lower intensity to higher intensity, lower attenuation to higher attenuation, etc., while processing the image. This allows the software to identify not only the scanlines that contain various morphologies (such as lipids), but also the offsets that the morphologies are found in. The scanline / offset information can be used to generate arcs of lumen-centered morphologies similar to calcium arcs, in cases such as lipid detection where scanline-based classification works but there is no mask to perform the analysis on.

[0083] The lumen center engine 225 can be configured to receive the image frames and calculate a respective lumen center for each image frame, as well as smooth the lumen centers across the sequence of image frames. As described in more detail herein with reference to Figures 8 and 9, aspects of the disclosure provide techniques for identifying lumen centers within individual image frames and applying a smoothing filter to the locations of the lumen centers across the sequence of image frames.

[0084] The calcium arc engine 230 can be configured to calculate the calcium arc of the lumen center, for example, as described in more detail with reference to Figures 3-6. The calcium arc engine 230 can receive as input the captured image frames from the imaging probe 204 through the optical receiver 210. The calcium arc engine 230 can also receive as input data identifying one or more calcium regions within the received image frames, which data may be generated, for example, by the calcium mask engine 220. The calcium arc engine 230 can also receive data corresponding to the lumen center and / or smoothed lumen center of the received image frames, for example, from the lumen center engine 225.

[0085] The calcium arc engine 230 can generate as output data corresponding to the generated lumen-centered calcium arc for each image frame and for each calcium region within the image frame. The calcium arc engine 230 can provide the lumen-centered calcium arc as a visual annotation to the received image frame, for example as shown in FIG. 1. In some implementations, the calcium arc engine 230 can adjust the lumen-centered calcium arc for display on the monitor based on the field of view of the display viewport in which the image of the lumen is displayed.

[0086] Alternatively or in addition, the calcium arc engine 230 can provide data corresponding to the calculated lumen-centered calcium arc to a downstream process configured to receive and process the lumen-centered calcium arc. The downstream process can be implemented entirely as one or more computer programs on the device 212 or on a separate device. This process can, for example, receive the calculated lumen-centered calcium arc and data corresponding to the image frames and use the data as part of a calcium scoring process for calcium regions identified within the image frames.

[0087] The image processing subsystem 208 can include a display 218 that outputs content to a user. As shown, the display 218 is separate from the computing device 212, although according to some examples, the display 218 can be part of the computing device 212. The display 218 can output image data related to one or more features detected within the lumen. For example, the output can include, but is not limited to, an image frame annotated with a lumen-centered calcium arc, a lumen center, and / or a smoothed lumen center for each calcium region identified within the image frame. The display 218 can display the output through a display viewport, such as a circular display viewport, in some examples. The display 218 can show the image frames in sequence against the field of view value.

[0088] As described herein with reference to FIG. 6, the system 200 can be configured to calculate a display angle corresponding to the coverage angle of an arc of the lumen center, taking into account the field of view value to which the display 218 is currently set. In response to an input that adjusts the FOV, the system 200 can be configured to subsequently recalculate the display angle of the corresponding coverage angle of the currently displayed calcium region. The display 218 can be configured in some implementations to output both the display angle and the coverage angle corresponding to the display angle. In some examples, the coverage angle value is shown, but the arc corresponding to the display angle is shown on the display 218.

[0089] The display 218 may show one or more image frames, for example, as a two-dimensional cross-section of the lumen 202 and surrounding tissue being imaged. The display 218 may also include one or more other views to show different perspectives of the imaged lumen or another region of interest within the patient's body. As an example, the display 218 may include a longitudinal view of the length of the lumen 202 from a start point to an end point. In some examples, the display 218 may highlight certain portions of the lumen 202 along the longitudinal view and at least partially occlude other portions that are not currently selected for viewing. In some examples, the display 218 is configured to receive input to scrub through different portions of the lumen 202 as shown in the longitudinal view.

[0090] The output may be displayed, for example, in real time during the procedure as the imaging probe 204 is maneuvered through the lumen 202. For example, other data that may be output in combination with the lumen center arc described above includes device center arc, cross-sectional scan data, longitudinal scans, diameter graphs, lumen boundary, plaque size, plaque circumference, visual indicia of plaque location, visual indicia of risk posed for stent expansion, flow rate, etc. The display 218 may identify features using text, arrows, color coding, highlighting, outlines, or other suitable human or machine readable indicia.

[0091] According to some examples, the display 218 may be a graphic user interface ("GUI"). One or more steps may be performed automatically or without user input to navigate the image, enter information, select, and / or interact with input. The display 218, alone or in combination with the computing device 212, may enable toggling between one or more viewing modes in response to user input. For example, a user may be able to toggle between different side branches on the display 218, such as by selecting a particular side branch and / or by selecting a view associated with a particular side branch.

[0092] In some examples, the display 218, alone or in combination with the computing device 212, may include a menu. The menu may allow a user to show or hide various features. There may be more than one menu. For example, there may be a menu to select which lumen features to display. Additionally or alternatively, there may be a menu to select a virtual camera angle for the display. In some examples, the display 218 may be configured to receive input. For example, the display 218 may include a touch screen configured to receive touch input to interact with menus or other interactable elements displayed on the display 218.

[0093] Computing device 212 can communicate directly and indirectly with other devices via network 260. Computing device 212 can set up listening sockets that can accept initiating connections to send and receive information. Network 260 itself can include a variety of configurations and protocols, including the Internet, the World Wide Web, intranets, virtual private networks, wide area networks, local networks, and private networks using one or more company-specific communication protocols. The network can support a variety of short-range and long-range connections. The short-range and long-range connections can be made across different bandwidths, such as 2.402 GHz to 2.480 GHz (commonly associated with the Bluetooth™ standard), 2.4 GHz and 5 GHz (commonly associated with the Wi-Fi™ communication protocol), or using a variety of communication standards, such as the LTE™ standard for wireless broadband communication. Network 260 can additionally or alternatively support wired connections between device 212 and one or more other devices (not shown), including via various types of Ethernet connections. For example, device 212 can communicate image frames annotated with lumen-centered arcs to one or more devices configured to process the annotated frames. Additionally or alternatively, device 212 can communicate between one or more other devices implementing at least a portion of system 200. As another example, device 212 can receive image frames via network 260 from an optical receiver connected to a device separate from device 212.

[0094] Exemplary Methods 3 is a flow chart of an exemplary process 300 for calculating lumen-centered calcium arcs according to an embodiment of the present disclosure. A system having one or more processors and appropriately programmed according to an embodiment of the present disclosure can perform the process 300. For example, an image processing system such as image processing system 200 of FIG. 2 can perform the process 300.

[0095] The system receives image frames and identification of calcium regions within the image frames, according to block 310. As described herein with reference to FIG. 2, a calcium mask engine for the system may be configured to process raw image frames from the imaging probe and generate a calcium mask for each frame. In other examples, the system may receive image frames and corresponding masks or other data identifying calcium regions entirely from another device. Although described with reference to a single image frame, the system may be configured to receive multiple image frames, e.g., as a stream of images in real time during an imaging procedure, or as a batch of data. In either case, the system may be configured to repeat process 300 for each image frame, e.g., sequentially or in parallel.

[0096] The system receives a lumen center of a lumen in an image frame according to block 330. The system may be configured to calculate a lumen center and / or a smoothed lumen center of the lumen, for example, using a lumen center engine configured to perform the processes described herein with reference to Figures 8 and 9. The lumen center may be represented, for example, in polar coordinates. The polar coordinates may be represented by two parameters, a radius and an angle, relative to a reference point, which may be initially relative to the device center.

[0097] The system generates calcium arcs from lumen center to lumen center of the lumen in the image frame according to block 340. The system can convert the lumen center from polar coordinates to Cartesian coordinates relative to the device center. For example, the system can calculate the Cartesian coordinate of a point expressed in polar coordinates by calculating:

number

[0098] As part of generating the lumen-centered calcium arc, the system identifies the endpoints of the calcium region as described herein with reference to Figures 5A, 5B, and 6. The endpoints and the lumen center define a line tangent to the calcium region. The system can calculate the Cartesian coordinates of the device-centered calcium arc endpoint and the lumen center. The lumen-centered calcium arc endpoint, together with the lumen center, define the exact coverage angle of the arc relative to the calcium region.

[0099] The system can calculate the coverage angle from the start and end angles of the lumen-centered calcium arc for the calcium region. For example, the system can calculate the coverage angle of the lumen-centered calcium arc as follows:

number

[0100] In embodiments where both lumen centers and smoothed lumen centers are calculated for an image frame, the system may calculate calcium arcs for each lumen center according to block 340 for both centers.

[0101] The system outputs the lumen center arc (or arcs for multiple calcium regions in the image frame) according to block 350. For example, as described with reference to FIG. 2, the system can output the lumen center arc on the display as an annotation on the image frame. The system can show the lumen center arc on the image frame and along a viewport boundary of the display viewport used to display the image frame. For example, the system can project a tangent line defined by the endpoints of the calcium region and the lumen center through the boundary. The system can annotate the portion of the boundary between the line projected through the boundary as a lumen center calcium arc.

[0102] In other examples, the system can output data defining the lumen center calcium arc for downstream processing, e.g., calcium scoring. The calcium scoring can be performed automatically by the system or another system, or the calcium scoring can be performed, e.g., manually. The system can output calcium arcs for both the lumen center and the smoothed lumen center in embodiments where both are calculated.

[0103] 4A shows calcium arcs 401 A and 401 B at the center of the device. The device center 410 is shown at the top of the lumen 402 captured in the image frame 400.

[0104] Figure 4B shows a calcium mask 415 of the calcium region shown in Figure 4A. The calcium mask 415 includes calcium identifiers 417A and 417B, each corresponding to a respective identified region of calcium identified in the image frame 400. In this example, the calcium identifiers 417A and 417B are regions of pixels of particular color values ​​that distinguish the calcium region from other regions of the image frame 400, such as regions depicting the lumen 402 or the media surrounding the lumen 402. The calcium mask 415 is an example of an identification of the calcium region in the image frame 400 and may be received by the system, for example, from a calcium mask engine implemented as part of the system or from another source.

[0105] The calcium identifier may be represented in any of a variety of different ways. In some examples, the calcium identifier may be represented as a mesh or patterned region of pixels, or as pixels corresponding to the outline of a calcium detection region. In other examples, the calcium identifier is represented as a matrix of values, where each value corresponds to a pixel in the image frame. Each value may be binary, for example having a value of 0 indicating that the corresponding pixel is not part of a calcium region, and a value of 1 indicating that the corresponding pixel is part of a calcium region.

[0106] FIG. 5A shows coverage angles 502A and 502B for both lumen-centered calcium arc 501A and device-centered calcium arc 501B in image frame 500A. Image frame 500A includes lumen center 515 and device center 510. Endpoints 503A are endpoints of the calcium region calculated, for example, as described herein with reference to FIGS. 3 and 6. Arc point 504A is the point where a line tangent to the calcium region passes through a viewport boundary (not shown). Using arc point 504A, the system can annotate for displaying device-centered calcium arc 501B along the viewport boundary. Lumen-centered calcium arc 501A is also shown. Unlike device-centered calcium arc 501B, lumen-centered calcium arc 501A can move along the viewport boundary when displayed for different FOV values ​​of the image frame.

[0107] The system can leverage the known device center to calculate calcium region and lumen center end points to arrive at a more accurate lumen center calcium arc than the device center calcium arc. In this way, the system can process image data from a variety of different sources, which may include models predicting features of the image data, such as calcium masks. In other words, the system is configured to calculate the lumen center and lumen center calcium arc at least as a post-processing step, and thus is more compatible with processing on raw image data.

[0108] Figure 5B shows a calcium mask 500B that identifies calcium regions between endpoints 503A. Figure 5B shows the relationship between endpoints 503A that are spaced along the maximum width of the calcium regions identified by calcium identifiers 517B. The calcium mask 500B also shows the calcium regions identified by calcium identifiers 517A. Figure 6 herein shows the process of calculating endpoints 503A.

[0109] 6 is a flow chart of an exemplary process 600 for identifying luminal-centered calcium arcs in an image frame according to an embodiment of the present disclosure. A system such as system 200 of FIG. 2 can perform process 600.

[0110] The system receives a calcium mask according to block 605. For ease of explanation, the calcium mask is assumed to identify a single calcium region, although it will be understood that the system can perform process 600 for each region identified in the calcium mask.

[0111] The system receives a location of the lumen center of the image frame corresponding to the calcium mask, according to block 610. The lumen center can be calculated, for example, as described herein with reference to Figure 8. The lumen center can also be a smoothed lumen center, generated, for example, according to process 900 of Figure 9.

[0112] The system determines whether there are additional surrounding pixels that identify calcium regions in the mask, according to block 615. The surrounding pixels may be pixels along the perimeter of the identified region, such as, by way of example, the pixels around identifier 417B in FIG. 4B. The system may iterate through each of these surrounding pixels.

[0113] If the system determines that there are more surrounding pixels to process ("YES"), the system converts the polar pixel locations to Cartesian coordinates according to block 620. The polar pixel locations are relative to the device center in the image frame, i.e., the polar radius and angle are expressed with the device center as the center. As part of the data defining the calcium mask, the system may receive coordinate data corresponding to the locations of pixels that collectively identify the calcium region. The coordinate data may initially be relative to the device center, at least since the device center is available in the raw image frames that are processed to generate the corresponding calcium mask.

[0114] The system converts the Cartesian coordinates converted according to block 620 to lumen-centered polar coordinates according to block 622. As described herein, any coordinate system can be used instead of polar coordinates. In some embodiments, the system is configured to perform any necessary conversions from a conversion from a start coordinate system, such as a device-centered polar coordinate system, to an end coordinate system, such as a lumen-centered polar coordinate system. The system can convert coordinates between intermediate coordinate systems, such as device-centered polar coordinates of peripheral pixels to Cartesian coordinates.

[0115] The system calculates the angle of the vector that intersects both the lumen center and the pixel location, per block 625. The calculated angle can be relative to a common axis, and the system can also check to account for wrapping around the common axis, and adjust the angle measurement accordingly by adding or subtracting 360 degrees, if necessary.

[0116] The system compares the calculated angle to the current minimum and maximum angles, per block 630. Initially, the current minimum and maximum angles may be set to the value of the angle formed by a vector passing through the first calculated pixel location, per block 625. The system updates the minimum angle and the minimum endpoint if the calculated angle is less than the minimum angle, per block 635. The minimum endpoint tracks the pixel location of the pixel that corresponds to the current minimum angle. Similarly, the system updates the maximum angle and the maximum endpoint if the calculated angle is greater than the current maximum angle, per block 640.

[0117] Returning to block 615, if the system determines that there are no more peripheral pixels to process ("NO"), the system calculates the coverage of the lumen-centered calcium arc using the current maximum and minimum endpoints, according to block 645. For example, the system may calculate the coverage angle as described herein with reference to Figure 3. The endpoints border the identified calcium region, for example, as shown in Figure 5B by endpoint 503A.

[0118] The system determines an arc point of the lumen-centered calcium arc for display, per block 650. The arc point is used to identify a display angle that corresponds to the coverage angle calculated, for example, per block 645. For example, the arc point may be arc point 504A, which represents the intersection between a line passing through the lumen center and a boundary of the display viewport. Calculating the arc point may be useful to better display the lumen-centered calcium arc on a display having a viewport set to a particular field of view, as described herein.

[0119] The arc points can be calculated by identifying the points where a line through the center of the lumen and the endpoint intersects with a circular display viewport when the image frame is displayed. For example, the system can be configured to represent the circumference of the display viewport as a circle in Cartesian coordinates, e.g., relative to the center of the device depicted in the image frame. The system can be configured to calculate where the two lines that intersect with the center of the lumen also intersect with the circumference of the display port itself. As an example, the system can fit the lines and circumference of the viewport to standard equations in Cartesian space (e.g., y=mx+b for lines and x for the circle formed by the circumference of the display viewport). 2 +y 2 =r 2 ) and solve for the intersections of the line equations with the circle equation. Each line intersects the circumference at exactly two points.

[0120] The system may identify the set of points closest to the calcium region corresponding to the coverage angle calculated, for example, according to block 645, and select those points as the arc points of the viewing angle. The other two points may be discarded, or in some examples, the system may use the coverage angle to identify which of the two sets of points are the arc points of the viewing angle. The system may calculate a viewing angle formed by the center of the lumen and the two arc points. The system may be further configured to display the arc formed by the viewing angle. The system may be configured to repeat calculating the viewing angle as described with respect to block 650, for example, in response to an input received on a display device that displays the image frame. In some examples, if the system receives an input indicating that the FOV value has changed, such as in response to a user input, the system may responsively automatically recalculate the viewing angle as described herein and display the updated arc according to the viewing angle.

[0121] The system outputs the lumen-centered calcium arc according to block 655. The system can display the lumen-centered calcium arc as part of the data displayed through the device's display viewport. The lumen-centered calcium arc can be shown along the radius of the display viewport using arc points. Using arc points and their corresponding coverage angles relative to the lumen center instead of endpoints can help account for variations in the field of view of the display viewport. The coverage angle used to display the lumen-centered calcium arc on the display is based on the display angle defined by the arc points and may differ from the coverage angle based on the endpoints sent downstream for further processing, for example as part of calcium scoring.

[0122] In some examples, the system checks for calcium regions along the top and bottom of the image frame and combines the representation of the region as a single region with the corresponding lumen center arc.

[0123] 7 shows a calcium mask 700 spanning the top and bottom of an image frame. Also shown are calcium identifiers 705A-705C. As described herein with reference to FIG. 6, the system combines arcs formed from identifiers spanning the top and bottom of the image frame that correspond to identifiers 705A and 705C in the calcium mask 700.

[0124] 8 is a flow chart of an example process 800 for identifying a lumen center in an image frame according to an embodiment of the present disclosure. A system, such as the system 200 of FIG.

[0125] As with process 300, process 800 is described as being performed on a single image frame, it being understood that the system may be configured to receive multiple image frames and repeat process 800 for each frame to generate a respective lumen center.

[0126] The system receives an image frame of the lumen, according to block 810 .

[0127] The system calculates the lumen contour of the lumen in the image frame according to block 820. The lumen contour may be a shape that defines the perimeter of the lumen. The system may approximate the lumen contour as a circle, an ellipse, or a polygon, by way of example.

[0128] The system calculates a spline estimate of the lumen contour according to block 830. The spline estimate is a function of one or more polynomials that approximates the shape of the lumen contour. The system can apply any of a variety of different techniques to estimate the spline of the lumen contour. For example, the spline can be interpolated or approximated from multiple points of the lumen contour. In some examples, the system calculates the lumen contour and the spline estimate together. The system can generate a spline estimate expressed as a polar coordinate relative to the device center in the image frame, since the device center is available as a reference point in every image frame.

[0129] The system estimates the lumen center as the centroid from multiple samples of the spline estimate, according to block 840. Each sample can be a point on the spline estimate. i ,y i ) may be the coordinate of the i-th sample. If the samples are not already represented as Cartesian coordinates, the system may first convert each sample as a Cartesian coordinate. The average for each dimension is calculated over the number of samples for each dimension, as shown in Equation 4 and Equation 5.

number

[0130] Equations [4] and [5] are x-dimensional

number

number

[0131] Next, the second and third moments are calculated using Equations 6-12 as shown below.

number

[0132] The system can then calculate the coordinates of the estimated lumen centroid calculated from the sample, and the second and third moments calculated from Equations 6 to 12, according to Equations 13 to 16 below.

number

[0133] The system outputs the centroid as the lumen center according to block 850. This lumen center can be used to generate calcium arcs of the lumen centers of calcium regions in the image frames, as described herein with reference to Figure 3. The lumen centers of the sequence of frames can be further smoothed, as described herein with reference to Figure 9.

[0134] 9 is a flowchart of an exemplary process 900 for smoothing the center of a lumen for a sequence of image frames according to an embodiment of the present disclosure. A system such as the system 200 of FIG.

[0135] The system receives coordinates defining a center of the lumen of the sequence of image frames, according to block 910. As described herein with reference to FIG. 8, the system can calculate the lumen center from the centroid of each image frame of the sequence of frames.

[0136] The system generates a lumen center array according to block 920. The lumen center array may be symmetric about the lumen center of the intermediate image frames. For example, if the image sequence includes five frames, the lumen center of frame 3 is the middle value of the lumen center array.

[0137] The system detects and filters out image frames depicting side branches to the imaged lumen and replaces the respective lumen center of each filtered image frame with a respective alternate lumen center, according to block 930. In doing so, the system can mitigate interference in smoothing the lumen centers that may be caused by additional information from side branches depicted in the image frames. As an example, the alternate lumen centers of the image frames depicting side branches can be linearly interpolated using the positions of the lumen centers in adjacent image frames in the sequence.

[0138] The system may pad the lumen center array with zeros. Padding the lumen center array is performed prior to applying a low pass filter of one or more filter coefficients in accordance with block 940. The values ​​of the filter coefficients may be symmetric about the intermediate frame lumen center and may vary proportionally with the relative frame index to the intermediate image frame. The low pass filter modifies the filtered values ​​in accordance with the filter coefficients to globally smooth the distribution of values.

[0139] For example, a filter for a lumen-centered array of nine image frame sequences may have the following coefficients according to Table 1:

[0140] [Table 1]

[0141] For example, lumen centers in the array that are 2 locations away from the lumen center at index 0 are filtered with a filter coefficient (or weight) of 4698. As another example, lumen centers in the array that are 4 locations away from the intermediate image frame are filtered with a filter coefficient of -60.

[0142] 10 is a chart 1000 illustrating the relationship between relative frame index 1005 and filter weights 1010. The y-axis represents filter weights 1010 and the x-axis represents relative frame index 1005 relative to the lumen center of the intermediate image frame. Note that the chart 1000 is symmetrical with respect to the intermediate image frame. Also note that the filter coefficient values ​​decrease gradually for image frames further away from the intermediate image frame, reflecting the gradual reduction of lumen center smoothing from the intermediate image frame.

[0143] 9, as part of applying the low pass filter according to block 940, the system may apply the filter separately to each dimension of the lumen center, for example along each Cartesian coordinate dimension for each lumen center. In some embodiments, the system applies the filter twice per dimension.

[0144] The system normalizes the smoothed lumen center by the sum of the filter coefficients, according to block 950. The normalization serves to produce a zero DC gain resulting from the smoothing.

[0145] The system outputs the smoothed lumen center according to block 960. As described herein with reference to Figures 2 and 3, the system may be configured to generate calcium arcs for the lumen center of both the input image frame and the smoothed lumen center.

[0146] Aspects of the present disclosure may be implemented as one or more computer programs in a digital circuit, in a computer-readable storage medium, or as a combination of one or more of the above. The computer-readable storage medium may be non-transitory, for example executable by a cloud computing platform, as one or more instructions stored on a tangible storage device.

[0147] In this specification, the phrase "configured to" is used in different contexts relating to a computer system, hardware, or part of a computer program, engine, or module. When a system is said to be configured to perform one or more operations, this means that the system has appropriate software, firmware, and / or hardware installed on the system that, during operation, causes the system to perform one or more operations. When a piece of hardware is said to be configured to perform one or more operations, this means that the hardware includes one or more circuits that, during operation, receive inputs and generate outputs corresponding to the one or more operations according to the inputs. When a computer program, engine, or module is said to be configured to perform one or more operations, this means that the computer program includes one or more program instructions that, when executed by one or more computers, cause the one or more computers to perform one or more operations.

[0148] Although the operations illustrated in the figures and recited in the claims are shown in a particular order, it is understood that the operations may be performed in an order different from that shown, and that some operations may be omitted, performed more than once, and / or performed in parallel with other operations. Furthermore, the separation of different system components configured to perform different operations should not be understood as requiring the components to be separated. The described components, modules, programs, and engines may be integrated together as a single system or may be part of multiple systems.

[0149] Unless otherwise indicated, the alternative examples described above are not mutually exclusive and can be implemented in various combinations to achieve their own advantages. These features and other variations and combinations described above can be utilized without departing from the subject matter defined by the claims, and therefore the description of the above-mentioned embodiments should be construed as illustrative, rather than limiting, the subject matter defined by the claims. In addition, the provision of examples described herein and clauses such as "such as," "including," and the like should not be construed as limiting the subject matter of the claims to specific examples, but rather, such examples are intended to illustrate only one of many possible embodiments. Furthermore, the same reference numbers in various figures can identify the same or similar elements.

Claims

1. receiving, by one or more processors, an image frame and an identification of a plaque region within the image frame, the image frame being captured by an imaging device while the imaging device is within a lumen depicted in the image frame; the one or more processors identifying a lumen center of the lumen depicted in the image frame; generating, by the one or more processors, an arc of a lumen center using at least the lumen center and having a coverage angle centered about the lumen center; The method comprising:

2. The method of claim 1 , wherein the identification of the plaque region comprises a mask, a region of pixels in the mask corresponding to the plaque region depicted in the image frame.

3. 2. The method of claim 1, further comprising identifying an endpoint of the plaque region, a line tangent to the plaque region passing through the endpoint and the lumen center, the line tangent to the plaque region defining the coverage angle of an arc of the lumen center.

4. the identification of the plaque region includes a mask, a region of pixels in the mask corresponding to the plaque region in the image frame; the identification information includes data defining a location of the pixel in the mask corresponding to the plaque region, expressed as a polar coordinate relative to a device center of the imaging device while the imaging device was within the lumen; The method of claim 1 , further comprising converting the pixel location from polar coordinates relative to the device center to polar coordinates relative to the lumen center.

5. The step of identifying the lumen center includes: calculating a lumen contour of the lumen; generating a spline estimate of the lumen contour; identifying the lumen center as an estimate of the center of gravity from a plurality of samples of the spline estimate; The method of claim 1 , comprising:

6. the image frame is a first image frame and is one of a plurality of image frames in a sequence; the lumen center is a first lumen center; identifying a respective lumen center for each of the plurality of image frames in the sequence; for each of the plurality of image frames including the first image frame and the first lumen center, applying a low pass smoothing filter to the respective lumen center; generating a second lumen center arc having a coverage angle centered on the smoothed lumen center using at least the lumen center after applying the low pass smoothing filter and an end point of a device center arc; The method of claim 1 further comprising:

7. The step of applying a low-pass smoothing filter comprises:

7. The method of claim 6, further comprising applying the low pass smoothing filter on an array of lumen centers of the plurality of image frames that is symmetric about the lumen center of the image frame in the middle of the sequence, the filter further comprising one or more filter coefficients applied to the first image frame, the one or more filter coefficients depending at least on a relative frame index between the image frame in the middle of the sequence and the first image frame.

8. The step of identifying a respective lumen center for each of the plurality of image frames in the sequence comprises: identifying one or more image frames depicting each side branch from the lumen; for each of the one or more image frames, interpolating a respective lumen center from lumen centers of adjacent image frames in the sequence that do not depict the respective side branch; The method of claim 6 further comprising:

9. 2. The method of claim 1, further comprising generating, by the one or more processors, one or more calcium scoring metric values ​​corresponding to the plaque region using, at least in part, the coverage angle of the lumen center arc.

10. the plaque region is a first plaque region of a plurality of plaque regions in the image frame; The method of claim 1 , further comprising generating, for each of the plurality of plaque regions, a respective lumen-centered arc having a respective coverage angle about the lumen center.

11. A system comprising one or more processors for executing a method according to any one of claims 1 to 10.

Images