Processing apparatus, oct apparatus, and program

Abstract

The change in the position of the blood vessel can be suitably grasped.SOLUTION: The processing device includes an image acquisition unit that acquires a first image obtained by imaging in a first time period and a second image obtained by imaging in a time period different from the first time period among images obtained by optical coherence tomography angiography on the same part of the same subject, a first point setting unit that sets a plurality of first points indicating any part of the subject in the first image, and a second point setting unit that sets a plurality of second points indicating any part of the subject in the second image. A second point specifying unit configured to specify, for each of the first points, a second point at which a part of the subject indicated by the first point is indicated in the second image, a change amount calculation unit configured to calculate, for each of the first points, an amount of change from a position of the first point to a position of the second point indicating the same part as the first point, and an output unit configured to output a result of the calculation.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a processing device, an OCT device, and a program. [Background technology]

[0002] The capillary plexus in the macula of the retina, which corresponds to the center of the visual field, is closely related to visual function and is extremely important in predicting the prognosis of visual function. Because it is difficult to detect subtle changes in the retina and retinal blood vessels with the naked eye, retinal vascular plexuses are observed using a technique called optical coherence tomography angiography (OCTA), which can non-invasively depict high-resolution three-dimensional images. In addition, a technique has been disclosed for analyzing the relationship with retinal disease based on the position of retinal blood vessels at a certain point in time (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2023 / 211483 Summary of the Invention [Problem to be solved by the invention]

[0004] Generally, changes in the position of each capillary are measured by the examiner (measurer) manually identifying characteristic points such as the branching points of the capillaries. However, the results of manual measurements vary in the number of measurements and measurement range from examiner to examiner, making it difficult to eliminate individual differences and arbitrariness between examiners. Furthermore, manually measuring changes in the position of capillaries for a large number of subjects is a time-consuming and difficult process.

[0005] The present invention has been made in view of the above-mentioned points, and has an object to provide a technique that can suitably grasp changes in the position of blood vessels. [Means for solving the problem]

[0006] One aspect of the present invention is a processing device comprising: an image acquisition unit that acquires, from among images obtained by optical coherence tomography angiography (OCTA) of the same location of the same subject, a first image obtained by taking the image at a first time period and a second image obtained by taking the image at a time period different from the first time period; a first point setting unit that sets multiple first points in the first image, each indicating a part of the subject; a second point identification unit that identifies, for each of the first points, a second point at which the part of the subject indicated by the first point is shown in the second image, based on similar shapes between the subject shown in the first image and the subject shown in the second image; a change amount calculation unit that calculates, for each of the first points, the amount of change from the position of the first point to the position of the second point indicating the same part as the first point; and an output unit that outputs the calculated results.

[0007] In one aspect of the present invention, the amount of change is a vector indicating a direction from the first point to the second point and a distance from the first point to the second point.

[0008] In one aspect of the present invention, each of the first points is set for each area in which one or more pixels are regularly integrated.

[0009] In addition, in one aspect of the present invention, the processing device further includes a similarity calculation unit that calculates the similarity between the map and the template by comparing a map showing the position and the amount of change in the first image for each of the first points with a template that is defined for each disease and shows the amount of change at multiple positions, and the output unit outputs information based on the similarity.

[0010] Another aspect of the present invention is an OCT device including the above-described processing device.

[0011] Another aspect of the present invention is a program that causes a computer to execute the following steps: an image acquisition step of acquiring, from among images obtained by optical coherence tomography angiography (OCTA) of the same location of the same subject, a first image obtained by taking the image at a first time period and a second image obtained by taking the image at a time period different from the first time period; a first point setting step of setting multiple first points in the first image, each indicating a part of the subject; a second point identification step of identifying, for each of the first points, a second point at which the part of the subject indicated by the first point is shown in the second image, based on similar shapes between the subject shown in the first image and the subject shown in the second image; a change amount calculation step of calculating, for each of the first points, the amount of change from the position of the first point to the position of the second point indicating the same part as the first point; and an output step of outputting the calculated results. [Effects of the Invention]

[0012] According to the present invention, changes in the position of blood vessels can be appropriately detected. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 2 is a diagram showing an example of a location where OCTA imaging is performed by the OCT device according to the present embodiment. [Figure 2] 1 is a block diagram illustrating an example of the functional configuration of an OCT device according to an embodiment of the present invention. [Figure 3] FIG. 2 is a block diagram illustrating an example of a functional configuration of a processing unit according to the present embodiment. [Figure 4] 4 is a schematic diagram for explaining an example of processing performed by a first point setting unit to a change amount calculation unit in the present embodiment. FIG. [Figure 5] FIG. 4 is a diagram for explaining an example of a first point map according to the present embodiment. [Figure 6] 10A and 10B are diagrams for explaining an example of calculating the amount of change in the present embodiment. [Figure 7] 10A and 10B are diagrams illustrating an example of a result calculated by a change amount calculation unit according to the embodiment. [Figure 8] FIG. 10 is a diagram illustrating an example of a template according to the present embodiment. [Figure 9] 10A and 10B are diagrams for explaining an example of processing performed by a similarity calculation unit according to the embodiment; [Figure 10] FIG. 10 is a diagram showing an example of a determination result when subjective classification is performed. [Figure 11] FIG. 1 is a diagram for explaining a problem in subjective classification. [Figure 12] FIG. 10 is a diagram comparing the judgment results of subjective classification and mathematical classification. [Figure 13] FIG. 10 is a diagram showing an example of retinal movement when the determination results differ between subjective classification and mathematical classification. [Figure 14] 10 is a diagram illustrating an example of a technique for bringing mathematical classification results closer to subjective classification results. [Figure 15] 10 is a flowchart illustrating an example of a processing flow of a processing unit according to the present embodiment. [Figure 16] FIG. 2 is a block diagram showing an example of the internal configuration of a processing unit according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] [Embodiment] To address the above-mentioned challenges, we propose a processing device that uses an image registration technique to comprehensively and mechanically acquire information (e.g., vectors, scalars) about changes in the position of retinal blood vessels for each pixel from two OCTA images taken at different times. Furthermore, by comparing the information about changes in the position of retinal blood vessels with a template that indicates expected changes in the position of retinal blood vessels for each disease (hereinafter sometimes referred to as a retinal movement pattern), the presence or absence of a given retinal movement pattern can be objectively evaluated. Because the above-described processing is performed mechanically, the influence of individual differences between examiners can be eliminated. Furthermore, the above-described processing allows the evaluation of the effectiveness of surgical treatment using pre- and post-operative OCTA images. Furthermore, the OCTA images are not limited to those taken before and after surgery. By performing the above-described processing using two OCTA images taken at different times, the progression and onset of a disease can be evaluated.

[0015] The following describes in detail preferred embodiments of a processing device, OCT device, and program for executing the above-described processes, with reference to the accompanying drawings. In the drawings, identical or similar parts are designated by identical or similar reference numerals. Note that the present embodiment is not limited to these embodiments and includes various modifications and improvements. In other words, the components described below include those that would be easily conceivable to a person skilled in the art and those that are substantially identical, and the components described below can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of components may be made in the present embodiment without departing from the spirit and scope of the present invention.

[0016] First, an OCT (optical coherence tomography and angiography) device according to an embodiment will be described with reference to FIGS.

[0017] FIG. 1 is a diagram showing an example of a location where an OCT device according to this embodiment performs OCTA imaging. In this embodiment, the OCT device 100 will be described using an example in which the capillary plexus in the macular region of a subject's retina is observed. The macular region is the central part of the retina, corresponding to the center of the visual field. Because photoreceptor cells are densely packed in the macular region, damage to the macular region can cause visual dysfunction, such as decreased vision and distorted vision. The position of photoreceptor cells changes with changes in the position of retinal blood vessels during disease progression or postoperative recovery, affecting the subject's visual function. Therefore, by observing changes in the position of retinal blood vessels, the examiner can predict the subject's prognosis for visual function. In this embodiment, the examiner grasps the changes in the position of retinal blood vessels using the OCT device 100. In the following description, a location refers to the part of the subject that is imaged by OCTA.

[0018] Although the above description illustrates an example in which the OCT device 100 according to the present embodiment is used to observe changes in the position of a subject's retinal blood vessels, the present embodiment is not limited to this example. For example, the examiner may use the OCT device 100 to observe changes in the position of blood vessels in the choroid (choroidal vessels) beneath the subject's retina. By observing and understanding changes in the position of the choroidal vessels, the examiner can understand the onset, progression, and treatment status of choroidal neovascularization, which occurs when new blood vessels grow from the choroidal vessels and push up against the retina. The OCT device 100 according to the present embodiment may also be used to observe changes in the position of various blood vessels.

[0019] 2 is a block diagram illustrating an example of the functional configuration of the OCT device 100 according to this embodiment. The OCT device 100 includes an imaging unit 110, an image supply unit 120, a processing unit 130, and a display unit 140.

[0020] The imaging unit 110 performs OCTA imaging during a predetermined time period. The imaging unit 110 repeatedly images the same cross section using OCTA imaging, and generates motion contrast data by detecting changes in the subject over time (such as changes in intensity or phase of the OCT signal) between the images. The imaging unit 110 may also generate three-dimensional motion contrast data by performing continuous OCTA imaging in the normal direction of the cross section.

[0021] In this embodiment, the imaging unit 110 performs OCTA imaging of the same location (e.g., the macula) of the same subject (hereinafter sometimes referred to as the subject) at different time periods. In the following description, any time period during which OCTA imaging is performed will be referred to as a first time period, and any time period different from the first time period will be referred to as a second time period. The second time period will be described as a time period after the first time period. The first and second time periods are, for example, pre-surgery and post-surgery. The first and second time periods are not limited to pre-surgery and post-surgery, but may be any time period. By regarding the first and second time periods as pre-surgery and post-surgery, the examiner can determine the therapeutic effect of surgery. By regarding the first and second time periods as any different time period other than pre-surgery and post-surgery, the examiner can determine the progression of a disease, whether or not a disease has developed, and whether or not surgery is necessary.

[0022] The image providing unit 120 acquires motion contrast data captured by OCTA imaging from the imaging unit 110. For example, the image providing unit 120 generates an OCTA image of a desired layer by projecting, from the three-dimensional motion contrast data, motion contrast data at any depth in the normal direction of each layer of the retina onto a two-dimensional plane. In the following description, the OCTA image obtained by OCTA imaging in a first time period is referred to as a first image IM1, and the OCTA image obtained by OCTA imaging in a second time period is referred to as a second image IM2.

[0023] The processing unit 130 may also be referred to as a processing device. The processing unit 130 generates information regarding changes in the positions of the retinal blood vessels based on the first image IM1 and the second image IM2, which were captured at different times. The processing unit 130 also compares the information regarding changes in the positions of the retinal blood vessels with a template indicating a retinal movement pattern to determine whether or not any retinal movement pattern exists. The processing unit 130 outputs the information obtained by processing to the display unit 140.

[0024] The display unit 140 displays the OCTA image captured by the imaging unit 110, the information obtained by the processing unit 130, and the like to the examiner. The display unit 140 may be, for example, a display provided in the OCT device 100. The display unit 140 may also be a monitor or the like connected to the OCT device 100. Based on the content displayed on the display unit 140, the examiner can predict the state of the retina, visual function, and the like of the subject.

[0025] FIG. 3 is a block diagram illustrating an example of the functional configuration of the processing unit 130 according to this embodiment. The processing unit 130 includes an image acquisition unit 131, a first point setting unit 132, a second point identification unit 133, a change amount calculation unit 134, a template storage unit 135, a similarity calculation unit 136, and an output unit 137. Each of these functional units is realized, for example, using a computer including a CPU (Central Processing Unit) and memory, and software. Each functional unit may also be realized using an electronic circuit, if necessary. Furthermore, each functional unit does not have to be included in a single device, and the processing unit 130 may be configured from multiple devices.

[0026] The image acquisition unit 131 acquires the first image IM1 and the second image IM2 from the image supply unit 120 or an OCTA image storage unit (not shown) in which the OCTA images output from the image supply unit 120 and the like are stored.

[0027] The processes performed by the first point setting unit 132, the second point specifying unit 133, and the change amount calculation unit 134 will be described with reference to FIGS.

[0028] FIG. 4 is a schematic diagram illustrating an example of processing performed by the first point setting unit 132 and the change amount calculation unit 134 in this embodiment. The first point setting unit 132 acquires a first image IM1. The first point setting unit 132 sets multiple first points P1 in a map (hereinafter, sometimes referred to as a first point map PM1), which is a matrix, image, or the like, having the same size as the first image IM1. The first points P1 are associated with any part of the subject shown in the first image IM1. The part does not necessarily have to be an index (reference) for manual identification, i.e., a distinctive part such as a branching point of a blood vessel, but may be any position within a blood vessel or avascular region that does not have a distinctive shape and therefore is difficult to use as a reference for identification. The first points P1 may indicate parts other than the distinctive parts of the subject, and therefore may be set for each region (hereinafter, sometimes referred to as an integrated region) in which one or more pixels are integrated. The shape of the integrated region is determined based on a user operation. The shape of the integrated regions is preferably square or rectangular so that pixels not included in any of the integrated regions do not occur between the integrated regions, i.e., so that the integrated regions are set continuously. Setting a first point P1 for each region where one pixel is integrated means that a first point P1 is set for each pixel in the first image IM1. Setting a first point P1 in a region where multiple pixels are integrated may mean, for example, setting a single first point P1 to represent the integrated region. The first point map PM1 may be set, for example, superimposed on the first image IM1, or may be set using a map different from that of the first image IM1.

[0029] In the example shown in Figure 4, the integrated region is a square (including 1 pixel x 1 pixel), so the first point P1 is regularly set in a first direction within the first point map PM1 (e.g., the horizontal direction within the first point map PM1) and a second direction different from the first direction (e.g., the vertical direction within the first point map PM1). Furthermore, the first point map PM1 includes, as a component, grid lines L that pass through the first point P1 and are drawn in the first or second direction. Because the first point map PM1 includes not only the first point P1 but also the grid lines L as a component, the examiner can relatively easily determine how the positions of the retinal blood vessels have changed from the way the grid lines L are deformed.

[0030] The second point identification unit 133 uses a non-rigid image registration technique to identify the position (second point P2 described later) where the part indicated by the first point P1 in the first image IM1 is shown in the second image IM2 from the correspondence relationship between the first image IM1 and the second image IM2 (a relationship based on similarities in shape, etc.).

[0031] For example, the second point identification unit 133 identifies the correspondence between the first image IM1 and the second image IM2 (hereinafter also referred to as mapping or geometric transformation) by transforming the first image IM1 into the second image IM2 based on the similar shapes of the subject shown in the first image IM1 and the subject shown in the second image IM2. The second point identification unit 133 generates the second point map PM2 by transforming the first point map PM1 in the same way as when the first image IM1 is transformed into the second image IM2 based on the identified correspondence. The transformation of the first point map PM1 changes the position of each first point P1 in the first point map PM1. Hereinafter, each first point P1 in the transformed first point map PM1 (i.e., the second point map PM2) will be referred to as a second point P2. Because the first point map PM1 is transformed in the same way as the first image IM1, the part of the subject indicated by the first point P1 and the part of the subject indicated by the second point P2 are substantially the same. The second point identification unit 133 can identify the position where the part indicated by each first point P1 in the first image IM1 is shown in the second image IM2 by deforming the first point map PM1 to identify the second point P2 corresponding to each first point P1.

[0032] Furthermore, the second point identification unit 133 may, for example, move the position of the first point P1 in the first point map PM1 (first image IM1) and deform the first image IM1 so that the location indicated by the first point P1 does not change. In this case, the second point identification unit 133 moves the first point P1 and deforms the first image IM1 so that the mutual information indicating the degree of match between the deformed first image IM1 and the second image IM2 increases. When the second point identification unit 133 deforms the first image IM1 into the second image IM2, the position of each first point P1 in the first point map PM1 moves to the position of the second point P2 in the second point map PM2. In other words, the second point identification unit 133 can identify the position of each second point P2 by moving the first point P1 so that the first image IM1 resembles the second image IM2. When the first point P1 is moved, each pixel in the integrated region is moved according to the direction and distance of movement of the first point P1. Although an example of the processing performed by the second point identification unit 133 has been described above, the present embodiment is not necessarily limited to this example.

[0033] FIG. 5 is a diagram illustrating an example of a first point map PM1 according to this embodiment. The first point map PM1 shown in FIG. 4 illustrates an example in which first points P1 are set at equal intervals in the vertical and horizontal directions. However, this embodiment is not limited to this example. The first point map PM1 in FIG. 5(A) is similar to the first point map PM1 shown in FIG. 4 in that the first points P1 are set regularly in a first direction and a second direction perpendicular to the first direction. The first point map PM1 in FIG. 5(A) differs from the first point map PM1 shown in FIG. 4 in that the first direction is a direction (diagonal direction) different from the vertical and horizontal directions. That is, the first direction and the second direction may be set arbitrarily. Furthermore, the first direction and the second direction do not necessarily need to be perpendicular to each other.

[0034] The first point map PM1 in Fig. 5(B) is similar to the first point map PM1 shown in Fig. 4 in that the points are set regularly in a first direction (e.g., the vertical direction) and regularly in a second direction (e.g., the horizontal direction). The first point map PM1 in Fig. 5(B) differs from the first point map PM1 shown in Fig. 4 in that the integrated region is rectangular.

[0035] The first point map PM1 in Fig. 5(C) is similar to the first point map PM1 shown in Fig. 4 in that first points P1 are set regularly in a first direction (vertical direction) and a second direction (horizontal direction) perpendicular to the first direction. The first point map PM1 in Fig. 5(C) differs from the first point map PM1 shown in Fig. 4 in that it does not have, as a component, grid lines L passing through each first point map PM1.

[0036] Next, an example of processing performed by the change amount calculation unit 134 will be described. FIG. 6 is a diagram illustrating an example of calculating the change amount V in this embodiment. FIG. 6(A) is a diagram illustrating the position of a first point P1 in a first point map PM1. FIG. 6(B) is a diagram illustrating the position of a second point P2 in a second point map PM2, which indicates the same part as the first point P1 in FIG. 6(A). The second point identification unit 133 calculates, for each first point P1, the change amount V from the position of the first point P1 to the position of the second point P2, which indicates the same part as the first point P1. That is, the second point identification unit 133 identifies, for each first point P1, the change amount V, which indicates changes in each part of the subject, such as retinal blood vessels, that occurred between the time when the first image IM1 was captured and the time when the second image IM2 was captured. The change amount V may be the distance (scalar) between the first point P1 and the second point P2, or may be a vector having the distance and a direction from the first point P1 to the second point P2. FIG. 6C shows an example in which a vector from the first point P1 to the second point P2 is calculated as the amount of change V.

[0037] Fig. 7 is a diagram showing an example of the result calculated by the change amount calculation unit 134 according to this embodiment. In Fig. 7, the vertical direction represents the SUPERIOR (above) and INFERIOR (below) of the subject, and the horizontal direction represents the TEMPORAL (ear side) and NASAL (nose side) of the subject.

[0038] Figure 7(A) shows an example of a heat map showing the results of calculating the amount of change V using a scalar. By calculating the amount of change V using a scalar, it is possible to easily grasp the positions where there is a large change. Figure 7(A), in which the amount of change V is calculated using a scalar, shows that there is a large change in the position of the retinal blood vessels slightly below the center of the image.

[0039] Figure 7(B) shows an example of the calculation results of the change amount V using a vector. By calculating the change amount V as a vector, the magnitude of the change and the movement direction of each retinal blood vessel can be easily grasped. Figure 7(B), in which the change amount V is calculated as a vector, allows for the identification of retinal movement, including movement toward the movement center and movement perpendicular to that direction, away from the movement center. The movement center is the point at the center of the characteristic movement. This retinal movement is a characteristic movement pattern often seen before and after surgery for epiretinal membrane (premacular membrane) and is sometimes referred to as diamond movement. Figure 7(B) allows for the easy identification of characteristic retinal movement patterns identified for each disease. Note that Figure 7(B) also shows the vectors of each first point P1 set for each square integrated area formed by integrating multiple pixels to ensure visibility.

[0040] The template storage unit 135 stores a template T indicating changes in the position of blood vessels determined for each disease. FIG. 8 is a diagram showing an example of the template T according to this embodiment. FIG. 8(A) is a template T showing a retinal movement pattern (diamond-shaped movement) observed before and after surgery for macular membranes and the like. FIG. 8(A) shows a retinal movement pattern including movement toward the center of movement and movement perpendicular to the center of movement, away from the center of movement. In FIG. 8(A), the vector of the first point P1 close to the center of movement is small, and the vector of the first point P1 far from the center of movement is large. When a patient suffers from a premacular membrane, wrinkles form in the macular region of the retina. When wrinkles form, the retina moves significantly toward the center of movement, and some of the retina pushed out by the wrinkles moves in a direction perpendicular to the direction of movement toward the center of movement. When a premacular membrane is treated, movement occurs in the opposite direction to that during disease progression, and a diamond-shaped movement in the opposite direction is observed.

[0041] Figure 8(B) is a template T showing the retinal movement pattern (horizontal movement) observed before and after surgery for retinal detachment, severe macular epithelium, etc. Figure 8(B) shows a retinal movement pattern in which each first point P1 moves in the same direction and by the same amount. Before and after retinal detachment surgery, when the retina reattaches, a wide area of ​​the retina moves closer to its original position, and parallel movement is observed in part of the retina. Furthermore, in the case of severe macular epithelium, the retina is strongly pulled by the macular epithelium, and parallel movement is observed in part of the retina.

[0042] Figure 8(C) is a template T showing retinal movement patterns (suction) observed before and after surgery for macular holes and the like. Figure 8(C) shows a retinal movement pattern in which each first point P1 moves toward the center of movement. In Figure 8(C), the vector of the first point P1 close to the center of movement is small, and the vector of the first point P1 far from the center of movement is large. When a macular hole occurs, the retina around the hole moves toward the hole as the hole closes, and suction is observed in the retina around the hole.

[0043] 8, the template T stored in the template storage unit 135 is shown as an example of a 9×9 square. However, this embodiment is not necessarily limited to this example. The template T may be any shape, such as a rectangle or a circle.

[0044] The similarity calculation unit 136 acquires a vector map VM indicating the vector change V calculated by the change amount calculation unit 134 for each first point P1. The similarity calculation unit 136 also acquires a template T from the template storage unit 135. The similarity calculation unit 136 compares the vector map VM with the template T to calculate the similarity of the positional changes of the retinal blood vessels between the vector map VM and the template T. That is, the similarity calculation unit 136 calculates the similarity between the retinal movement of the subject and a template T of retinal movement determined for each disease. The similarity calculation unit 136 may output the similarity calculated by, for example, comparing one template determined based on the examiner's operation with the vector map VM. The similarity calculation unit 136 may also compare the calculated similarity with a predetermined threshold to output information indicating whether the retina of the subject (vector map VM) exhibits retinal movement similar to that of the template T. This allows the examiner to grasp the retinal movement due to surgery or the passage of time and, based on this, easily evaluate changes in the subject's visual function. In addition, in the case of the same disease, the retinal movement pattern during disease progression and the retinal movement pattern during surgical treatment of the disease are opposite movements, so in some cases it may be possible to use the same template T. If the retinal movement pattern during disease progression and the retinal movement pattern during surgical treatment of the disease are not opposite movements, different templates T may be used depending on whether it is desired to evaluate disease progression or surgical treatment of the disease. In other words, the template T may be determined for each evaluation purpose. This allows the similarity to be calculated using the template T that best suits the evaluation purpose.

[0045] Furthermore, when the similarity calculation unit 136 acquires multiple templates determined based on the operation of the examiner or all of the templates T stored in the template storage unit 135, it may output the name of the disease indicated by the template T having the highest similarity to the vector map VM. Furthermore, the similarity calculation unit 136 may output the name of the template T identified according to the similarity together with the similarity of the template T.

[0046] FIG. 9 is a diagram illustrating an example of processing performed by the similarity calculation unit 136 according to this embodiment. In FIG. 9, the similarity calculation unit 136 uses a template T that exhibits diamond-shaped movement. The similarity calculation unit 136 calculates the inner product of the template T and the vector map VM multiple times by performing rigid transformation (rotation, translation, etc.) on the template T. In this case, the similarity calculation unit 136 determines the highest value of the inner product calculated multiple times as the similarity (ALGORITHM FITTING SCORE) between the template T and the vector map VM. Furthermore, if the similarity is equal to or greater than a threshold, the similarity calculation unit 136 determines that the vector map VM has undergone the same retinal movement as the template T. In the following description, the determination performed by the similarity calculation unit 136 may be referred to as mathematical classification. Since the ALGORITHM FITTING SCORE is calculated by the inner product, the value varies depending on the size of the vector of the template T.

[0047] Figure 10 shows an example of the results of subjective classification. In subjective classification, the examiner visually determines the presence or absence of diamond-shaped movement, horizontal movement, suction, and other retinal movement patterns from OCTA images of retinal blood vessels. In Figure 10, the examiner visually inspects the vector maps VM of 37 OCTA images and determines that diamond-shaped movement was present in 17 OCTA images (P1 to P17) (POSITIVE) and that diamond-shaped movement was absent in 20 OCTA images (N1 to N20) (NEGATIVE). The areas in P1 to P17 where diamond-shaped movement was determined to have occurred are indicated by circles.

[0048] Figure 11 is a diagram illustrating the challenges of subjective classification. Of the 37 OCTA images shown in Figure 10, Figure 11 shows P4 and P10, which were determined to have diamond-shaped movement, and N12 and N16, which were determined not to have diamond-shaped movement. P4, shown in Figure 11(A), has multiple vectors and a layout consistent with the diamond-shaped movement template T, making it easy to determine that diamond-shaped movement has occurred. Meanwhile, N12, shown in Figure 11(C), has multiple vectors and a layout significantly different from the diamond-shaped movement template T, making it easy to determine that diamond-shaped movement has not occurred.

[0049] In contrast, in P10 shown in FIG. 11(B), the multiple vectors and their arrangement are slightly distorted from those of the diamond-shaped movement template T. Furthermore, in N16 shown in FIG. 11(D), the vectors outside the dotted rectangle have the same orientation and arrangement as those of the diamond-shaped movement template T, but the orientation of the vectors inside the dotted rectangle differs from those of the diamond-shaped movement template T. Therefore, depending on the skill of the examiner, it may be difficult to accurately determine whether diamond-shaped movement exists for P10 or for N16, unless the examiner is an experienced examiner. Furthermore, because retinal movement such as diamond-shaped movement does not necessarily occur in the center of the OCTA image, the examiner must first identify a position where movement similar to that of template T is likely to occur, and then determine whether movement similar to that of template T has occurred at the identified position. This may take time even for an experienced examiner to make this determination.

[0050] According to the method for calculating similarity in this embodiment, mathematical classification is performed based on a fixed standard, preventing variations in judgment results due to individual differences between examiners and enabling accurate judgment of the presence or absence of a retinal movement pattern. Furthermore, since the method in this embodiment calculates similarity using a predetermined calculation called an inner product, it is possible to quickly judge the presence or absence of a retinal movement pattern for multiple OCTA images.

[0051] FIG. 12 is a diagram comparing the judgment results of subjective classification with those of mathematical classification. In FIG. 12, when an examiner checks an OCTA image of retinal blood vessels and judges that diamond-shaped movement is present (SUBJECTIVE CLASSIFICATION=POSITIVE), the judgment results are almost identical to the judgment results of mathematical classification. Therefore, it can be seen that the method of calculating similarity in this embodiment can judge the presence or absence of retinal movement indicated by template T to the same degree as subjective classification. Note that in FIG. 12, some of the classification results based on mathematical classification differ from the classification results based on subjective classification. With reference to FIGS. 13 and 14, the reasons for the difference in judgment results between subjective classification and mathematical classification and an example of a method for bringing mathematical classification closer to subjective classification will be explained.

[0052] FIG. 13 shows an example of retinal movement when the results of subjective classification and mathematical classification differ. In mathematical classification, similarity is calculated using all data for each pixel. To ensure vector visibility, FIG. 13 shows the vectors of each first point P1 set for each square integrated region where multiple pixels are integrated. In other words, FIG. 13 is a diagram that ensures visibility for explaining mathematical classification, and also serves as a vector map VM for subjective classification. An example of retinal movement that deviates slightly from the diamond-shaped retinal movement pattern is shown in Figure 13. Specifically, in the retinal movement shown in Figure 13, the line (first line L1) extending a vector from the center of the retinal movement toward one diagonal corner of the diamond (lower left) does not match the line (second line L2) extending a vector from the center of the retinal movement toward the other diagonal corner of the diamond (upper right).

[0053] In the case of subjective classification, the presence or absence of a retinal movement pattern is determined based on whether the orientation and arrangement of each vector in the vector map VM are roughly similar to the orientation and arrangement of each vector in the template T. Therefore, even if the first line L1 and the second line L2 do not match, the examiner may determine that the retinal movement pattern shown in the template T has been confirmed for the retinal movement shown in Figure 13 because the vector map VM and the template T are similar.

[0054] In contrast, in the case of mathematical classification, the retinal movement pattern is determined based on whether the orientation and arrangement of vectors in the vector map VM match the orientation and arrangement of each vector in the template T (the inner product becomes larger when the orientations match). Therefore, if the first line and the second line do not match, the similarity calculation unit 136 may calculate a low similarity, and determine that the retinal movement pattern shown in the template T could not be confirmed for the retinal movement shown in FIG. 13.

[0055] FIG. 14 is a diagram illustrating an example of a method for bringing mathematical classification results closer to subjective classification results in such cases. Note that FIG. 14 also shows vectors for each first point P1 set for each square integrated region formed by integrating multiple pixels to ensure vector visibility. In FIG. 14, in the mathematical classification according to the embodiment, the first point P1 is set for each pixel. Also, in FIG. 14, the first line L1 and the second line L2 do not coincide (as in FIG. 13). In this case, rather than simply calculating the similarity from a mathematical inner product, the similarity can be increased by finding a line (third line L3) including a vector located nearby on the first line L1 and a vector located nearby on the second line L2 in the opposite direction to the vector. In this way, the similarity calculation unit 136 sets the first point P1 for each pixel and calculates the similarity by correcting the distortion of the vector map VM indicating the subject's retinal movement according to a typical pattern of the template T. This allows the similarity calculation unit 136 to make a determination closer to the subjective classification.

[0056] In the above-described embodiment, an example is shown in which the similarity calculation unit 136 calculates the similarity using the amount of change V, which is a vector. However, this embodiment is not limited to this example, and the similarity may be calculated using the amount of change V, which is a scalar. In this case, the template T may be determined by the magnitude of movement of the first point P1 and the position of the first point P1.

[0057] The output unit 137 outputs information (e.g., a vector map VM) indicating the change amount V calculated by the change amount calculation unit 134 and information regarding the similarity calculated or determined by the similarity calculation unit 136 to the display unit 140. The subject can easily understand changes in the subject's visual function based on the content displayed on the display unit 140. Furthermore, when the similarity calculation unit 136 calculates a similarity that may determine that a retinal movement pattern can be confirmed in one of the subjective classification and the mathematical classification, but may determine that a retinal movement pattern cannot be confirmed in the other, that is, a similarity value within a predetermined range that is neither large nor small, the output unit 137 may output a warning indicating that the accuracy of the determination result of the similarity calculation unit 136 is insufficient. The predetermined range of values ​​that is neither large nor small may be, for example, an initial setting range stored in advance in a storage unit (not shown) in the OCT device 100, or may be a range that is preset by the examiner before calculation. By outputting the warning content from the output unit 137, the examiner can realize that the retinal movement of the subject is slightly deviated from the retinal movement pattern and therefore needs to be careful when determining whether or not there is a retinal movement pattern, thereby reducing the possibility of making an erroneous judgment. Note that the warning content output by the output unit 137 may be, for example, a message urging the examiner to visually check, such as "The accuracy of the judgment result is insufficient. Please check visually."

[0058] In the above-described embodiment, an example is shown in which the processing unit 130 performs processing using a two-dimensional OCTA image. However, this embodiment is not limited to this example, and a three-dimensional OCTA image may also be used. By having the processing unit 130 perform processing using a three-dimensional OCTA image, the examiner can grasp changes in the position of blood vessels three-dimensionally. When the processing unit 130 performs processing using a three-dimensional OCTA image, the template T stored in the template storage unit 135 may be a three-dimensional figure.

[0059] In the above-described embodiment, the processing unit 130 is provided in the OCT device 100 as an example. However, the present embodiment is not limited to this example, and the processing unit 130 may be provided in various devices. For example, the processing unit 130 may be provided in an information processing device such as a smartphone, a tablet terminal, or a personal computer. Furthermore, the processing unit 130 may be a program that executes the above-described processing in the information processing device.

[0060] 15 is a flowchart for explaining an example of the processing flow of the processing unit 130 according to this embodiment. The processing flow of the processing unit 130 will be specifically explained with reference to the same figure.

[0061] (Step S101) The processing unit 130 acquires an OCTA image (first image IM1) taken in a first time period and an OCTA image (second image IM2) taken in a second time period from the imaging unit 110, the image supply unit 120, or an OCTA image storage unit not shown.

[0062] (Step S102) The processing unit 130 generates a first point map PM1 by setting a plurality of first points P1 in the first image IM1.

[0063] (Step S103) The processing unit 130 identifies information indicating a mapping from the first image IM1 to the second image IM2 as a correspondence relationship between the first image IM1 and the second image IM2. The mapping may be identified based on a part that is common to the subject shown in the first image IM1 and the subject shown in the second image IM2 and has a characteristic shape. Alternatively, the mapping may be identified by deforming the first image IM1 so as to maximize the mutual information with the second image IM2.

[0064] (Step S104) The processing unit 130 identifies the position of the second point P2 indicating the same part as the first point P1 by moving each first point P1 in the first point map PM1 based on the mapping from the first image IM1 to the second image IM2.

[0065] (Step S105) The processing unit 130 generates a vector map VM by calculating, for each first point P1, a vector from the first point P1 to a second point P2 that indicates the same part as the first point P1.

[0066] (Step S106) The processing unit 130 calculates the similarity based on the inner product of the generated vector map VM and the template T stored in the template storage unit 135. Furthermore, the processing unit 130 may determine, based on the calculated similarity, whether or not the retinal movement pattern indicated by the template T has been confirmed in the subject's retina.

[0067] (Step S107) The processing unit 130 outputs to the display unit 140 information indicating at least one of the vector map VM, the similarity, and the presence or absence of a retinal movement pattern indicated by the template T.

[0068] [Summary of the embodiment] According to the above-described embodiment, the processing unit 130 performs optical coherence tomography (OCT) angiography on the same location of the same subject. The system includes an image acquisition unit 131 that acquires, among OCTA images obtained by OCTA (Optical Celestial Angiography (OCTA)), a first image IM1 obtained by capturing an image in a first time period and a second image IM2 obtained by capturing an image in a time period different from the first time period; a first point setting unit 132 that sets a plurality of first points P1 in the first image IM1, each of which indicates a region of the subject; a second point identification unit 133 that identifies, for each first point P1, a second point P2, which is a position in the second image IM2 where the region of the subject indicated by the first point P1 is located, based on similar shapes between the subject shown in the first image IM1 and the subject shown in the second image IM2; a change amount calculation unit 134 that calculates, for each first point P1, a change amount V from the position of the first point P1 to the position of the second point P2 indicating the same region as the first point P1; and an output unit 137 that outputs the calculated result. The processing unit 130 mechanically calculates the change in blood vessel position, making it easy to comprehensively identify changes in blood vessel position. The processing unit 130 comprehensively identifies changes in blood vessel position, allowing the examiner to grasp the retinal movement of the subject in more detail. Furthermore, because the processing unit 130 identifies retinal movement, the influence of individual differences and arbitrariness among examiners can be eliminated, compared to manually identifying retinal movement, and the retinal movement of a large number of subjects can be identified in a short time. This allows the examiner to easily grasp the retinal movement of a large number of subjects and easily analyze the retinal movement patterns for each disease. Furthermore, the processing unit 130 identifies retinal movement from two OCTA images, allowing the examiner to grasp the retinal movement of the subject and evaluate whether the disease is progressing and surgery is required, whether the disease has been cured by surgery, etc.

[0069] Furthermore, according to the above-described embodiment, the amount of change V is a vector indicating the direction from the first point P1 to the second point P2 and the distance from the first point P1 to the second point P2. The processing unit 130 calculates the vector from the first point P1 to the second point P2 and displays it as a vector map VM, thereby enabling the examiner to easily grasp the presence or absence of a retinal movement pattern confirmed for each disease.

[0070] Furthermore, according to the above-described embodiment, each first point P1 is set for each region where one or more pixels are regularly integrated. Specifically, if the first points P1 are not set regularly, the arrangement and position of vectors appearing on the vector map VM due to diamond-shaped movement may differ depending on the position where diamond-shaped movement is observed on the retina and the direction of wrinkles. By regularly setting the first points P1 in the vertical and horizontal directions, each vector located within the range where diamond-shaped movement is observed will have a consistent arrangement regardless of the position or angle of retinal wrinkles. This allows the examiner to more accurately analyze retinal movement patterns expected for each disease and identify retinal movement occurring in the subject's retina.

[0071] Furthermore, by setting the first point P1 regularly, the processing unit 130 can prevent the value of the calculated similarity (for example, the inner product) from differing depending on the setting position of the first point P1.

[0072] Furthermore, according to the above-described embodiment, the processing unit 130 further includes a similarity calculation unit 136 that calculates the similarity between the map and the template T by comparing a map (see FIG. 7) indicating the position and amount of change V in the first image IM1 for each first point P1 with a template T that is defined for each disease and indicates the amount of change V at multiple positions, and the output unit 137 outputs information based on the similarity. By the processing unit 130 calculating the similarity between the subject's retinal movement and the template T pattern, it is possible to objectively determine the presence or absence of a retinal movement pattern indicated by the template T, and to eliminate individual differences and arbitrariness between examiners.

[0073] FIG. 16 is a block diagram showing an example of the internal configuration of the processing unit 130 according to this embodiment. At least some of the functions of the processing unit 130 can be implemented using a computer, i.e., an information processing device. As shown in the figure, the computer includes a central processing unit 901, a RAM 902, an input / output port 903, input / output devices 904 and 905, and a bus 906. The computer itself can be implemented using existing technology. The central processing unit 901 executes instructions contained in a program read from the RAM 902 or the like. In accordance with each instruction, the central processing unit 901 writes data to the RAM 902, reads data from the RAM 902, and performs arithmetic and logical operations. The RAM 902 stores data and programs. Each element included in the RAM 902 has an address and can be accessed using the address. RAM stands for "random access memory." The input / output port 903 is a port through which the central processing unit 901 exchanges data with external input / output devices. The input / output devices 904 and 905 are input / output devices. The input / output devices 904 and 905 exchange data with the central processing unit 901 via the input / output port 903. The bus 906 is a common communication path used within the computer. For example, the central processing unit 901 reads and writes data from and to the RAM 902 via the bus 906. Also, for example, the central processing unit 901 accesses the input / output port via the bus 906. Furthermore, all or part of the functional units included in the processing unit 130 may be realized using hardware such as an ASIC, a PLD, or an FPGA. Furthermore, all or part of the functional units may be realized by a combination of software and hardware.

[0074] Note that all or part of the functions of each unit included in the processing unit 130 in the above-described embodiment may be realized by recording a program for realizing these functions on a computer-readable recording medium, and reading and executing the program recorded on the recording medium into a computer system. Note that the term "computer system" here includes hardware such as an OS and peripheral devices.

[0075] Furthermore, "computer-readable recording media" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage units such as hard disks built into computer systems. Furthermore, "computer-readable recording media" may also include devices that dynamically store programs for a short period of time, such as communication lines used when transmitting programs over networks like the Internet or communication lines like telephone lines, or devices that store programs for a fixed period of time, such as volatile memory within a computer system that serves as a server or client in such cases. Furthermore, the program may be one that realizes part of the aforementioned functions, or may be one that can realize the aforementioned functions in combination with a program already stored in the computer system.

[0076] Although one embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to the above, and various design modifications can be made without departing from the spirit of the present invention. Furthermore, the configurations described in the above-described embodiments and examples can be combined. [Explanation of symbols]

[0077] 100...OCT device, 110...imaging unit, 120...image supply unit, 130...processing unit, 140...display unit, 131...image acquisition unit, 132...first point setting unit, 133...second point identification unit, 134...change amount calculation unit, 135...template storage unit, 136...similarity calculation unit, 137...output unit, P1...first point, P2...second point, L...grid line, PM1...first point map, PM2...second point map, VM...vector map, V...change amount, IM1...first image, IM2...second image

Claims

1. an image acquisition unit that acquires, among images obtained by optical coherence tomography angiography (OCTA) of the same location of the same subject, a first image obtained by capturing an image in a first time period and a second image obtained by capturing an image in a time period different from the first time period; a first point setting unit that sets a plurality of first points indicating any part of the subject in the first image; a second point specifying unit that specifies, for each of the first points, a second point that is a position in the second image where a part of the subject indicated by the first point is shown, based on a similar shape between the subject shown in the first image and the subject shown in the second image; a change amount calculation unit that calculates, for each of the first points, an amount of change from a position of the first point to a position of the second point that indicates the same part as the first point; an output unit that outputs the calculated result; A processing device comprising:

2. The amount of change is a vector indicating a direction from the first point to the second point and a distance from the first point to the second point. The processing device of claim 1 .

3. Each of the first points is set for each area where one or more pixels are regularly integrated. The processing device of claim 1 .

4. a similarity calculation unit that calculates a similarity between a map indicating the position and the amount of change in the first image for each of the first points and a template that is determined for each disease and indicates the amount of change at a plurality of positions, and the output unit outputs information based on the similarity. The processing device of claim 1 .

5. An OCT device comprising the processing device according to any one of claims 1 to 4.

6. On the computer, an image acquisition step of acquiring a first image obtained by capturing an image in a first time period and a second image obtained by capturing an image in a time period different from the first time period, among images obtained by optical coherence tomography angiography (OCTA) of the same location of the same subject; a first point setting step of setting a plurality of first points indicating any part of the subject in the first image; a second point specifying step of specifying, for each of the first points, a second point at which a portion of the subject indicated by the first point is shown in the second image, based on a similar shape between the subject shown in the first image and the subject shown in the second image; a change amount calculation step of calculating, for each of the first points, a change amount from a position of the first point to a position of the second point indicating the same portion as the first point; an output step for outputting the calculated result; A program that executes the following.

Images