Image processing device, optical tomographic image imaging device equipped therewith, and computer program

Abstract

This image processing device comprises: an acquisition unit that acquires a plurality of tomographic images obtained by imaging cross-sections of an eye-under-test at different positions; a display unit that displays the tomographic images acquired by the acquisition unit; a specifying unit; and a calculation unit. The specifying unit is operated by a user, and specifies, with respect to a tomographic image displayed on the display unit, the location of a boundary line of each layer in the tomographic image. The calculation unit is configured such that, if an unspecified tomographic image, in which the location of a boundary line is not specified, is located between two specified tomographic images in which cross-sections are adjacent, the two specified tomographic images being among specified tomographic images in which locations of boundary lines are specified by the specifying unit, interpolation processing can be executed for interpolation on the basis of boundary lines in the two specified tomographic images to determine the boundary line in the unspecified tomographic image.

Description

Image processing device, optical tomographic imaging device equipped with the same, and computer program

[0001] The present disclosure relates to an image processing device, an optical tomographic imaging apparatus including the image processing device, and a computer program, and more particularly to a technology for processing a tomographic image of an eye to be examined.

[0002] An examiner, such as a doctor, may use a tomographic image of the subject's eye to diagnose the condition of the subject's eye. When the examiner diagnoses the subject's eye by observing the tomographic image of the subject's eye, drawing a boundary line between each layer of the subject's eye makes it easier to understand the tissues within the subject's eye. For example, Patent Document 1 discloses an image processing device that adds a boundary line between each layer of the subject's eye.

[0003] Japanese Patent Application Laid-Open No. 2019-88957

[0004] The image processing device described in Patent Document 1 assigns boundary lines between each layer of the subject's eye in a tomographic image, but generally cannot accurately identify the position of all boundaries. For example, in areas where the boundaries between layers are not clearly imaged, such as diseased areas, the image processing device may not be able to accurately identify the position of the boundary between layers. For this reason, in areas where the boundary lines are misaligned, the user of the image processing device must manually correct the position of the boundary line. Because the number of tomographic images taken of one subject's eye is enormous, manually correcting the position of the boundary line for all tomographic images requires a significant amount of time, which is a problem.

[0005] The present specification discloses an image processing device that accurately adds boundary lines between layers of a tomographic image of an eye to be examined while reducing the burden on the user.

[0006] In a first aspect of the technology disclosed in this specification, an image processing device includes an acquisition unit that acquires multiple tomographic images capturing cross sections of a subject's eye at different positions, a display unit that displays the tomographic images acquired by the acquisition unit, a designation unit, and a calculation unit. The designation unit is operated by a user to designate the position of a boundary line of each layer in the tomographic image displayed on the display unit. The calculation unit is configured to perform an interpolation process to determine a boundary line of the non-designated tomographic image by interpolating based on the boundary lines of the two designated tomographic images when a non-designated tomographic image for which a boundary line position is not designated is located between two designated tomographic images whose cross sections are adjacent to each other, among the designated tomographic images for which a boundary line position is designated by the designation unit.

[0007] In the image processing device described above, during the interpolation process, the boundary line of a non-designated tomographic image located between two designated tomographic images is interpolated based on the boundary lines of the two designated tomographic images located on either side (i.e., the corrected boundary lines). Therefore, if a user of the image processing device (hereinafter simply referred to as "user") instructs correction of the position of the boundary line for two tomographic images, the boundary line of a tomographic image (non-designated tomographic image) sandwiched between the two tomographic images (designated tomographic images) is corrected to a nearly accurate position by the interpolation process, even without the need for a correction instruction. This allows the user to reduce the number of tomographic images for which correction is required, thereby reducing the workload on the user.

[0008] In another aspect of the technology disclosed herein, an image processing device includes an acquisition unit that acquires a three-dimensional image generated from multiple tomographic images of cross sections of a subject's eye at different positions, a display unit that displays the three-dimensional image acquired by the acquisition unit, a designation unit, and a calculation unit. The designation unit is operated by a user to designate the position of a boundary line of each layer in the three-dimensional image displayed on the display unit. The calculation unit is configured to execute, for the three-dimensional image acquired by the acquisition unit, a specific range designation process that designates a specific range in which the position of the boundary line is to be corrected, and a spatial interpolation process that corrects the boundary line of the specific range by spatial interpolation based on the position of the boundary line of a non-specific range not designated in the specific range designation process. The designation unit designates a boundary line for the three-dimensional image on which the spatial interpolation process has been executed. The calculation unit further performs spatial correction based on the boundary line designated by the designation unit to determine the boundary line of the specific range.

[0009] In the image processing device described above, a specific range is specified in a three-dimensional image of the subject's eye, and the boundary line of the specific range is spatially interpolated based on the boundary line of the non-specific range. This corrects any deviations in the boundary line within the specific range to the correct position or its vicinity, reducing the number of areas where the user must correct the boundary line position. Furthermore, after the boundary line of the specific range is spatially interpolated based on the boundary line of the non-specific range (after the spatial interpolation process), if the position of the boundary line of the specific range is further specified, the position of the boundary line is further spatially interpolated based on the specified position. This corrects any deviations in the boundary line that were not corrected by the spatial interpolation process, allowing the position of the boundary line to be determined with high accuracy.

[0010] In another aspect of the technology disclosed herein, an image processing device includes an acquisition unit that acquires multiple tomographic images of cross sections of a subject's eye at different positions, a calculation unit, a display unit, and a correction instruction unit. The calculation unit is configured to perform a segmentation process that determines the boundary lines of each layer by performing segmentation using machine learning on each of the multiple tomographic images acquired by the acquisition unit, and a specific range designation process that designates a specific range in which the position of the boundary lines is to be corrected for the tomographic images whose boundary lines have been determined. The display unit displays the tomographic images acquired by the acquisition unit and the specific range designated by the calculation unit. The correction instruction unit is operated by a user to instruct correction of the position of the boundary lines of the tomographic images within the specific range for the tomographic images displayed on the display unit.

[0011] In the image processing device described above, the calculation unit specifies a specific range, allowing the user to narrow down the range in which the boundary line position is to be corrected, thereby reducing the workload imposed on the user when correcting the boundary line position.

[0012] In addition, the optical tomographic imaging device disclosed in this specification includes an imaging unit that captures tomographic images at cross sections at different positions of the subject's eye, and any of the image processing devices described above that processes the multiple tomographic images captured by the imaging unit.

[0013] The optical tomographic imaging apparatus includes any one of the image processing devices described above, and therefore can achieve the same effects as those of the image processing device described above.

[0014] The present specification also discloses a computer program for processing a plurality of tomographic images obtained by capturing cross sections of a subject's eye at different positions. The computer program causes a computer to function as an acquisition unit that acquires a plurality of tomographic images, a receiving unit that receives a designation instruction from a user that designates the position of a boundary line of each layer in the tomographic image, and an interpolation unit that, when a non-designated tomographic image whose boundary line position is not designated is located between two designated tomographic images whose cross sections are adjacent to each other and whose boundary line positions are designated in accordance with the designation instruction received from the receiving unit, determines a boundary line of the non-designated tomographic image by interpolating based on the boundary lines of the two designated tomographic images.

[0015] 6A to 6C are block diagrams showing the schematic configuration of optical tomographic imaging apparatuses according to Examples 1 to 4; FIG. 6B is a flowchart showing an example of processing in which an image processing apparatus corrects boundary lines of each layer in a tomographic image of a subject's eye in Example 1; FIG. 6C is a diagram showing an en-face image on which a mark indicating a specific range is superimposed; FIG. 6D is a diagram showing a tomographic image on which a boundary line is superimposed, where (a) is a tomographic image on which a boundary line identified by a segmentation process is superimposed, and (b) is a tomographic image in which the position of the boundary line has been corrected with respect to the tomographic image of (a); FIG. 6D is a flowchart showing an example of processing in which an image processing apparatus corrects boundary lines of each layer in a tomographic image of a subject's eye in Example 4; FIG. 6A is an en-face image showing a specific range and a non-specific range, where the thickness of the arrows indicates the weighting in the first spatial interpolation process; FIG. 6B is an en-face image showing a specific range and a non-specific range, where (a) is a correction position added to FIG. 6, and (b) is a diagram showing the weighting in the second spatial interpolation process, where the thickness of the arrows indicates the weighting in the second spatial interpolation process. 8. A diagram showing an example of an image displayed on the display unit when a correction position of a boundary line is input in Example 5. An enlarged view showing a main part IX of Figure 8. An enlarged view showing a main part X of Figure 8. A diagram showing an example of an image when a specific range is specified using a polarization tomographic image in Example 5. A diagram showing an example of an image after correcting the position of a boundary line within the specified specific range in Example 5.

[0016] The main features of the embodiments described below are listed below. Note that the technical elements described below are independent technical elements that exhibit technical utility alone or in various combinations, and are not limited to the combinations described in the claims at the time of filing.

[0017] In a second aspect of the technology disclosed in this specification, in the first aspect described above, the calculation unit may be further configured to perform a segmentation process for determining the boundary lines of each layer by performing segmentation using machine learning on each of the multiple tomographic images acquired by the acquisition unit. The display unit may display the tomographic image on which the boundary lines determined by the segmentation process are superimposed. The designation unit may instruct the display unit to correct the position of the boundary lines superimposed on the tomographic image. With this configuration, the calculation unit automatically determines the boundary lines based on machine learning. This makes it possible to acquire multiple tomographic images in which the positions of the boundary lines are identified with relatively high accuracy, thereby reducing the workload imposed on the user when correcting the positions of the boundary lines.

[0018] In a third aspect of the technology disclosed in this specification, in the second aspect described above, the multiple tomographic images may include non-polarized tomographic images showing the tissue in the subject's eye in terms of scattering intensity and polarized tomographic images showing the polarization state in the subject's eye. The calculation unit may perform segmentation processing based on the non-polarized tomographic images and the polarized tomographic images. This configuration increases the amount of reference information regarding the position of the boundary line. Therefore, the calculation unit can more accurately determine the position of the boundary line in the segmentation processing.

[0019] In a fourth aspect of the technology disclosed in this specification, in the second or third aspect described above, the calculation unit may be further configured to perform a specific range designation process for designating a specific range in which the position of the boundary line is to be corrected for the tomographic image in which the boundary line has been determined by the segmentation process. With this configuration, the calculation unit can designate the specific range, allowing the user to narrow down the range in which the position of the boundary line is to be corrected to only the specific range. This reduces the workload on the user.

[0020] In a fifth aspect of the technology disclosed in this specification, in the fourth aspect described above, the calculation unit may further calculate reliability information regarding the reliability of the boundary line position determined by the segmentation process, and in the specific range designation process, may designate a specific range so as to include boundary line positions with low reliability based on the calculated reliability information. In areas where the reliability of the boundary line position determined in the segmentation process is low, the boundary line is likely to be misaligned. By designating the specific range so as to include the areas with low reliability, the range in which the user corrects the boundary line position can be narrowed down to an area where the boundary line is likely to be misaligned.

[0021] In a sixth aspect of the technology disclosed in this specification, in the fourth aspect described above, the calculation unit may, in the specific range designation process, designate a specific range to include two adjacent pixels on a boundary determined by the segmentation process if the positions of the adjacent pixels differ in the depth direction by a predetermined value or more. The boundary line is generally gently curved, and if the position of the boundary line between adjacent pixels differs significantly, it is likely to be a region with a distinctive shape or a diseased area. In regions with distinctive shapes or diseased areas, it is difficult to automatically draw a boundary line, and the boundary line is likely to be misaligned. By designating a specific range to include pixels where the position of the boundary line between adjacent pixels differs significantly, the user can narrow down the range in which the boundary line position is to be corrected to a region where the boundary line is likely to be misaligned.

[0022] In a seventh aspect of the technology disclosed in this specification, in the fourth aspect described above, the calculation unit may specify the specific range in the specific range specification process so as to include pixels on the boundary line determined by the segmentation process whose luminance values ​​are equal to or less than a predetermined value. If the boundary line determined by the segmentation process is shifted to a position other than the boundary between tissue layers of the subject's eye, the luminance value will be low. By specifying the specific range so as to include pixels whose luminance values ​​are equal to or less than the predetermined value, the user can narrow down the range in which the boundary line position is to be corrected to a range where the boundary line is likely to be shifted.

[0023] In an eighth aspect of the technology disclosed in this specification, in the fourth aspect described above, the calculation unit may be configured to further execute a fixation failure identification process for identifying a location where a fixation failure occurs in a plurality of tomographic images. The calculation unit may specify a specific range to include the location where the fixation failure occurs identified by the fixation failure identification process. In the location where the fixation failure occurs, the subject's eye is likely to shift in the depth direction, making it highly likely that the boundary line is misaligned. By specifying the specific range to include the location where the fixation failure occurs, the user can narrow down the range in which the boundary line position is corrected to a range where the boundary line is highly likely to be misaligned.

[0024] In a ninth aspect of the technology disclosed herein, in any one of the fourth to eighth aspects, the plurality of tomographic images may include a non-polarized light tomographic image showing the tissue in the subject's eye in terms of scattering intensity, and a polarized light tomographic image showing the polarization state in the subject's eye. The calculation unit may perform a specific range designation process based on the non-polarized light tomographic image and the polarized light tomographic image. This configuration increases the amount of referenceable information regarding the position of the boundary line. This allows the calculation unit to more accurately designate the specific range in the specific range designation process.

[0025] In a tenth aspect of the technology disclosed in this specification, in any one of the second to ninth aspects, the calculation unit may be configured to execute a specific tomographic image designation process for designating a specific tomographic image, which is a tomographic image for which the position of a boundary line is to be corrected, among the plurality of tomographic images. With this configuration, the calculation unit designates the specific tomographic image, so that the user only needs to correct the position of the boundary line for the designated tomographic image (i.e., the specific tomographic image) among the plurality of tomographic images. This reduces the workload on the user.

[0026] In an eleventh aspect of the technology disclosed in this specification, in the above-mentioned tenth aspect, the calculation unit may further calculate reliability information regarding the reliability of the position of the boundary line determined by the segmentation process, and in the specific tomographic image designation process, based on the calculated reliability information, designate a tomographic image including the position of the boundary line with low reliability as the specific tomographic image.

[0027] In a twelfth aspect of the technology disclosed in this specification, in the above-mentioned tenth aspect, the calculation unit may designate as the specific tomographic image, in the specific tomographic image designation process, a tomographic image in which the positions of two adjacent pixels on a boundary line determined by the segmentation process differ in the depth direction by a predetermined value or more.

[0028] In a thirteenth aspect of the technology disclosed in this specification, in the above tenth aspect, the calculation unit may designate as the specific tomographic image a tomographic image that includes pixels whose brightness values ​​on the boundary line determined by the segmentation process are equal to or less than a predetermined value in the specific tomographic image designation process.

[0029] In a fourteenth aspect of the technology disclosed in the present specification, in the tenth aspect, the calculation unit may be further configured to be capable of executing a fixation failure identification process for identifying a location where a fixation failure occurs in a plurality of tomographic images. The calculation unit may designate a tomographic image where a fixation failure occurs, identified by the fixation failure identification process, as a specific tomographic image.

[0030] In a fifteenth aspect of the technology disclosed herein, in any one of the tenth to fourteenth aspects, the plurality of tomographic images may include a non-polarized tomographic image showing the tissue in the subject's eye in terms of scattering intensity, and a polarized tomographic image showing the polarization state in the subject's eye. The calculation unit may perform a specific tomographic image designation process based on the non-polarized tomographic image and the polarized tomographic image. This configuration increases the amount of referenceable information regarding the position of the boundary line. As a result, the calculation unit can more accurately designate a specific tomographic image in the specific tomographic image designation process.

[0031] In a sixteenth aspect of the technology disclosed in the present specification, in the first or second aspect described above, the designation unit may designate a specific area of ​​the subject's eye in the tomographic image in accordance with a user's operation and instruct correction of the position of the boundary line within the designated specific area. With this configuration, for example, the user designates a range in which the user determines that the position of the boundary line is misaligned as the specific area. This allows the user to narrow down the area in which the position of the boundary line is to be corrected to just the specific area, thereby reducing the user's workload.

[0032] In a seventeenth aspect of the technology disclosed herein, in any one of the first to sixteenth aspects, the plurality of tomographic images may include a non-polarized tomographic image showing the tissue in the subject's eye in terms of scattering intensity, and a polarized tomographic image showing the polarization state in the subject's eye. The display unit may display the non-polarized tomographic image and the polarized tomographic image. This configuration increases the amount of reference information regarding the position of the boundary line displayed on the display unit. This allows the user to more accurately correct the position of the boundary line.

[0033] Example 1 An optical tomographic imaging apparatus 100 according to an example embodiment will be described with reference to the drawings. As shown in FIG. 1 , the optical tomographic imaging apparatus 100 includes an image processing device 10 and an imaging unit 40. The imaging unit 40 captures a tomographic image of the subject's eye using optical coherence tomography (OCT). Note that the imaging unit 40 is not limited to one that uses optical coherence tomography as long as it can capture a tomographic image of the subject's eye. Furthermore, if optical coherence tomography is used, the type of optical coherence tomography is not particularly limited. For example, if the imaging unit 40 uses optical coherence tomography, the imaging unit 40 may be configured to capture a tomographic image showing the tissue in the subject's eye in terms of scattering intensity (a so-called normal tomographic image), or may be configured to capture a tomographic image showing the polarization state in the subject's eye (hereinafter also referred to as a polarization tomographic image). When capturing a polarization tomographic image, the imaging unit 40 may simultaneously capture a tomographic image captured by irradiating the subject's eye with a vertical wave and a tomographic image captured by irradiating the subject's eye with a horizontal wave. By using these two types of tomographic images, not only normal tomographic images but also polarization tomographic images can be generated. Polarization tomographic images include, for example, tomographic images showing the entropy within the subject's eye or tomographic images showing the birefringence within the subject's eye. Furthermore, the imaging unit 40 may be a unit used in a known optical tomographic imaging device, and therefore a detailed description thereof will be omitted.

[0034] The image processing device 10 includes a calculation unit 12, a communication unit 30, a display unit 32, and an input device 34. The calculation unit 12 is connected to the communication unit 30, the display unit 32, and the input device 34 via wiring such as a bus bar so that they can communicate with each other.

[0035] The calculation unit 12 is configured using a computer including a memory 14 and a CPU (not shown). The memory 14 includes hardware such as a ROM and a RAM. A calculation program is stored in the memory 14, and when the CPU executes the calculation program, the calculation unit 12 functions as the segmentation processing unit 16, interpolation processing unit 18, and the like shown in FIG. 1 . The processing of the segmentation processing unit 16 and the interpolation processing unit 18 will be described in detail later. The memory 14 also stores feature amounts for discriminating the boundary lines of each layer in a tomographic image of the eye, which are acquired in advance by machine learning. These feature amounts are used when the segmentation processing unit 16 performs segmentation processing by machine learning, which will be described later.

[0036] The communication unit 30 includes a communication interface that enables the image processing device 10 to communicate with the outside world via a wired or wireless connection. The calculation unit 12 can acquire various data from outside the image processing device 10 via the communication unit 30. Specifically, the communication unit 30 acquires, from the imaging unit 40, tomographic images of the subject's eye captured by the imaging unit 40. The communication unit 30 may be connected to devices outside the optical tomographic imaging apparatus 100 to acquire tomographic images of the subject's eye and other data from the devices outside the optical tomographic imaging apparatus 100.

[0037] In this embodiment, the display unit 32 is a monitor that displays tomographic images of the subject's eye (specifically, normal tomographic images and polarized tomographic images), en-face images and three-dimensional images generated from the tomographic images of the subject's eye. The display unit 32 also displays the boundary lines of each layer superimposed on the tomographic images and three-dimensional images of the subject's eye.

[0038] In this embodiment, the input device 34 is a mouse, and is configured to be able to input a correction position for the boundary line in the tomographic image of the subject's eye displayed on the display unit 32. In this embodiment, a user of the image processing device 10 (hereinafter also simply referred to as "user") can view the tomographic image of the subject's eye displayed on the display unit 32 and determine whether the position of the superimposed boundary line is misaligned. If the user determines that the position of the boundary line is misaligned, the user can use the input device 34 to input a correction position for the boundary line so that the boundary line is positioned at the layer boundary position (i.e., the correct position without any misalignment).

[0039] Next, a process in which the image processing device 10 corrects the boundary lines of each layer in a tomographic image of the test eye will be described. As will be described later, in this embodiment, the calculation unit 12 automatically identifies the boundary lines of each layer for each tomographic image of the test eye using machine learning. Although the automatically identified boundary lines of each layer are generally accurately located, some positional deviations occur. For example, in areas where each layer of the test eye is clearly imaged, the calculation unit 12 can accurately identify the position of the boundary lines. On the other hand, if the test eye contains a diseased region, the layer boundaries may not be clearly imaged in the diseased region. Therefore, in areas where the layer boundaries are not clearly imaged, such as diseased regions, the calculation unit 12 cannot accurately identify the position of the boundary lines, which may result in the boundary lines being shifted or being disconnected without being identified. Furthermore, while the boundaries of each layer of the test eye are generally gently curved, the position of the layer boundaries varies significantly in areas with distinctive morphologies, such as the macula and optic disc. In such areas, the calculation unit 12 may have difficulty accurately identifying the boundary lines, resulting in the boundary lines being shifted. If the automatically identified boundary line is misaligned, the user must manually correct the position of the boundary line while viewing the tomographic image of the subject's eye. If the number of corrections required by the user is large, the user's workload increases. Below, we will explain a process for correcting the position of the boundary line of each layer in the tomographic image of the subject's eye in order to reduce the number of corrections required by the user.

[0040] 2 , first, the calculation unit 12 acquires a tomographic image of a predetermined area of ​​the subject's eye (in this embodiment, the fundus) (S10). In this embodiment, the imaging unit 40 captures 256 tomographic images obtained by dividing (cutting) the subject's eye in 256 directions, and the calculation unit 12 acquires the 256 tomographic images from the imaging unit 40 via the communication unit 30. Note that the number of tomographic images of the subject's eye (i.e., the number of times the subject's eye is divided and captured) is not particularly limited.

[0041] Next, the calculation unit 12 generates an en-face image of the subject's eye from the multiple tomographic images acquired in step S10 (S12). Specifically, the calculation unit 12 compresses the three-dimensional image generated from the multiple tomographic images of the subject's eye into a two-dimensional front image to generate the en-face image. Note that in step S10, the calculation unit 12 may acquire the tomographic image of the subject's eye from a device external to the optical tomographic imaging apparatus 100 via the communication unit 30. Furthermore, when acquiring the tomographic image of the subject's eye from a device external to the optical tomographic imaging apparatus 100 in step S10, the calculation unit 12 may acquire the tomographic image of the subject's eye and an en-face image of the subject's eye generated from the tomographic image.

[0042] Next, the segmentation processing unit 16 performs segmentation processing on each of the multiple tomographic images using machine learning (S14). Specifically, the segmentation processing unit 16 identifies the boundary lines of each layer for each of the tomographic images of the subject's eye acquired in step S10 using feature quantities for identifying the boundary lines of each layer stored in the memory 14. Note that the machine learning used in the segmentation processing can use known machine learning techniques, and therefore a detailed description thereof will be omitted. By performing the segmentation processing using machine learning, the calculation unit 12 can accurately identify the positions of the boundary lines of each layer. Furthermore, in step S14, the calculation unit 12 may perform the segmentation processing using a known method that does not use machine learning (e.g., a known method disclosed in Japanese Patent Application Laid-Open No. 2019-88957, etc.) instead of the segmentation processing using machine learning.

[0043] Next, the calculation unit 12 evaluates the accuracy of the positions of the boundary lines identified in step S14 (S16). As described above, when segmentation processing is performed using machine learning, the positions of the boundary lines of each layer can be identified relatively accurately, but it is difficult to accurately identify the positions of all the boundary lines in multiple tomographic images. The calculation unit 12 evaluates the accuracy of the positions of the boundary lines identified in step S14 in order to estimate the positional deviation of the boundary lines identified in step S14 (more specifically, the locations where the positional deviation is likely to occur).

[0044] Specifically, the calculation unit 12 calculates the reliability of the boundary line position identified using machine learning in step S12. For example, in a region that does not have a distinctive shape and in which the layers are clearly imaged, the calculation unit 12 can accurately identify the boundary line based on machine learning. This results in a high reliability. On the other hand, in a region in which the layers are not clearly imaged or in which the layers have a distinctive shape, the calculation unit 12 has difficulty accurately identifying the boundary line. This results in a low reliability.

[0045] Next, the calculation unit 12 specifies a range including the low-rated area (hereinafter also referred to as a specific range) based on the evaluation result in step S14 (S18). Specifically, the calculation unit 12 specifies the specific range so as to include the low-rated area (in this embodiment, also referred to as an area with low reliability) and its surrounding areas.

[0046] Next, the calculation unit 12 displays on the display unit 32 an en-face image showing the specific range specified in step S18 and a tomographic image of the subject's eye in the specific range (S20). For example, as shown in FIG. 3, the calculation unit 12 displays on the display unit 32 an en-face image on which a mark indicating the specific range (a square in FIG. 3) is superimposed. Note that the specific range displayed and superimposed on the en-face image may be configured so that the user can change the range using the input device 34. The calculation unit 12 can also display on the display unit 32 a tomographic image of a range corresponding to the specific range. The display unit 32 displays a tomographic image switchably selected by the user using the input device 34. By superimposing the specific range on the en-face image and displaying it on the display unit 32, it is possible to notify the user of a range where the boundary line is likely to be misaligned. The user only needs to check the boundary line position misalignment by narrowing down the range where the boundary line is likely to be misaligned (i.e., the specific range), thereby reducing the user's workload.

[0047] Next, the interpolation processing unit 18 determines whether a command to complete correction has been input (S22). The user checks the positional deviation of the boundary lines within a specific range for the multiple tomographic images. If the user determines that there is no positional deviation of the boundary lines within the specific range, the user instructs the completion of correction. For example, the user instructs the completion of correction by using the input device 34 to click the "Complete" button displayed on the display unit 32. When the command to complete correction has been input (YES in step S22), the process of correcting the boundary lines of each layer in the tomographic image of the subject's eye in FIG. 2 is terminated.

[0048] On the other hand, if a command to complete the correction has not been input (NO in step S22), the calculation unit 12 determines whether a corrected position of the boundary line has been input (S24). The user checks the positional deviation of the boundary line within a specific range for multiple tomographic images, and if they determine that there is a positional deviation of the boundary line, they use the input device 34 to input the exact position of the boundary line (i.e., the corrected position). For example, FIG. 4(a) shows a tomographic image on which the boundary line identified by the segmentation process of step S14 is superimposed. For ease of explanation, eight marks (circles) are added to the uppermost boundary line in FIG. 4(a). As shown in FIG. 4(a), the boundary line identified by the segmentation process of step S14 includes a portion located on the layer boundary (indicated by the first, second, fourth, fifth, sixth, and eighth marks from the left in FIG. 4(a)) and a portion deviated from the layer boundary (indicated by the third and seventh marks from the left and the portion between the first and second marks from the left (indicated by the arrows) in FIG. 4(a)). As shown in FIG. 4B, the user inputs the corrected position of the boundary line using the input device 34 so that the portion where the boundary line is misaligned is positioned on the boundary of the layers.

[0049] When inputting the boundary correction position, the user only needs to input the boundary correction position for a few selected tomographic images from the multiple tomographic images. For example, the user can sequentially review the 256 tomographic images, select a few tomographic images determined to have a significant or no misalignment, and input the boundary correction position for only the selected few tomographic images. Alternatively, the user may input the boundary correction position within a specific range for every few (e.g., every fifth) of the 256 tomographic images. Hereinafter, a tomographic image for which a boundary correction position has been input may be referred to as a "designated tomographic image," and a tomographic image for which a boundary correction position has not been input may be referred to as a "non-designated tomographic image." When the user has completed the input of the boundary correction position, the user indicates completion of input. For example, before inputting the boundary correction position, the user may use the input device 34 to click the "Correction" button displayed on the display unit 32 to start the process of inputting the boundary correction position. Then, when the task of inputting the corrected position of the boundary line is completed, the user indicates completion of the task of inputting the corrected position of the boundary line by again clicking the "Correction" button displayed on the display unit 32 using the input device 34. If the corrected position of the boundary line has not been input (NO in step S24), the process returns to step S22, and the processes of steps S22 and S24 are repeated.

[0050] When the corrected boundary line positions are input (YES in step S24), the interpolation processing unit 18 executes interpolation processing based on the corrected boundary line positions input in step S24 (S28). Specifically, the interpolation processing unit 18 interpolates and corrects the positions of the surrounding boundary lines within a specified tomographic image based on the corrected positions input within the same specified tomographic image. The interpolation processing unit 18 also interpolates and corrects the positions of the boundary lines of non-specified tomographic images located between two specified tomographic images based on the corrected positions of the boundary lines within the two specified tomographic images. Note that a known interpolation processing technique can be used for the interpolation processing, and therefore a detailed description thereof will be omitted.

[0051] When the interpolation process is completed, the calculation unit 12 displays the interpolated tomographic image on the display unit 32 (S28). Next, the process returns to step S22, and steps S22 to S28 are repeated. That is, the user checks the tomographic image after the interpolation process, and if the boundary line position is still misaligned, inputs the corrected position of the boundary line again. The calculation unit 12 then interpolates and corrects the boundary line position based on the corrected position of the boundary line that has been re-input. These processes are repeated until the user determines that the boundary line position is not misaligned (until step S22 returns YES).

[0052] In this embodiment, the user inputs boundary line corrections only for a few tomographic images (specified tomographic images) selected from the plurality of tomographic images, and the interpolation processing unit 18 interpolates and corrects the positions of the boundary lines in the non-specified tomographic images based on the corrected positions of the boundary lines in the specified tomographic images. This allows the user to correct the positions of the boundary lines for all of the plurality of tomographic images without having to input the corrected positions of the boundary lines for all of the tomographic images. This reduces the user's workload. Furthermore, the calculation unit 12 evaluates the accuracy of the boundary line positions identified by the segmentation process in step S14 and specifies a specific range to include areas with low evaluations. This reduces the user's workload because the user only needs to input the corrected positions of the boundary lines for the specific range.

[0053] In this embodiment, in step S16, when evaluating the accuracy of the boundary line positions identified in the segmentation process of step S14, the reliability of the boundary line positions identified using machine learning is calculated and evaluated, but the present invention is not limited to this configuration. Below, other examples of methods for evaluating the accuracy of the boundary line positions identified in the segmentation process of step S14 will be described.

[0054] For example, if adjacent pixels on a boundary line identified by the segmentation process in the same tomographic image differ significantly in the depth direction, the calculation unit 12 may evaluate the accuracy of the boundary line position for these two adjacent pixels as low. As described above, each layer of the subject's eye is gently curved in many areas. On the other hand, the position of the boundary line may change significantly in areas with distinctive morphologies, such as the macula or optic disc, or in diseased areas. If the position of the boundary line changes significantly relative to adjacent areas, it becomes difficult to accurately identify the position of the boundary line in the segmentation process. Therefore, in such areas, the position of the boundary line identified by the segmentation process is likely to be misaligned. Therefore, if the depth position of adjacent pixels on the boundary line differs by more than a predetermined value, the calculation unit 12 may evaluate the accuracy of the boundary line position for these two pixels as low. Even when using this method, the user can correct the position of the boundary line by focusing on the area where the position of the boundary line changes significantly, thereby reducing the user's workload and accurately correcting the position of the boundary line.

[0055] The calculation unit 12 may also evaluate the accuracy of the boundary line position based on the luminance value on the boundary line identified by the segmentation process. Specifically, the calculation unit 12 may evaluate the accuracy of the boundary line position as low if the luminance value on the boundary line identified by the segmentation process is below a predetermined value. As shown in FIG. 4( a), if the boundary line position is shifted and the boundary line is located in a position where there is no tissue, the luminance value on the boundary line may be extremely low (e.g., the area indicated by the arrow in the center or on the right side of FIG. 4( a)). Alternatively, even if the boundary line is located in the same tissue, the luminance value at the boundary line position is low because there is no light reflection at the position where the tissue changes. In other words, if the luminance value on the boundary line is lower than a predetermined value, the boundary line is likely to be shifted. Therefore, if the luminance value on the boundary line is lower than the predetermined value, the accuracy of the boundary line position may be evaluated as low. Even when using such a method, the user can correct the boundary line position by narrowing down the area where the boundary line is likely to be shifted, thereby reducing the user's workload and accurately correcting the boundary line position.

[0056] The calculation unit 12 may also identify areas where fixation failure occurs from multiple tomographic images and evaluate the accuracy of the boundary line position at the areas where fixation failure occurs as low. For example, fixation failure occurs due to fixation disparity or blinking. Areas where fixation failure occurs are likely to result in images that are displaced in the depth direction. Therefore, the calculation unit 12 may evaluate the accuracy of the boundary line position at the areas where fixation failure occurs as low. Specifically, the calculation unit 12 compares multiple tomographic images to identify images in which the position of the subject's eye is displaced in the depth direction. Images that are displaced in the depth direction are likely to indicate that fixation disparity occurred during capture. The calculation unit 12 also compares multiple tomographic images to identify images with low overall image brightness. Images with low overall image brightness are likely to indicate that blinking occurred during capture. The calculation unit 12 evaluates the accuracy of the boundary line position at the identified image or at areas displaced in the depth direction relative to other images within the identified image as low. Even when using such a method, the user can correct the position of the boundary line by narrowing down the range where the boundary line is likely to be misaligned, and the position of the boundary line can be corrected with high accuracy while reducing the workload on the user.

[0057] (Example 2) In Example 1 described above, in step S18, the calculation unit 12 designated a specific range based on the evaluation result of the accuracy of the boundary line position. However, this configuration is not limited to this. For example, instead of designating a specific range based on the evaluation result of the accuracy of the boundary line position, the calculation unit 12 may designate a specific tomographic image (hereinafter also referred to as a specific tomographic image) from multiple tomographic images based on the evaluation result of the accuracy of the boundary line position. Specifically, the calculation unit 12 designates a tomographic image that includes many areas where the accuracy of the boundary line position is low as the specific tomographic image. Then, in step S24, the user inputs a correction position of the boundary line for the designated specific tomographic image. Even in this case, the user only needs to input the correction position of the boundary line for only a few designated tomographic images (i.e., the specific tomographic images) among the multiple tomographic images, thereby reducing the user's workload and enabling accurate correction of the boundary line position.

[0058] (Example 3) In Example 1, the calculation unit 12 specified a specific range in step S18, but this configuration is not limited to this. For example, instead of the calculation unit 12 specifying the specific range, the user may specify the specific range. In this case, instead of the processing of steps S16 and S18 in the flowchart of FIG. 2, the calculation unit 12 determines whether a specific range has been input, and after the specific range has been input, executes the processing of step S20 and subsequent steps. For example, the user checks the en-face image generated in step S12, identifies a range where the boundary line position is likely to be misaligned, and inputs that range using the input device 34. A range where the boundary line position is likely to be misaligned is often a range that includes, for example, an area with a characteristic morphology such as the macula or optic disc, or an area where a disease is suspected. If the user is a medical professional such as a doctor, the user can identify a range where the boundary line position is likely to be misaligned from the en-face image. Even if the user specifies a specific range, the user only needs to input the correction position of the boundary line for the specific range, thereby reducing the user's workload.

[0059] (Example 4) In the above Examples 1 to 3, the positions of boundary lines are corrected by interpolation within the same tomographic image or between tomographic images, but the present invention is not limited to such a configuration. For example, the positions of boundary lines may be corrected by spatial interpolation in a three-dimensional image generated from multiple tomographic images. The following describes a process for correcting the boundaries of each layer in a three-dimensional image by spatial interpolation.

[0060] 5, first, the calculation unit 12 acquires a tomographic image of the subject's eye (S100). Note that the process of step S100 is the same as the process of step S10 in the first embodiment, and therefore a detailed description thereof will be omitted.

[0061] Next, the calculation unit 12 generates a three-dimensional image of the subject's eye from the multiple tomographic images acquired in step S100 (S110). Note that in step S100, the calculation unit 12 may acquire a tomographic image of the subject's eye from a device external to the optical tomographic imaging apparatus 100 via the communication unit 30. Furthermore, when acquiring a tomographic image of the subject's eye from a device external to the optical tomographic imaging apparatus 100 in step S100, the calculation unit 12 may acquire a three-dimensional image of the subject's eye generated from the tomographic image together with the tomographic image of the subject's eye.

[0062] Next, the segmentation processing unit 16 performs segmentation processing on each of the multiple tomographic images using machine learning (S120). Note that the processing in step S120 is the same as the processing in step S14 in the first embodiment, and therefore a detailed description thereof will be omitted.

[0063] Next, the calculation unit 12 specifies a specific range (S130). The specific range is a range that includes a location where the boundary line position is likely to be misaligned. The calculation unit 12 may evaluate the accuracy of the boundary line position and specify the specific range based on the evaluation result, or the user may specify the specific range by inputting it using the input device 34. Note that the process of specifying the specific range by the calculation unit 12 can be similar to the processes in steps S16 and S18 in the first embodiment, and therefore a detailed description thereof will be omitted.

[0064] Next, the interpolation processing unit 18 performs spatial interpolation processing (hereinafter referred to as first spatial interpolation processing) on ​​the positions of boundary lines within the specific range specified in step S130 based on the positions of boundary lines within a range other than the specific range specified in step S130 (hereinafter referred to as a non-specific range) (S140). Note that a well-known spatial interpolation processing technique can be used for the spatial interpolation processing, and detailed description thereof will be omitted. As described above, the specific range includes a location where the boundary line position is likely to be misaligned. That is, the non-specific range is a range that does not include or barely includes a location where the boundary line position is likely to be misaligned, and boundary lines located within the non-specific range are unlikely to be misaligned. The interpolation processing unit 18 corrects the positions of boundary lines within the specific range (i.e., boundary lines likely to be misaligned) by spatial interpolation based on the positions of boundary lines within the non-specific range (i.e., boundary lines unlikely to be misaligned). This allows the interpolation processing unit 18 to correct the positional misalignment of boundary lines within the specific range before the user inputs the corrected boundary line position.

[0065] When performing the first spatial interpolation process in step S140, the weight may be changed depending on the distance from the non-specific range. FIG. 6 illustrates the specific range 50 and the non-specific range 52 using an en-face image of the subject's eye. In FIG. 6, the weight of the boundary information within the non-specific range 52 used when spatially interpolating position A on the boundary line within the specific range 50 is indicated by the thickness of the arrow. The closer the distance between position A and the non-specific range 52, the thicker the arrow, and the farther the distance between position A and the non-specific range 52, the thinner the arrow. The interpolation processing unit 18 increases the weight of the boundary information of the non-specific range 52 located closer to position A and decreases the weight of the boundary information of the non-specific range 52 located farther from position A, thereby spatially interpolating and correcting the position of the boundary line of position A within the specific range 50.

[0066] When the first spatial interpolation process is completed, the calculation unit 12 displays the three-dimensional image after the first spatial interpolation process on the display unit 32 (S150). The calculation unit 12 may also display an en-face image or a tomographic image on the display unit 32 together with the three-dimensional image.

[0067] Next, the interpolation processing unit 18 determines whether a correction completion instruction has been input (S160). The user checks the position of the boundary line within a specific range of the three-dimensional image displayed on the display unit 32, and if it is determined that the boundary line position is not misaligned, the user issues a correction completion instruction using the input device 34. If a correction completion instruction has been input (YES in step S160), the process of correcting the boundary line of each layer in the tomographic image of the subject's eye shown in FIG. 5 is terminated. If a correction completion instruction has not been input (NO in step S160), the calculation unit 12 determines whether a boundary line correction position has been input (S170). The user checks the boundary line position within a specific range of the three-dimensional image displayed on the display unit 32, and if it is determined that the boundary line position is misaligned, the calculation unit 12 inputs the exact boundary line position (i.e., the correction position) using the input device 34. If a boundary line correction position has not been input (NO in step S170), the process returns to step S160 and repeats the processes of steps S160 and S170. The processes of steps S160 and S170 are substantially the same as the processes of steps S22 and S24 in the first embodiment, and therefore detailed description thereof will be omitted.

[0068] When the corrected boundary position is input (YES in step S170), the interpolation processing unit 18 performs spatial interpolation processing (hereinafter also referred to as second spatial interpolation processing) for the boundary position within the specific range based on the corrected boundary position input in step S170 and the boundary position within the non-specific range (S180). For example, assume that the boundary position within the specific range 50 has been corrected for four locations indicated by "x" in FIG. 7A. In this case, as shown in FIG. 7B, when position A within the specific range 50 is closer to the non-specific range 52 than the corrected position indicated by "x", the interpolation processing unit 18 uses information on the boundary position within the non-specific range 52, whereas when position A is closer to the corrected position indicated by "x" than the non-specific range, the interpolation processing unit 18 uses information on the corrected position indicated by "x" to spatially interpolate and correct the boundary position of position A. Note that, as shown in FIG. 7B, when performing the second spatial interpolation processing in step S180, the weight may also be changed depending on the corrected boundary position or the distance from the non-specific range.

[0069] When the spatial interpolation process is complete, the calculation unit 12 displays the 3D image after the second spatial interpolation process on the display unit 32 (S190). Next, the process returns to step S160, and steps S160 to S190 are repeated. That is, the user checks the 3D image after the second spatial interpolation process, and if the boundary line position is still misaligned, the user again inputs the corrected position of the boundary line. Then, the calculation unit 12 corrects the boundary line position by spatial interpolation based on the corrected position of the boundary line that was input again. These processes are repeated until the user determines that the boundary line position is not misaligned (until step S160 returns YES).

[0070] In this embodiment, before the user corrects the position of the boundary line, the calculation unit 12 corrects the position of the boundary line within the specific range by spatial interpolation based on the position of the boundary line within the non-specific range. This corrects the position of the boundary line within the range where the position of the boundary line is likely to be misaligned, thereby reducing the positional deviation of the boundary line identified in the segmentation process. This reduces the number of areas where the user needs to correct the position of the boundary line, thereby reducing the workload on the user.

[0071] In the above-described first to fourth embodiments, only normal tomographic images are used to correct the boundary line positions, but polarized light tomographic images may also be used. For example, when inputting the corrected boundary line positions (e.g., step S24 in FIG. 2 and step S170 in FIG. 5), the calculation unit 12 may display not only normal tomographic images but also polarized light tomographic images on the display unit 32.

[0072] Fig. 8 shows an example of an image simultaneously displaying a normal tomographic image, a tomographic image showing birefringence, and a tomographic image showing entropy for a same cross-section. In Fig. 8, a portion (specifically, the center) of the tomographic image for the same cross-section is displayed as a normal tomographic image, another portion (specifically, the left side) is displayed as an image in which a tomographic image showing birefringence is superimposed on the normal tomographic image, and yet another portion (specifically, the right side) is displayed as an image in which a tomographic image showing entropy is superimposed on the normal tomographic image. Fig. 9 shows an image (left) in which a tomographic image showing birefringence is superimposed on a normal tomographic image, and a normal tomographic image (right). Fig. 10 shows a normal tomographic image (left) and an image (right) in which a tomographic image showing entropy is superimposed on a normal tomographic image. The dashed lines in Figs. 9 and 10 indicate layer boundaries.

[0073] As shown in Figure 9, a normal tomographic image (right) can make it difficult for a user to distinguish layer boundaries. On the other hand, in an image (left) in which a tomographic image showing birefringence is superimposed on a normal tomographic image, the birefringence intensity of the tissue within the layer differs from the birefringence intensity at the layer boundary, making it easy to distinguish layer boundaries. Also, as shown in Figure 10, in an normal tomographic image and an image (right) in which a tomographic image showing entropy is superimposed on a normal tomographic image, the entropy values ​​of the tissue within the layer differ from the entropy values ​​at the layer boundary, making it easy to distinguish layer boundaries. Thus, by using normal tomographic images and polarized tomographic images, the information available to the user is increased, making it easier for the user to distinguish layer boundaries compared to using only normal tomographic images. This allows the user to more accurately correct the position of boundary lines.

[0074] Regarding the tomographic images of the same cross section, the display method of the normal tomographic image, the tomographic image showing birefringence, and the tomographic image showing entropy is not particularly limited. For example, the normal tomographic image, the tomographic image showing birefringence, and the tomographic image showing entropy may be displayed separately on the same screen. Furthermore, the corresponding normal tomographic image, the tomographic image showing birefringence, and the tomographic image showing entropy may be switchable and displayable on the display unit 32.

[0075] In this embodiment, the normal tomographic image and the polarized light tomographic image are used when inputting the boundary line correction position (e.g., step S24 in FIG. 2 and step S170 in FIG. 5), but the present invention is not limited to this configuration. For example, both the normal tomographic image and the polarized light tomographic image may be used during the segmentation process (e.g., step S14 in FIG. 2 and step S120 in FIG. 5).

[0076] Furthermore, when specifying a specific range (e.g., step S18 in FIG. 2 and step S130 in FIG. 5), the calculation unit 12 may use a normal tomographic image and a polarized tomographic image. FIG. 11 shows an en-face image (left) at the position of the boundary line before correction and a tomographic image (right) of the cross section indicated by the arrow in the en-face image. FIG. 11 shows an image in which a tomographic image indicating entropy is superimposed on a normal tomographic image, and the boundary line identified in the segmentation process (step S14 in FIG. 2 and step S120 in FIG. 5) is further superimposed. The position of the boundary line is shifted at the position indicated by the arrow in the tomographic image (right). If the position of the boundary line is shifted, the en-face image (left) may show a large difference in entropy value compared to its surroundings, as indicated by the dashed circle. As described above, the entropy value of tissue within a layer differs from the entropy value of the layer boundary, resulting in different entropy values ​​between the location where the boundary line is misaligned (tissue within a layer) and the location where the boundary line is correctly positioned (layer boundary). The calculation unit 12 evaluates the range in the en-face image at the boundary line position identified in the segmentation process, where the entropy values ​​are significantly different (the range indicated by the dashed circle in FIG. 11 ), as having low accuracy of the boundary line position and designates it as a specific range (step S18 in FIG. 2 and step S130 in FIG. 5 ). The user corrects the boundary line position within the specified specific range. When the user corrects the boundary line position at the location where the boundary line position is misaligned in FIG. 11 , the image shown in FIG. 12 is obtained. As shown in FIG. 12 , after correcting the boundary line position, the en-face image (right) has approximately the same entropy value as its surroundings, as indicated by the dashed circle.

[0077] In this case, too, the use of both the normal tomographic image and the polarized tomographic image increases the amount of information available for reference regarding the position of the boundary line, allowing the calculation unit 12 to more accurately specify the range where the boundary line position is likely to be misaligned compared to when only the normal tomographic image is used.

[0078] In the above example, the calculation unit 12 specifies a specific range, but the present invention is not limited to this configuration. For example, the calculation unit 12 may correct the position of a boundary line evaluated to have low accuracy based on a normal tomographic image and a polarized tomographic image (e.g., a tomographic image showing entropy). For example, the calculation unit 12 shifts the position of a boundary line evaluated to have low accuracy to a position where the entropy is approximately the same as that of the surrounding area. In this case, the calculation unit 12 may notify the user of the area where the boundary line position has been automatically corrected and prompt the user to confirm whether the corrected boundary line position is correct. The method of notifying the area where the boundary line position has been automatically corrected is not particularly limited. For example, the calculation unit 12 may surround the area where the boundary line position has been automatically corrected with a mark such as a circle or a square, or may indicate it with an arrow. Furthermore, the calculation unit 12 may display the area where the boundary line position has been automatically corrected in a color different from that of the area where the boundary line position has not been corrected.

[0079] In this embodiment, the position of the boundary line is corrected using not only a normal tomographic image but also a polarized tomographic image. This makes it possible to more accurately determine deviations in boundary lines that are difficult to determine using a normal tomographic image. Therefore, deviations in boundary lines can be corrected more accurately. Because the position of the boundary line is corrected more accurately, when interpolation processing is performed, uncorrected boundary lines (i.e., uncorrected boundary lines within the same tomographic image and boundary lines within uncorrected tomographic images) can also be corrected more accurately.

[0080] In this embodiment, a tomographic image showing birefringence and a tomographic image showing entropy are used as the polarized light tomographic images, but other types of polarized light tomographic images (e.g., a tomographic image showing the direction of fiber running) may also be used. Also, in this embodiment, instead of specifying a specific range, the calculation unit 12 may specify a specific tomographic image from multiple tomographic images using normal tomographic images and polarized light tomographic images. The calculation unit 12 may also perform interpolation processing (e.g., step S26 in FIG. 2 , steps S140 and S180 in FIG. 5 ) using normal tomographic images and polarized light tomographic images.

[0081] Although specific examples of the technology disclosed in this specification have been described in detail above, these are merely examples and do not limit the scope of the claims. The technology described in the claims includes various modifications and variations of the specific examples exemplified above. Furthermore, the technical elements described in this specification or drawings exhibit technical utility alone or in various combinations, and are not limited to the combinations described in the claims at the time of filing. Furthermore, the technology exemplified in this specification or drawings simultaneously achieves multiple objectives, and achieving one of those objectives itself has technical utility.

Claims

1. An image processing device comprising: an acquisition unit that acquires a plurality of tomographic images of cross sections taken at different positions of a test eye; a display unit that displays the tomographic images acquired by the acquisition unit; a designation unit that is operated by a user and designates the position of a boundary line of each layer in the tomographic image for the tomographic image displayed on the display unit; and a calculation unit that is configured to be capable of performing an interpolation process to determine the boundary line of a non-designated tomographic image, when a non-designated tomographic image for which the position of the boundary line is not specified is located between two designated tomographic images whose cross-sectional positions are adjacent to each other, by interpolating based on the boundary lines of the two designated tomographic images.

2. The image processing device of claim 1, wherein the calculation unit is further configured to execute a segmentation process for determining the boundary lines of each layer by performing machine learning segmentation on each of the multiple tomographic images acquired by the acquisition unit, the display unit displays the tomographic image on which the boundary lines determined by the segmentation process are superimposed, and the designation unit instructs correction of the position of the boundary lines superimposed on the tomographic image displayed on the display unit.

3. The image processing device of claim 2, wherein the plurality of tomographic images include a non-polarized tomographic image showing the tissue in the test eye by scattering intensity and a polarized tomographic image showing the polarization state in the test eye, and the calculation unit performs the segmentation processing based on the non-polarized tomographic image and the polarized tomographic image.

4. The image processing device according to claim 2, wherein the calculation unit is further configured to execute a specific range designation process for designating a specific range in which the position of the boundary line is to be corrected for the tomographic image in which the boundary line has been determined by the segmentation process.

5. The image processing device described in claim 4, wherein the calculation unit further calculates reliability information regarding the reliability of the position of the boundary line determined by the segmentation process, and in the specific range designation process, specifies the specific range so as to include the position of the boundary line having a low reliability based on the calculated reliability information.

6. The image processing device of claim 4, wherein the calculation unit, in the specific range designation process, if the positions of two adjacent pixels on the boundary line determined by the segmentation process differ in the depth direction by a predetermined value or more, specifies the specific range to include the two adjacent pixels.

7. The image processing device according to claim 4, wherein the calculation unit, in the specific range designation process, designates the specific range so as to include pixels on the boundary line determined by the segmentation process whose luminance values ​​are equal to or less than a predetermined value.

8. The image processing device of claim 4, wherein the calculation unit is further configured to execute a fixation failure identification process for identifying a location where fixation failure is occurring in the multiple tomographic images, and the calculation unit specifies the specific range so as to include the location where the fixation failure is occurring identified by the fixation failure identification process.

9. The image processing device of claim 4, wherein the plurality of tomographic images include a non-polarized tomographic image showing the tissue in the test eye by scattering intensity and a polarized tomographic image showing the polarization state in the test eye, and the calculation unit executes the specific range designation processing based on the non-polarized tomographic image and the polarized tomographic image.

10. The image processing device according to claim 2, wherein the calculation unit is configured to be capable of executing a specific tomographic image designation process for designating a specific tomographic image, which is a tomographic image for which the position of the boundary line is to be corrected, among the plurality of tomographic images.

11. The image processing device described in claim 10, wherein the calculation unit further calculates reliability information regarding the reliability of the position of the boundary line determined by the segmentation process, and in the specific tomographic image designation process, based on the calculated reliability information, designates a tomographic image including the position of the boundary line having a low reliability as the specific tomographic image.

12. The image processing device according to claim 10, wherein the calculation unit, in the specific tomographic image designation process, designates as the specific tomographic image a tomographic image in which two adjacent pixels on the boundary line determined by the segmentation process differ in depth direction by a predetermined value or more.

13. The image processing device according to claim 10, wherein the calculation unit, in the specific tomographic image designation process, designates as the specific tomographic image a tomographic image including pixels on the boundary line determined by the segmentation process whose brightness value is equal to or less than a predetermined value.

14. The image processing device of claim 10, wherein the calculation unit is further configured to execute a fixation failure identification process for identifying a location where fixation failure occurs in the multiple tomographic images, and the calculation unit designates the tomographic image where the fixation failure occurs, identified by the fixation failure identification process, as the specific tomographic image.

15. The image processing device described in claim 10, wherein the multiple tomographic images include a non-polarized tomographic image showing the tissue in the test eye by scattering intensity and a polarized tomographic image showing the polarization state in the test eye, and the calculation unit executes the specific tomographic image designation process based on the non-polarized tomographic image and the polarized tomographic image.

16. The image processing device according to claim 1, wherein the designation unit, in accordance with an operation by the user, designates a specific range of the subject's eye in the tomographic image and instructs correction of the position of a boundary line within the specified specific range.

17. The image processing device of claim 1, wherein the plurality of tomographic images include a non-polarized tomographic image showing the tissue in the test eye by scattering intensity and a polarized tomographic image showing the polarization state in the test eye, and the display unit displays the non-polarized tomographic image and the polarized tomographic image.

18. An image processing device comprising: an acquisition unit that acquires a three-dimensional image generated from a plurality of tomographic images taken of cross sections at different positions of the test eye; a display unit that displays the three-dimensional image acquired by the acquisition unit; a designation unit that is operated by a user and designates the position of a boundary line of each layer in the three-dimensional image for the three-dimensional image displayed on the display unit; and a calculation unit, wherein the calculation unit is configured to execute, for the three-dimensional image acquired by the acquisition unit, a specific range designation process that designates a specific range in which the position of the boundary line is to be corrected; and a spatial interpolation process that corrects the boundary line of the specific range by spatial interpolation based on the position of the boundary line of a non-specific range not designated in the specific range designation process, wherein the designation unit designates the boundary line for the three-dimensional image on which the spatial interpolation process has been executed, and the calculation unit further determines the boundary line of the specific range by spatial correction based on the boundary line designated by the designation unit.

19. An image processing device comprising: an acquisition unit that acquires a plurality of tomographic images of cross sections taken at different positions of a test eye; a calculation unit that is configured to execute a segmentation process that determines a boundary line of each layer by performing machine learning segmentation on each of the plurality of tomographic images acquired by the acquisition unit; and a specific range designation process that designates a specific range in which to correct the position of the boundary line for the tomographic image in which the boundary line has been determined; a display unit that displays the tomographic images acquired by the acquisition unit and the specific range designated by the calculation unit; and a correction instruction unit that is operated by a user and instructs correction of the position of the boundary line of the tomographic image within the specific range for the tomographic image displayed on the display unit.

20. An optical tomographic imaging device comprising: an imaging unit which captures tomographic images at cross sections at different positions of a subject's eye; and an image processing device according to any one of claims 1 to 19 which processes the multiple tomographic images captured by the imaging unit.

21. A computer program for processing a plurality of tomographic images obtained by photographing cross sections at different positions of a test eye, the computer being configured to function as: an acquisition unit that acquires the plurality of tomographic images; a receiving unit that receives a designation instruction from a user that designates the position of the boundary line of each layer in the tomographic image with respect to the tomographic image; and an interpolation unit that, when a non-designated tomographic image for which the position of the boundary line is not specified is located between two designated tomographic images in which the cross-sectional positions are adjacent, interpolates based on the boundary lines of the two designated tomographic images to determine the boundary line of the non-designated tomographic image.

Images