Image processing device, image processing program, and image processing method

Abstract

To correct contour lines of blood vessels extracted from captured images.SOLUTION: An image processing device 1 corrects contour lines of blood vessels generated on the basis of captured images in which the blood vessels are captured. The contour lines include a first contour line and a second contour line. The image processing device comprises: a storage unit 12 which stores the captured image in association with position information of contour pixels corresponding to the contour lines of the blood vessels; a contour line selection unit 131 which selects one of the first contour line and the second contour line as a target contour line of correction; an extraction unit 132 which extracts the plurality of contour pixels as extraction pixels on the basis of the position information of the contour pixels; a pixel selection unit 133 which selects two extraction pixels having a prescribed positional relation as selection pixels; a correction pixel determination unit 134 which determines correction pixels corresponding to the target contour line after the correction on the basis of pixel information on a plurality of reference pixels arranged at prescribed positions from the two selection pixels; and a correction contour line identifying unit 136 which identifies the target contour line after the correction on the basis of the position information of the correction pixels.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an image processing device, an image processing program, and an image processing method. [Background technology]

[0002] For early detection of vascular disorders in the brain, medical devices that visualize the inside of the human body, such as CT (Computed Tomography) and MRI (Magnetic Resonance Imaging), are used. However, these medical devices are expensive. Therefore, the burden of capital investment is large, and only a limited number of hospitals have these medical devices. In addition, medical expenses are high, so the medical expense burden on patients is also large.

[0003] The brain is the same organ as the eye in terms of development. Therefore, the state of blood vessels in the brain can be inferred from the state of retinal blood vessels in the fundus. Therefore, fundus examinations, which do not require the aforementioned medical equipment, have been used to diagnose the risk of vascular disorders in the brain. In recent years, fundus examinations have been incorporated into internal medicine examinations due to the revision of the third stage of specific health checkups. However, fundus examinations are largely dependent on the experience and skill of the assessing physician. Therefore, it is difficult for non-specialist internal medicine physicians to perform accurate fundus examinations. Therefore, there is a demand for the automation of fundus examinations. To automate fundus examinations, image processing technology is required to accurately extract blood vessels from images of the fundus.

[0004] Image processing techniques for extracting blood vessels from captured images have been proposed (for example, see Non-Patent Document 1).

[0005] The technology disclosed in Non-Patent Document 1 selects solution candidates for vascular walls by searching using an energy function, and extracts the solution candidate with the highest similarity to the vascular wall as the vascular contour. However, the visibility of fundus images is poor, and the accuracy of extraction of vascular contours by the technology disclosed in Non-Patent Document 1 is not necessarily good. Therefore, there is a demand for a technology to correct the vascular contours extracted from a captured image. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] Yuki Okami and 3 others, “Calculation of the diameter of the fundus blood vessels in cataract patients”, Proceedings of the 50th Japan Ergonomics Society Chugoku-Shikoku Branch Conference, 2017 / 12 / 9, p.49-50 Summary of the Invention [Problem to be solved by the invention]

[0007] An object of the present invention is to provide a technique for correcting the contour lines of blood vessels extracted from a captured image. [Means for solving the problem]

[0008] The image processing device of the present invention is an image processing device that corrects a blood vessel contour line generated based on an captured image in which the blood vessel is imaged, the contour line including a first contour line indicating the contour of one side of the blood vessel in the width direction and a second contour line indicating the contour of the other side in the width direction, and is characterized in that the image processing device has: a memory unit that stores the captured image and positional information of contour pixels corresponding to the blood vessel contour line captured in the captured image in association with each other; a contour line selection unit that selects one of the first contour line or the second contour line as a target contour line for correction; an extraction unit that extracts a plurality of contour pixels as extracted pixels from among the contour pixels corresponding to the target contour line based on the positional information of the contour pixels; a pixel selection unit that selects two extraction pixels having a predetermined positional relationship from among the plurality of extracted pixels as selected pixels; a correction pixel determination unit that determines a correction pixel corresponding to the target contour line after correction from among the pixels constituting the captured image based on pixel information of each of a plurality of reference pixels that are arranged at predetermined positions from the two selected selection pixels for each of the two selected selection pixels; and a correction contour line identification unit that identifies the target contour line after correction based on the positional information of the correction pixels. Effect of the Invention

[0009] According to the present invention, it is possible to correct the contour lines of blood vessels extracted from a captured image. [Brief description of the drawings]

[0010] [Figure 1] 1 is a network connection diagram showing an embodiment of an image processing apparatus according to the present invention; [Diagram 2] 1 is a functional block diagram showing an embodiment of an image processing apparatus according to the present invention; [Diagram 3] 3 is a schematic diagram showing an example of a captured image stored in a storage unit included in the image processing device of FIG. 2; [Figure 4] 1 is a flowchart illustrating an embodiment of an image processing method according to the present invention. [Diagram 5] 5 is a flowchart of a primary correction process included in the image processing method of FIG. 4. [Figure 6] 6A to 6C are schematic diagrams showing captured images at each stage of the first correction processing in FIG. 5. [Figure 7] 6 is a schematic diagram showing a captured image in the middle of the first correction process of FIG. 5. FIG. [Figure 8] 6 is a schematic diagram showing an example of luminance values ​​of reference pixels in the first correction process of FIG. 5 and the amount of change therein. [Figure 9] 5 is a flowchart of a secondary correction process included in the image processing method of FIG. 4. [Figure 10] 9A to 9C are schematic diagrams showing captured images at each stage of the secondary correction processing in FIG. 8. [Figure 11] 9 is a schematic diagram showing a virtual circle generated in the secondary correction process of FIG. 8. [Figure 12] FIG. 11 is a functional block diagram showing another embodiment of an image processing device according to the present invention. [Figure 13] 5 is a flowchart illustrating another embodiment of an image processing method according to the present invention. [Figure 14] 13 is a flowchart of a primary correction process included in the image processing method of FIG. 12. [Figure 15] 14A to 14C are schematic diagrams showing captured images at each stage of the first correction processing in FIG. 13. [Figure 16] 14 is a schematic diagram showing a captured image in the middle of the first correction process of FIG. 13. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Below, with reference to the drawings, we will explain embodiments of an image processing device (hereinafter referred to as "this device"), an image processing program (hereinafter referred to as "this program"), and an image processing method (hereinafter referred to as "this method") according to the present invention.

[0012] The present invention corrects a blood vessel contour line generated based on a captured image of the blood vessel. Specifically, the present invention determines pixels (hereinafter referred to as "correction pixels") corresponding to the corrected contour line from among the pixels constituting the captured image based on pixel information of the pixels constituting the captured image, and identifies the corrected contour line based on position information of the correction pixels. The present invention also corrects the position information of the correction pixels based on the curvature of each correction pixel.

[0013] The "captured image" includes, for example, a fundus image captured by a fundus camera, an X-ray image captured by an X-ray imaging device, a CT image captured by a CT device, and also an image captured by a normal optical camera. In the present invention, the captured image includes at least blood vessels as a subject. As described above, in the present embodiment, the captured image is a fundus image.

[0014] The term "blood vessels" includes not only human blood vessels but also blood vessels of animals other than humans. In this embodiment, the blood vessels are blood vessels in the fundus of a human eye.

[0015] A "contour line" is a line tracing the contour of a blood vessel captured in a captured image. In this embodiment, the contour line is generated, for example, by performing known image processing on the captured image. Here, since a blood vessel is a tubular body having a width, the contour line of one blood vessel captured in a captured image includes a contour line showing the contour of one side in the width direction of the blood vessel (hereinafter referred to as a "first contour line") and a contour line showing the contour of the other side in the same width direction (hereinafter referred to as a "second contour line").

[0016] "Pixel information" is information for each pixel. In this embodiment, the pixel information includes the luminance value of the pixel and the amount of change in luminance value between adjacent pixels (luminance gradient).

[0017] "Position information" is information relating to the position of a pixel, that is, information indicating the coordinates of a pixel.

[0018] ●Image processing device(1)● First, an embodiment (first embodiment) of the present device will be described.

[0019] FIG. 1 is a network connection diagram showing an embodiment of the present device. The figure shows that the device 1 is connected to an information storage device 2 via a network (communication line) N that uses a wired communication method or a wireless communication method.

[0020] The present device 1 corrects the contours of blood vessels captured in a captured image i1 based on pixel information of pixels constituting the captured image i1 (see FIG. 3; the same applies below). A specific configuration and operation of the present device 1 will be described later.

[0021] The information storage device 2 stores the captured image i1 in association with position information of pixels corresponding to the contour of the blood vessel captured in the captured image i1 (hereinafter referred to as "contour pixels"). The information storage device 2 is, for example, an information storage server, a NAS, or a personal computer.

[0022] The network N is, for example, a communication network such as the Internet, a mobile communication network, a local area network (LAN), a wide area network (WAN), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] Configuration of image processing device (1) FIG. 2 is a functional block diagram showing an embodiment of the present device 1. As shown in FIG.

[0024] The present device 1 is realized, for example, by a personal computer. The present program runs on the present device 1, and the present program cooperates with the hardware resources of the present device 1 to realize the present method.

[0025] Here, by causing a computer (not shown) to execute this program, the program can cause the computer to function in the same manner as the device 1, and to execute the method.

[0026] The device 1 includes a communication unit 11, a storage unit 12, and a control unit 13.

[0027] The communication unit 11 acquires the captured image i1 from the information storage device 2 via the network N. The communication unit 11 is configured with, for example, a communication module and an antenna. The captured image i1 is stored in the storage unit 12.

[0028] The storage unit 12 stores information necessary for the device 1 to execute the method described below. The storage unit 12 is composed of, for example, a recording device such as a hard disk drive (HDD) or a solid state drive (SSD) provided in the device 1, a volatile memory such as a random access memory (RAM), and / or a portable storage medium such as a flash memory. In this embodiment, the storage unit 12 stores the captured image i1 received in advance from the information storage device 2 in association with position information of the contour pixels of the blood vessel captured in the captured image i1.

[0029] The control unit 13 controls the operation of the entire device 1 and executes the method described below. The control unit 13 is composed of, for example, a CPU (Central Processing Unit) provided in the device 1, a volatile memory such as a RAM that functions as a working area for the CPU, and a non-volatile memory such as a ROM (Read Only Memory) that stores various information such as the program. The control unit 13 includes a contour line selection unit 131, an extraction unit 132, a pixel selection unit 133, a correction pixel determination unit 134, a secondary correction unit 135, and a correction contour line specification unit 136.

[0030] The contour line selection unit 131 selects one of the first contour line and the second contour line from among the contour lines as a contour line to be corrected (hereinafter referred to as a "target contour line"). A specific operation of the contour line selection unit 131 will be described later.

[0031] The extraction unit 132 extracts a plurality of contour pixels as extracted pixels from among the contour pixels that form the target contour (corresponding to the target contour) based on the position information of the contour pixels. A specific operation of the extraction unit 132 will be described later.

[0032] The pixel selection unit 133 selects two extracted pixels having a predetermined positional relationship as selected pixels from among the plurality of extracted pixels. The specific operation of the pixel selection unit 133 and the predetermined positional relationship will be described later.

[0033] The correction pixel determination unit 134 determines a correction pixel corresponding to the target contour after correction from among the pixels constituting the captured image i1, based on pixel information of each of a plurality of pixels (hereinafter referred to as "reference pixels") that are arranged at positions with the same distance from the two selected pixels for each of the two selected pixels. The specific operation of the correction pixel determination unit 134 will be described later.

[0034] The "position at the same distance" is an example of a predetermined position in the present invention, and means a position in the captured image i1 where the linear distance from the two selected pixels is the same. Here, since the captured image i1 is composed of a plurality of pixels, there may be no single pixel at a position where the linear distance from the two selected pixels is completely the same. Therefore, the pixels arranged at the same distance may include not only pixels arranged at positions where the linear distance from the two selected pixels is the same, but also pixels adjacent to the same position (pixels where the linear distance from the two selected pixels is "almost the same"). That is, for example, if there is an even number of pixels (for example, 10) between the two selected pixels, a pixel located 6 pixels away from one selected pixel (a pixel located 4 pixels away from the other selected pixel) may be a pixel where the linear distance from the two selected pixels is the same.

[0035] The secondary correction unit 135 corrects the position information of the correction pixel. A specific operation of the secondary correction unit 135 will be described later.

[0036] The correction contour specifying unit 136 specifies the target contour after correction based on the position information of the correction pixel. The specific operation of the correction contour specifying unit 136 will be described later.

[0037] ● Operation of image processing device (1) Next, the operation of the present device 1, that is, an embodiment of the present method executed by the present device 1, will be described with reference to FIG.

[0038] In the following description, it is assumed that the device 1 acquires the captured image i1 and the position information of the contour pixels corresponding to the contour of the blood vessel captured in the captured image i1 from the information storage device 2 via the network N, and stores them in association with each other in the storage unit 12. It is also assumed that the captured image i1 used in the following description has a contour superimposed thereon. Furthermore, in the following description, it is assumed that the contour is given the symbol "L", the first contour is given the symbol "L1", the second contour is given the symbol "L2", and the target contour is given the symbol "Lt". Furthermore, in the following description, it is assumed that the pixels are given symbols beginning with "P" in order to distinguish between them.

[0039] FIG. 3 is a schematic diagram showing an example of a captured image i1. For ease of explanation, the figure shows an enlarged view of a portion of one blood vessel. The figure shows that in the captured image i1, a contour line L (a first contour line L1 and a second contour line L2) is superimposed on a portion of the captured blood vessel.

[0040] FIG. 4 is a flow chart illustrating an embodiment of the method.

[0041] This method performs a first correction process (S1) and a second correction process (S2).

[0042] ● Primary correction process First, the device 1 executes a primary correction process (S1).

[0043] FIG. 5 is a flowchart of the first correction process (S1). FIG. 6 is a schematic diagram showing a captured image i1 at each stage of the first correction process (S1).

[0044] The "primary correction process (S1)" is a process for correcting position information of contour pixels corresponding to the contour line L associated with the captured image i1, based on pixel information of pixels constituting the captured image i1.

[0045] First, the contour line selection unit 131 selects either the first contour line L1 or the second contour line L2 as the target contour line Lt (S101). That is, for example, the contour line selection unit 131 selects the target contour line Lt based on the positional relationship between the first contour line L1 and the second contour line L2 in the captured image i1 (for example, the distance from the origin (0,0) of the coordinates of the pixels constituting the captured image i1).

[0046] In addition, the contour selection unit of the present invention may select as the target contour a contour selected by a user of the device using a selection means (not shown), or may select the target contour based on its relative positional relationship with the blood vessel (such as up, down, left, right, etc. with respect to the blood vessel).

[0047] Next, the correction pixel determination unit 134 performs a mask process on pixels included in a predetermined area including a contour L (in this embodiment, the second contour L2) that is not selected as the target contour Lt (S102), and excludes the predetermined area from targets for correction pixels, which will be described later. Here, the correction pixel determination unit 134 performs, for example, a known expansion process on the second contour L2, and defines an area (predetermined area) covered by the expanded second contour L2.

[0048] In addition, the predetermined area to be excluded from the target of the correction pixel in the masking process may be a fixed area based on a contour that is not selected as the target contour. Moreover, the masking process may be performed before the calculation of the amount of change in the luminance value of the reference pixel, which will be described later.

[0049] Next, the extraction unit 132 extracts multiple contour pixels corresponding to the target contour line Lt as extracted pixels P1(n) from among the pixels constituting the captured image i1, based on position information of the contour pixels corresponding to the target contour line Lt (S103). That is, for example, the extraction unit 132 extracts a certain number of contour pixels at approximately equal intervals from the multiple contour pixels as extracted pixels P1(n). In the following description, it is assumed that the extraction unit 132 extracts six extracted pixels P1(1) to P1(6).

[0050] The extraction section in the present invention may extract all of the contour pixels corresponding to the target contour as extracted pixels.

[0051] Next, the pixel selection unit 133 selects two adjacent extracted pixels P1(n) and P1(n+1) from among the extracted pixels P1(1) to P1(6) as selected pixels P2(1) and P2(2) (S104). That is, for example, when the six extracted pixels P1(1) to P1(6) are arranged in numerical order, the pixel selection unit 133 selects the extracted pixels P1(1) and P1(2) as selected pixels P2(1) and P2(2).

[0052] Here, in this embodiment, the "adjacent" positional relationship is an example of a predetermined positional relationship in the present invention.

[0053] FIG. 7 is a schematic diagram showing a captured image i1 in the middle of the primary correction process (S1). For ease of explanation, the drawing shows an enlarged view of the vicinity of two selected pixels P2(1) and P2(2).

[0054] Return to Figures 5 to 7. Next, the correction pixel determination unit 134 generates a virtual line segment Lm connecting the two selected pixels P2(1) and P2(2) and generates a virtual perpendicular line Ln perpendicular to the virtual line segment Lm at the midpoint of the virtual line segment Lm (S105). Here, the length of the virtual perpendicular line Ln is preset to a predetermined length (for example, 9 pixels with the midpoint as the center pixel).

[0055] The length of the virtual perpendicular line may be set based on the distance between the first contour line and the second contour line (i.e., the width or caliber of the blood vessel). That is, for example, the length of the virtual perpendicular line may be set to less than twice the distance.

[0056] Next, the correction pixel determination unit 134 specifies, among the pixels constituting the captured image i1, a number of pixels corresponding to the virtual perpendicular line Ln as reference pixels P3(n) (S106). That is, for example, if the length of the virtual perpendicular line Ln is nine pixels, nine pixels are specified as reference pixels P3(1) to P3(9). Here, the pixels corresponding to the virtual perpendicular line Ln are pixels located at positions that are approximately the same distance from the two selected pixels P2(1) and P2(2). That is, the pixels corresponding to the virtual perpendicular line Ln are pixels located at predetermined positions in the present invention.

[0057] Next, the correction pixel determination unit 134 acquires the luminance value of each of the reference pixels P3(1)-P3(9) from the storage unit 12, and acquires the amount of change in luminance value for each of the two adjacent reference pixels P3(n) and P3(n+1) based on the luminance values ​​of each of the reference pixels P3(1)-P3(9) (S107). That is, for example, the correction pixel determination unit 134 calculates the primary luminance gradient and the secondary luminance gradient for each of the reference pixels P3(1)-P3(9).

[0058] It should be noted that the correction pixel determination unit in the present invention may calculate only the first-order luminance gradient, or may calculate the amount of change in luminance value by other calculations.

[0059] 8 is a schematic diagram showing an example of the pixel value of each reference pixel P3(n) and the amount of change (first-order gradient) of the pixel value. The diagram shows that the amount of change in luminance value of the reference pixel P3(6) is the largest.

[0060] Return to Figures 5 to 8. Next, the correction pixel determination unit 134 determines the reference pixel P3(n) having the largest change in luminance value from among the multiple reference pixels P3(1) to P3(9) as the correction pixel P4(n) (S108). In general, blood vessels, which are the subject of the captured image i1, are captured darker than the background. Therefore, pixels with a large change in luminance value can be regarded as pixels corresponding to the edge of the blood vessel.

[0061] Here, the virtual perpendicular line Ln may pass through a contour L (second contour L2) that is not selected as the target contour Lt. Therefore, the reference pixel P3(n) having the largest change in pixel value may also be present near the second contour L2. As a result, the correction pixel determination unit 134 may determine the reference pixel P3(n) near the second contour L2 as the correction pixel P4(n). However, in this embodiment, in the mask process of the process (S102), the reference pixel P3(n) near the second contour L2 is excluded from the correction pixel P4(n). Therefore, in the process (S108), the pixel having the largest change in luminance value is the single reference pixel P3(n).

[0062] The correction pixel determination unit of the present invention does not need to perform the above-mentioned masking process. In this case, the correction pixel determination unit of the present invention determines, for example, the pixel closest to the target contour line among two reference pixels having a large amount of change in pixel value as the correction pixel.

[0063] Next, the control unit 13 (for example, the correction pixel determination unit 134) determines whether there is an extracted pixel P1(n) that has not been selected as the selected pixel P2(1), P2(2) (S109). When there is an extracted pixel P1(n) that has not been selected as the selected pixel P2(1), P2(2) ("Y" in S109), the first correction process (S1) returns to the process (S103). That is, for example, the pixel selection unit 133 selects, as the next selected pixels P2(1), P2(2), an extracted pixel P1(3) adjacent to the extracted pixel P1(2) that has already been selected among the extracted pixels P1(3) to P1(6) that have not been selected, and an extracted pixel P1(2) adjacent to the extracted pixel P1(3) among the extracted pixels P1(1), P1(2) that have already been selected.

[0064] On the other hand, when there is no extracted pixel P1(n) that has not been selected as the selected pixels P2(1), P2(2) ("N" in S109), the correction pixel determination unit 134 stores the position information of all correction pixels P4(1) to P4(5) in the memory unit 12 as position information of a correction pixel group in association with the captured image i1 (S110).

[0065] Next, the control unit 13 (e.g., the correction pixel determination unit 134) determines whether or not there is a contour line L that has not been selected as the target contour line Lt (S111). If there is a contour line L that has not been selected as the target contour line Lt ("Y" in S111), the primary correction process (S1) returns to the process (S101). On the other hand, if there is no contour line L that has not been selected as the target contour line Lt ("N" in S111), the primary correction process (S1) ends.

[0066] ●Secondary correction processing Next, the device 1 executes a secondary correction process (S2).

[0067] FIG. 9 is a flowchart of the secondary correction process (S2). FIG. 10 is a schematic diagram showing a captured image i1 at each stage of the secondary correction process (S2). FIG. 10 shows a vein crossing an artery such that part of the vein is obscured by part of the artery.

[0068] The "secondary correction process (S2)" is a process for correcting the position information of the correction pixel P4(n) determined in the primary correction process (S1). As described above, since blood vessels are imaged darker than the background in the captured image i1, the correction pixel P4(n) determined in the primary correction process (S1) for correction based on the amount of change in luminance value corresponds approximately exactly to the contour of the blood vessel. However, for example, at a crossing point of a blood vessel where an artery and a vein cross, the difference between the luminance value of the artery and the luminance value of the vein is small. Therefore, for example, the correction pixel P4(n) of the vein at the crossing point may protrude toward the artery or recess into the vein. The secondary correction process (S2) is a process for correcting the position information of the correction pixel P4(n) determined in such a region where the difference in luminance value is small, based on a factor (curvature) different from the luminance value.

[0069] First, the secondary correction unit 135 reads out a group of correction pixels to be subjected to secondary correction from the storage unit 12 (S201). In the following description, it is assumed that the group of correction pixels is composed of five correction pixels P4(11) to P4(15).

[0070] Next, the secondary correction unit 135 calculates the curvature corresponding to each of the three correction pixels P4(12) to P4(14) among the correction pixels P4(11) to P4(15) constituting the correction pixel group, excluding the correction pixel P4(11) serving as the start point and the correction pixel P4(15) serving as the end point. Here, the start point and the end point are each determined in advance, for example, based on the determination order of the correction pixel P4(n).

[0071] Specifically, the secondary correction unit 135 selects, from among the correction pixels P4(12) to P4(14), the correction pixel P4(12) that is closest to the starting point and whose curvature has not been calculated, as the calculation pixel P5(1) (S202). Next, the secondary correction unit 135 selects two correction pixels P4(11) and P4(13) that are arranged on both sides of the calculation pixel P5(1) as the calculation auxiliary pixels P5(2) and P5(3) (S203).

[0072] Next, the secondary correction unit 135 generates an imaginary circle C that connects the calculated pixel P5(1) and the two calculated auxiliary pixels P5(2) and P5(3) (S204).

[0073] FIG. 11 is a schematic diagram showing an imaginary circle C. For ease of explanation, the figure also shows parameters used to calculate the radius "R" of the imaginary circle C.

[0074] Return to Figures 9 to 11. Next, the secondary correction unit 135 calculates the radius "R" of the imaginary circle C by using the following equation (1) (S205).

[0075]

number

[0076] Next, the secondary correction unit 135 calculates the curvature "k i " is calculated (S206).

[0077]

number

[0078] The calculated curvature "k i " is stored in the storage unit 12 in association with the corresponding calculated pixel P5(1) (corrected pixel P4(12)).

[0079] Next, the secondary correction unit 135 calculates the curvature "k i It is determined whether or not there is a correction pixel P4(n) for which the curvature "k" has not been calculated (S207). iIf there is a corrected pixel P4(n) for which "" has not been calculated ("Y" in S207), the secondary correction process (S2) returns to the process (S202).

[0080] On the other hand, the curvature “k i When there is no correction pixel P4(n) for which the curvature "k" has not been calculated ("N" in S207), the secondary correction unit 135 i "The average value of "V ave. " and the standard deviation "σ" are calculated (S208).

[0081] Next, the secondary correction unit 135 calculates the average value “V ave. Based on the standard deviation "σ", the curvature "k i In other words, for example, the secondary correction unit 135 determines whether or not there is a correction pixel P4(n) with a relatively large average value "V ave. The curvature "k" that deviates from "±2σ" of the normal distribution centered on i In other words, in this embodiment, the presence or absence of a correction pixel P4(n) having a curvature "k i " is relatively large" is the curvature "k i " is the average value "V ave. " means that it deviates from the "±2σ" of the normal distribution centered on ".

[0082] Curvature “k i If there is a correction pixel P4(n) with a relatively large " ("Y" in S209), the secondary correction unit 135 selects, from among the correction pixels P4(n), the correction pixel P4(n) that is closest to the starting point and has not been selected as the position correction pixel P6(n), as the position correction pixel P6(1) (S210). In the following explanation, it is assumed that the correction pixel P4(13) has been selected as the position correction pixel P6(1).

[0083] Next, the secondary correction unit 135 selects two correction pixels P4(12) and P4(14) arranged on both sides of the position correction pixel P6(1) as reference pixels P7(1) and P7(2) (S211).

[0084] Next, the secondary correction unit 135 calculates position information of a pixel Po corresponding to the midpoint between the reference pixels P7(1) and P7(2) (hereinafter referred to as the "midpoint pixel") (S212).

[0085] Next, the secondary correction unit 135 corrects the position information of the position correction pixel P6(1) to the position information of the midpoint pixel Po (S213). That is, the secondary correction unit 135 moves the position of the correction pixel P4(13) corresponding to the position correction pixel P6(1) to the position of the midpoint pixel Po. That is, the secondary correction unit 135 corrects the position information of the position correction pixel P6(1) based on the position information of the reference pixels P7(1) and P7(2).

[0086] Next, the secondary correction unit 135 determines whether or not there is a correction pixel P4(n) that has not been selected as the position correction pixel P6(n) (S214). When there is a correction pixel P4(n) that has not been selected as the position correction pixel P6(n) ("Y" in S214), the secondary correction process (S2) returns to process (S210).

[0087] On the other hand, the curvature “k i When there is no correction pixel P4(n) with a relatively large " ("N" in S209), or when there is no correction pixel P4(n) that has not been selected as the position correction pixel P6(n) ("N" in S214), the secondary correction unit 135 stores the position information of the correction pixel P4(n) whose position information has not been corrected and the position information of the correction pixel P4(n) whose position information has been corrected in the memory unit 12 as position information of the pixels corresponding to the corrected contour line Lx (S215).

[0088] Next, the corrected contour line specifying unit 136 specifies the corrected contour line Lx based on the position information of the pixels corresponding to the corrected contour line Lx (S216).

[0089] In this manner, in the first correction process (S1), the device 1 determines the correction pixel P4(n) based on the pixel information of the extracted pixel P1(n) extracted from the contour line L to correct the position information of the contour line L, and in the second correction process (S2), it further determines the curvature "k iIn other words, the device 1 corrects the position information of the correction pixel P4(n) based on the pixel information and the curvature "k i " The position information of the contour line L is corrected in two stages based on these two different pieces of information. As a result, the present device 1 can correct the contour line L of the blood vessel extracted from the captured image i1 with high accuracy.

[0090] Summary (1) According to the embodiment described above, the device 1 includes a contour selection unit 131 that selects either the first contour line L1 or the second contour line L2 as a target contour line Lt to be corrected, an extraction unit 132 that extracts a plurality of contour pixels as extracted pixels P1(n) from among the contour pixels corresponding to the target contour line Lt based on position information of the contour pixels, a pixel selection unit 133 that selects two extraction pixels P1(n) having a predetermined positional relationship from the plurality of extraction pixels P1(n) as selected pixels P2(1), P2(2), a correction pixel determination unit 134 that determines, for each of the two selected selection pixels P2(1), P2(2), a correction pixel determination unit 134 that determines a correction pixel P4(n) corresponding to the target contour line Lt after correction from among the pixels constituting the captured image i1 based on pixel information of each of a plurality of reference pixels P3(n) that are arranged at predetermined positions from the two selection pixels P2(1), P2(2), and a correction contour identification unit 136 that identifies the target contour line Lt after correction based on the position information of the correction pixel P4(n). According to this configuration, the device 1 can determine one correction pixel P4(n) for each of two extracted pixels P1(n) based on pixel information (amount of change in luminance value) of the reference pixel P3(n). Then, the device 1 can specify the corrected target contour line Lt (contour line L) based on position information of the determined correction pixel P4(n). In other words, the device 1 can correct the contour line L of the blood vessel extracted from the captured image i1.

[0091] Moreover, according to the embodiment described above, the pixel information includes the luminance value of the reference pixel P3(n). Furthermore, the correction pixel determination unit 134 obtains the amount of change in luminance value between two adjacent reference pixels P3(n) based on the luminance value of each of the plurality of reference pixels P3(n), and determines the reference pixel P3(n) with the largest amount of change as the correction pixel P4(n) among the plurality of reference pixels P3(n). Usually, in the captured image i1, the blood vessels, which are the subject of the image, are captured darker than the background. Therefore, pixels with a large amount of change in pixel value can be regarded as pixels corresponding to the edge of the blood vessel. According to this configuration, the present device 1 can easily identify pixels corresponding to the edge of the blood vessel (if relevant, pixels of the corrected contour line Lx).

[0092] Furthermore, according to the embodiment described above, when the first contour line L1 is selected, the correction pixel determination unit 134 excludes pixels included in a predetermined area including the second contour line L2 from the targets of the correction pixel P4(n), and when the second contour line L2 is selected, the correction pixel determination unit 134 excludes pixels included in a predetermined area including the first contour line L1 from the targets of the correction pixel P4(n). According to this configuration, when calculating the amount of change in luminance value in the virtual perpendicular line Ln, the change in luminance value near the contour line L that is not selected as the target contour line Lt is not affected. As a result, the pixel with the largest amount of change in luminance value becomes the single reference pixel P3(n), and can be easily identified.

[0093] Furthermore, according to the embodiment described above, the correction pixel determination unit 134 defines the predetermined area by performing an expansion process on the first contour line L1 or the second contour line L2. With this configuration, the device 1 can easily determine the predetermined area to be masked.

[0094] Furthermore, according to the embodiment described above, the present device 1 has a secondary correction unit 135 that corrects the position information of the correction pixel P4(n). The secondary correction unit 135 calculates a curvature corresponding to each of the multiple correction pixels P4(n), and corrects the position information of the correction pixel P4(n) based on the curvature of each of the correction pixels P4(n). With this configuration, the present device 1 can further correct the position information of the correction pixel P4(n) in a portion where there is no significant difference in the amount of change in luminance value and where the position information of the correction pixel P4(n) is likely to be erroneous. In other words, the present device 1 can correct the contour line L with high accuracy. In addition, the present device 1 can identify the correction pixel P4(n) whose position information is to be corrected based on the curvature, i.e., the difference in relative position information with respect to other correction pixels P4(n).

[0095] Furthermore, according to the embodiment described above, the secondary correction unit 135 selects the correction pixel P4(n) to be corrected as the position correction pixel P6(n) based on the calculated curvature. With this configuration, the device 1 can identify the correction pixel P4(n) whose relative position information with respect to other correction pixels P4(n) is significantly shifted based on the curvature, and correct only the position information of the correction pixel P4(n). In addition, the secondary correction unit 135 selects the two correction pixels P4(n) arranged on both sides of the position correction pixel P6(n) as the reference pixels P7(1) and P7(2), and corrects the position information of the position correction pixel P6(n) based on the position information of the two reference pixels P7(1) and P7(2). According to this configuration, the present device 1 can correct the position information of the position correction pixel P6(n) based on the position information of pixels (reference pixels P7(1), P7(2)) whose position information is not shifted from the blood vessel contour more than the position correction pixel P6(n) and whose position information is closer to the correct position of the position correction pixel P6(n). That is, the present device 1 can correct the blood vessel contour line L extracted from the captured image i1 with high accuracy.

[0096] Furthermore, according to the embodiment described above, the secondary correction unit 135 selects the correction pixel P4(n) with a relatively large curvature as the position correction pixel P6(n). With this configuration, the device 1 can identify only the correction pixel P4(n) whose relative position information with respect to other correction pixels P4(n) is significantly deviated based on the curvature, and correct only the position information of the correction pixel P4(n).

[0097] Furthermore, according to the embodiment described above, the secondary correction unit 135 selects, from among the multiple correction pixels P4(n), the correction pixel P4(n) for which curvature is to be calculated as the calculation pixel P5(1), selects the two correction pixels P4(n) arranged on both sides of the calculation pixel P5(1) as the calculation auxiliary pixels P5(2) and P5(3), and calculates the curvature of the calculation pixel P5(1) based on the radius of a virtual circle C that virtually connects the calculation pixel P5(1) and the two calculation auxiliary pixels P5(2) and P5(3). According to this configuration, the device 1 can easily calculate the curvature based only on the position information of the correction pixel P4(n).

[0098] ●Image processing device(2)● Next, another embodiment (second embodiment) of the present device will be described, focusing on the differences from the first embodiment described above. The present device of the second embodiment differs from the first embodiment in the pixel selection unit and correction pixel determination unit of the control unit.

[0099] ●Configuration of image processing device (2) FIG. 12 is a functional block diagram showing another embodiment of the present device.

[0100] The device 1A includes a communication unit 11, a storage unit 12, and a control unit 13A.

[0101] The control unit 13A controls the operation of the entire device 1A and executes the method described below. The physical configuration of the control unit 13A is the same as that of the control unit 13 in the first embodiment. The control unit 13A includes a contour line selection unit 131, an extraction unit 132, a pixel selection unit 133A, a correction pixel determination unit 134A, a secondary correction unit 135, and a correction contour line specification unit 136.

[0102] The pixel selection unit 133A selects two extracted pixels having a predetermined positional relationship as selected pixels from among the extracted pixels. The specific operation of the pixel selection unit 133A and the predetermined positional relationship will be described later.

[0103] The correction pixel determination unit 134A determines, for each of the two selected pixels, a correction pixel corresponding to the target contour line after correction from among the pixels constituting the captured image i1, based on pixel information of each of a plurality of pixels (reference pixels) arranged at positions at approximately the same distance from the two selected pixels. A specific operation of the correction pixel determination unit 134A will be described later.

[0104] ● Operation of image processing device (2) Next, the operation of the present device 1A, that is, another embodiment of the present method executed by the present device 1A will be described with reference to FIG.

[0105] FIG. 13 is a flow chart illustrating another embodiment of the method.

[0106] This method executes a first correction process (S1A) and a second correction process (S2). In this embodiment, the second correction process (S2) is common to the second correction process (S2) in the first embodiment.

[0107] ● Primary correction process First, the device 1A executes a first correction process (S1A).

[0108] FIG. 14 is a flowchart of the first correction process (S1A). FIG. 15 is a schematic diagram showing a captured image i1 at each stage of the first correction process (S1A).

[0109] The "primary correction process (S1A)" is a process for correcting position information of contour pixels corresponding to the contour line L associated with the captured image i1, based on pixel information of pixels constituting the captured image i1.

[0110] First, the contour line selection unit 131 selects either the first contour line L1 or the second contour line L2 as a target contour line Lt (S101).

[0111] Next, the extraction unit 132 extracts multiple contour pixels corresponding to the target contour line Lt as extracted pixels P1(n) from among the pixels constituting the captured image i1, based on position information of the contour pixels corresponding to the target contour line Lt (S102A). That is, for example, the extraction unit 132 extracts a certain number of contour pixels at approximately equal intervals from the multiple contour pixels as extracted pixels P1(n). Here, the process (S102A) is the same as the process (S103) in the first embodiment. In the following description, it is assumed that the extraction unit 132 extracts six extracted pixels P1(1) to P1(6).

[0112] Next, the pixel selection unit 133A selects an extracted pixel P1(n) that is a candidate for correction from among the extracted pixels P1(1) to P1(6) as a candidate pixel P8(n) (S103A). In this method, the candidate pixel P8(n) is selected from the four extracted pixels P1(2) to P1(5) excluding the extracted pixel P1(1) that is the starting point and the extracted pixel P1(6) that is the end point. Specifically, the pixel selection unit 133A selects, from among the extracted pixels P1(2) to P1(5), the extracted pixel P1(2) that is closest to the starting point and has not been selected as the candidate pixel P8(n), as the candidate pixel P8(1).

[0113] Next, the pixel selection unit 133A selects two extracted pixels P1(1) and P1(3) located on both sides of the candidate pixel P8(1) as selected pixels P2A(1) and P2A(2) (S104A). That is, the pixel selection unit 133A selects two extracted pixels P1(n) and P1(n+2) located on both sides of the candidate pixel P8(n) (every other pixel) from among the extracted pixels P1(1) to P1(6) as selected pixels P2A(1) and P2A(2) (S104A).

[0114] In this embodiment, the positional relationship of "both sides of the candidate pixel P8(n) (every other pixel)" is an example of a predetermined positional relationship in the present invention.

[0115] FIG. 16 is a schematic diagram showing a captured image i1 in the middle of the first correction process (S1A). For ease of explanation, the drawing shows an enlarged view of the vicinity of two selected pixels P2A(1) and PA2(2).

[0116] Return to Figures 14 to 16. Next, the correction pixel determination unit 134A generates a virtual line segment Lm connecting the two selected pixels P2A(1) and P2A(2) and generates a virtual perpendicular line Ln perpendicular to the virtual line segment Lm at the midpoint of the virtual line segment Lm (S105). Here, the length of the virtual perpendicular line Ln is preset to a predetermined length (for example, 9 pixels with the midpoint as the center pixel).

[0117] Next, the correction pixel determination unit 134A specifies, among the pixels constituting the captured image i1, a plurality of pixels corresponding to the virtual perpendicular line Ln as reference pixels P3(n) (S106). That is, for example, if the length of the virtual perpendicular line Ln is nine pixels, nine pixels are determined as reference pixels P3(1) to P3(9). Here, the pixels corresponding to the virtual perpendicular line Ln are pixels disposed at positions approximately the same distance from the two selected pixels P2A(1) and P2A(2). That is, the pixels corresponding to the virtual perpendicular line Ln are pixels disposed at predetermined positions in the present invention.

[0118] Next, the correction pixel determination unit 134A acquires the luminance value of each of the reference pixels P3(1)-P3(9) from the storage unit 12, and acquires the amount of change in luminance value for each of the two adjacent reference pixels P3(n) and P3(n+1) based on the luminance values ​​of each of the reference pixels P3(1)-P3(9) (S107). That is, for example, the correction pixel determination unit 134 calculates the primary luminance gradient and the secondary luminance gradient for each of the reference pixels P3(1)-P3(9).

[0119] Next, the correction pixel determination unit 134A determines, from among the multiple reference pixels P3(1) to P3(9), the reference pixel P3(n) that is closest to the target contour and has the largest amount of change, as the correction pixel P4(n) (S108).

[0120] The correction pixel determination unit in the present invention may execute a mask process similar to the present method in the first embodiment.

[0121] Next, the control unit 13A (for example, the correction pixel determination unit 134A) determines whether or not there is an extracted pixel P1(n) that has not been selected as a candidate pixel P8(n) (S109A). If there is an extracted pixel P1(n) that has not been selected ("Y" in S109A), the first correction process (S1A) returns to the process (S103A). That is, for example, the pixel selection unit 133A selects the extracted pixel P1(3) adjacent to the extracted pixel P1(2) that has already been selected, from among the extracted pixels P1(3) to P1(5) that have not been selected, as the next candidate pixel P8(2).

[0122] On the other hand, when there is no unselected extracted pixel P1(n) ("N" in S109A), the correction pixel determination unit 134A corrects the position information of all candidate pixels P8(1)-P8(4) (i.e., extracted pixels P1(2)-P1(5)) to the position information of the corresponding correction pixels P4(1)-P4(4) (S110A). In other words, the correction pixel determination unit 134A moves the position of the extracted pixel P1(n) corresponding to the candidate pixel P8(n) to the position of the correction pixel P4(n).

[0123] Next, the correction pixel determination unit 134A stores the position information of all the correction pixels P4(2) to P4(5) in the storage unit 12 as position information of a correction pixel group in association with the captured image i1 (S111A).

[0124] Next, the control unit 13A (e.g., the correction pixel determination unit 134A) determines whether or not there is an unselected contour line L as the target contour line Lt (S112A). If there is an unselected contour line L ("Y" in S112A), the first correction process (S1A) returns to process (S101). On the other hand, if there is no unselected contour line L ("N" in S112A), the first correction process (S1A) ends. Here, the process (S112A) is common to the process (S111) in the first embodiment.

[0125] Summary (2) According to the embodiment described above, the pixel selection unit 133 selects, from among a plurality of extracted pixels P1(n), an extracted pixel P1(n) that is a candidate for correction as a candidate pixel P8(n), and selects two extracted pixels P1(n) arranged on both sides of the candidate pixel P8(n) as selected pixels P2A(1) and P2A(2). The correction pixel determination unit 134 corrects the position information of the candidate pixel P8(n) to the position information of the correction pixel P4(n). According to this configuration, the present device 1 can correct the position information itself of the extracted pixel P1(n).

[0126] ●Summary (Other) In each of the above-described embodiments, the device 1, 1A executes the secondary correction process (S2). Alternatively, the device does not need to execute the secondary correction process if the contour line can be corrected with a desired accuracy in the primary correction process.

[0127] Furthermore, in the primary correction process, the present device may combine the selection of selected pixels in the first embodiment with the selection of selected pixels in the second embodiment. That is, for example, for extracted pixels P1(1) to P1(6), the present device may select extracted pixel P1(1) corresponding to the start point and extracted pixel P1(2) adjacent to extracted pixel P1(1) as selected pixels, and then select extracted pixel P1(2) and every other extracted pixel P1(4) as selected pixels. [Explanation of symbols]

[0128] 1 Image processing device 12 Storage section 131 Contour Selection Section 132 Extraction part 133 Pixel selection unit 134 Correction pixel determination unit 135 Secondary correction section 136 Correction contour line specification section 1A Image processing device 133A Pixel selection unit 134A Correction pixel determination unit

Claims

1. 1. An image processing device that corrects a contour line of a blood vessel generated based on a captured image of the blood vessel, comprising: The contour line is A first contour line indicating a contour of one side of the blood vessel in a width direction; A second contour line indicating the other contour in the width direction; Including, a storage unit that stores the captured image and position information of contour pixels corresponding to the contour of the blood vessel captured in the captured image in association with each other; a contour line selection unit that selects one of the first contour line and the second contour line as a contour line to be corrected; an extracting unit that extracts, as extracted pixels, a plurality of the contour pixels from among the contour pixels corresponding to the target contour line based on position information of the contour pixels; a pixel selection unit that selects, from among the plurality of extracted pixels, two extracted pixels having a predetermined positional relationship as selected pixels; a correction pixel determination unit that determines, for each of the two selected selection pixels, a correction pixel corresponding to the target contour after correction from among pixels constituting the captured image, based on pixel information of each of a plurality of reference pixels that are arranged at predetermined positions from the two selected pixels; a correction contour specifying unit that specifies the target contour after correction based on position information of the correction pixels; and The correction pixel determination unit When the first contour line is selected, pixels included in a predetermined area including the second contour line are excluded from the pixels to be corrected; When the second contour line is selected, pixels included in a predetermined area including the first contour line are excluded from the pixels to be corrected.

13. An image processing device comprising:

2. The pixel information is the luminance value of the reference pixel; Including, The correction pixel determination unit obtaining an amount of change in the luminance value between two adjacent reference pixels based on the luminance values ​​of the plurality of reference pixels; determining, as the correction pixel, the reference pixel having the largest amount of change among the plurality of reference pixels; 2. The image processing device according to claim 1.

3. The pixel selection unit is selecting two adjacent extracted pixels from the plurality of extracted pixels as the selected pixels; 3. The image processing device according to claim 2.

4. The pixel selection unit is selecting, from among the plurality of extracted pixels, the extracted pixel that is a candidate for correction as a candidate pixel, and selecting two extracted pixels disposed on both sides of the candidate pixel as the selected pixels; 3. The image processing device according to claim 2.

5. the correction pixel determination unit corrects position information of the candidate pixel to position information of the correction pixel.

5. The image processing device according to claim 4.

6. a secondary correction unit that corrects position information of the correction pixel; and The secondary correction unit is Calculating a curvature corresponding to each of the plurality of correction pixels; correcting position information of the correction pixels based on the curvature of each of the correction pixels; 6. The image processing device according to claim 1.

7. the secondary correction unit calculates the curvature corresponding to each of the correction pixels, excluding the correction pixels corresponding to a start point and an end point of the target contour after correction, among the plurality of correction pixels; 7. The image processing device according to claim 6.

8. The secondary correction unit is selecting the correction pixel to be corrected as a position correction pixel based on the calculated curvature; Selecting two of the correction pixels disposed on both sides of the position correction pixel as reference pixels; correcting the position information of the position correction pixel based on the position information of the two reference pixels; 8. The image processing device according to claim 7.

9. the secondary correction unit selects the correction pixel having a relatively large curvature as the position correction pixel; 9. The image processing device according to claim 8.

10. the secondary correction unit corrects position information of the position correction pixel to position information of a pixel corresponding to a midpoint of the two reference pixels; 9. The image processing device according to claim 8.

11. The secondary correction unit is selecting the correction pixel for calculating the curvature as a calculation pixel from among the plurality of correction pixels; selecting two of the correction pixels disposed on both sides of the calculation pixel as calculation auxiliary pixels; calculating the curvature of the calculation pixel based on a radius of a virtual circle virtually connecting the calculation pixel and the two calculation auxiliary pixels; 7. The image processing device according to claim 6.

12. A computer is caused to function as the image processing device according to claim 1.

2. An image processing program comprising:

13. 1. An image processing method executed by an image processing device for correcting a contour line of a blood vessel generated based on an image of the blood vessel, the method comprising: The contour line is A first contour line indicating a contour of one side of the blood vessel in a width direction; A second contour line indicating the other contour in the width direction; Including, The image processing device includes: a storage unit that stores the captured image and position information of contour pixels corresponding to the contour of the blood vessel captured in the captured image in association with each other; Equipped with The image processing device, selecting one of the first contour line and the second contour line as a contour line to be corrected; excluding pixels included in a predetermined area including the contour that is not selected as the target contour from the targets of correction pixels corresponding to the target contour after correction; extracting, as extracted pixels, a plurality of the contour pixels from among the contour pixels corresponding to the target contour line based on position information of the contour pixels; selecting, as selected pixels, two of the extracted pixels having a predetermined positional relationship from among the plurality of extracted pixels; determining, for each of the two selected selection pixels, the correction pixel from among pixels constituting the captured image, based on pixel information of each of a plurality of reference pixels arranged at predetermined positions from the two selected pixels; identifying the corrected target contour based on position information of the correction pixels; Including, 13. An image processing method comprising:

14. correcting position information of the correction pixel; Including, The image processing method according to claim 13.

Images