Capillary vessel imaging system, server device for capillary vessel imaging system, and capillary vessel imaging program
The capillary imaging system addresses the challenge of capturing clear capillary images by using focus evaluation and selection methods, enabling accurate health assessments through well-focused image capture.
Patent Information
- Application Number
- JP2022023939
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2022-02-18
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2042-02-18
AI Technical Summary
Capturing clear images of capillaries is challenging due to the difficulty in keeping subjects still during imaging, leading to out-of-focus images that reduce diagnosis accuracy.
A capillary imaging system that includes an imaging unit, image acquisition unit, and evaluation units to quantify focus, select focused images, and adjust focus based on individual determination thresholds, allowing for the capture of well-focused capillary images.
The system effectively selects and captures well-focused capillary images, improving the accuracy of health condition assessments by ensuring clear capillary images are obtained despite subject movement.
Smart Images

Figure 0007589893000001 
Figure 0007589893000002 
Figure 0007589893000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a capillary imaging system for imaging capillaries, a server device for the capillary imaging system, and a capillary imaging program. [Background technology]
[0002] The capillaries of the human body generally change due to the living environment and external factors. Therefore, by observing the state of the subcutaneous capillaries in fingertips, etc., or the capillaries in the fundus of the eye, or by analyzing the capillary images by image processing and digitization, etc., it is possible to grasp the health condition of the subject. Capillaries are extremely tiny (a few dozen μm in diameter, a hundred and several tens of μm in length, etc.) and cannot be seen with the naked eye, but they can be observed using an optical microscope or a digital microscope.
[0003] In view of this, a fingertip blood vessel observation device is known that diagnoses the health condition from an image of the capillaries in the fingertip (see, for example, Patent Document 1). [Prior art documents] [Patent documents]
[0004] [Patent Document 1] Patent No. 6186663 Summary of the Invention [Problem to be solved by the invention]
[0005] Incidentally, when photographing capillaries of living organisms, including humans and animals, it is difficult to keep the subject, such as a finger, completely still because the subject is alive. Therefore, slight movements of the subject cannot be avoided. On the other hand, microscopes have high magnification and a narrow focusing range, so even slight movements of the subject will cause it to become out of focus. If health conditions are diagnosed based on low-quality images that are out of focus, the accuracy of the diagnosis will decrease.
[0006] An object of the present invention is to provide a capillary blood vessel imaging system, a server device for the capillary blood vessel imaging system, and a capillary blood vessel imaging program, which make it easy to obtain capillary blood vessel images with a good degree of focus. [Means for solving the problem]
[0007] The capillary imaging system of the present invention includes an image acquisition unit that acquires multiple images of capillaries of an organism from an imaging unit that captures the multiple images, a focus evaluation value calculation unit that calculates a focus evaluation value that indicates the degree of focus in a numerical value for each of the multiple images, and a focused image selection unit that selects an image with a good degree of focus from the multiple images as a focused image by comparing each focus evaluation value with a judgment threshold for determining whether the degree of focus is good or not.
[0008] According to this configuration, an image with a good degree of focus can be selected as a focused image from among multiple images obtained by capturing multiple images of the capillaries of a living organism. This makes it easy to obtain an image of the capillaries with a good degree of focus even if the living measurement subject moves.
[0009] In addition, it is preferable that the image processing device further includes a judgment threshold setting unit that executes a judgment threshold setting process to set the focus evaluation value that indicates the best focus among the multiple focus evaluation values calculated by the focus evaluation value calculation unit as the judgment threshold, and that the focused image selection unit selects, after the judgment threshold setting process, an image having the best degree of focus from among the multiple images newly acquired by the image acquisition unit as the focused image.
[0010] When using an image of capillaries for diagnosis, a capillary image with good overall image quality is required within a certain area where the characteristic shape of the capillaries can be grasped. Therefore, it is necessary to judge the degree of focus overall within a certain area, but the shape and number of capillaries vary from person to person and from animal to animal. Therefore, if the judgment threshold is set to a fixed value, it may not be possible to properly judge whether the degree of focus is good or not. Therefore, the focus evaluation value that shows the best focus among multiple focus evaluation values obtained from multiple images captured of an individual to be imaged is set as the judgment threshold, and then a focused image is selected from multiple images captured of the same individual based on the judgment threshold set for that individual, thereby improving the accuracy of selecting an image with a good degree of focus as a focused image.
[0011] It is preferable that the focused image selection section further selects, as the focused image, an image for which a focus evaluation value indicating the best focus is obtained in the determination threshold setting process.
[0012] The image for which the focus evaluation value indicating the best focus is obtained in the determination threshold setting process is considered to be an image with a good degree of focus, and therefore, by adding this image as a focused image, the number of focused images can be increased.
[0013] It is also preferable that the image capturing device further includes an evaluation target area receiving unit that receives a designation of any area range within the imaging range of the imaging unit as an evaluation target area, and the focus evaluation value calculation unit calculates each of the focus evaluation values for the evaluation target areas in the multiple images.
[0014] According to this configuration, the user can set the area to be evaluated. For example, by setting a part that is important for evaluating the health condition, such as the vicinity of the tip of a capillary, as the area to be evaluated, it is possible to select an image with good focus on the part that is important for evaluating the health condition as the focused image.
[0015] It is also preferable that the imaging unit allows a user to adjust the focus, and further includes a focus evaluation value display unit that displays the focus evaluation value calculated by the focus evaluation value calculation unit in real time during an imaging period by the imaging unit.
[0016] According to this configuration, when the user adjusts the focus, the user can adjust the focus while checking the focus evaluation value displayed in real time, which facilitates better focusing operations.
[0017] In addition, it is preferable that the imaging unit allows a user to adjust the focus, and further includes an operation guide notification unit that notifies the user of operation guide encouraging the user to adjust the focus, and that the imaging unit repeatedly captures the image during a period during which the operation guide is being notified.
[0018] According to this configuration, since a plurality of images are captured during the period in which the user is adjusting the focus in response to the operation guidance, the possibility of capturing an image with a better degree of focus is increased.
[0019] It is also preferable that the image sensor further includes a display unit that displays an image captured by the imaging unit in real time, and the focus evaluation value calculation unit determines the degree of focus of each pixel of the image captured by the imaging unit in real time, and causes the display unit to highlight focused pixels, which are pixels whose degree of focus exceeds a predetermined reference level.
[0020] With this configuration, the user can know which part is in focus.
[0021] It is preferable that the imaging device further comprises a best image selection section which selects one of the plurality of focused images as a best image based on the focus evaluation value of each of the focused images.
[0022] According to this configuration, the best image is selected from the plurality of focused images based on the focus evaluation value, so that an image with a better degree of focus can be obtained.
[0023] a blur evaluation value calculation unit that calculates a blur evaluation value that indicates the degree of blur of each of the focused images;
[0024] It is preferable that the imaging device further comprises a best image selection section which further selects one of the plurality of focused images as a best image based on a blur evaluation value of each of the focused images.
[0025] According to this configuration, the best image is selected from the plurality of focused images based on the blur evaluation value, so that a good image with less blur can be obtained.
[0026] It is also preferable that the image processing device further comprises a reflection evaluation value calculation unit which calculates a reflection evaluation value which indicates the degree of reflection of a light source in each of the focused images, and a best image selection unit which further selects one of the plurality of focused images as a best image based on the reflection evaluation value of each of the focused images.
[0027] According to this configuration, the best image is selected from among the multiple focused images based on the reflection evaluation value, so that a good image with even less reflection of the light source can be obtained.
[0028] It is also preferable to further provide a best image selection unit that selects one of the multiple focused images as the best image based on at least one of a focus evaluation value of each focused image, a blur evaluation value indicating the degree of blur of each focused image, and a reflection evaluation value indicating the degree of reflection of a light source in each focused image.
[0029] According to this configuration, the best image is selected from the plurality of focused images based on at least one of the focus evaluation value, the blur evaluation value, and the reflection evaluation value, so that a better image can be obtained.
[0030] It is also preferable that the imaging device further comprises a health condition evaluation unit that evaluates a health condition of the living organism based on the focused image.
[0031] According to this configuration, the health condition of the living organism can be evaluated based on a well-focused image of the capillaries, thereby improving the accuracy of the health condition evaluation.
[0032] It is also preferable that the imaging device further comprises a health condition assessment unit for assessing a health condition of the living organism based on the best image.
[0033] According to this configuration, the health condition of the living organism can be evaluated based on a better focused image of the capillaries, thereby improving the accuracy of the health condition evaluation.
[0034] It is also preferable that the imaging section captures the images as frames of a moving image.
[0035] According to this configuration, a plurality of images can be captured by capturing moving images of the capillaries of the living organism.
[0036] It is also preferable that the focus evaluation value calculation unit calculates the focus evaluation value each time the image is captured by the imaging unit, and the focused image selection unit selects the image each time the focus evaluation value is calculated.
[0037] According to this configuration, since the in-focus image is selected in real time during the period in which the imaging section captures a plurality of images, the time from when the images are captured until when the in-focus image is selected is shortened.
[0038] It is also preferable that the imaging unit be further provided.
[0039] According to this configuration, the imaging unit is included in the capillary blood vessel imaging system.
[0040] In addition, the server device for a capillary imaging system according to the present invention includes a focused image receiving unit that receives input of a plurality of focused images, which are images of capillaries of a living organism, and a best image selection unit that selects one of the plurality of focused images as a best image based on at least one of a focus evaluation value that indicates the degree of focus of each focused image by quantifying it, a blur evaluation value that indicates the degree of blur of each focused image, and a reflection evaluation value that indicates the degree of reflection of a light source in each focused image.
[0041] According to this configuration, the best image is selected from the plurality of focused images based on at least one of the focus evaluation value, the blur evaluation value, and the reflection evaluation value, so that a better image can be obtained.
[0042] A capillary imaging program according to the present invention causes a computer to function as the above-mentioned capillary imaging system.
[0043] This capillary imaging program enables a computer to function as the above-mentioned capillary imaging system.
[0044] A capillary blood vessel imaging program according to the present invention causes a computer to function as the server device for the capillary blood vessel imaging system described above.
[0045] According to this capillary blood vessel imaging program, a computer can function as the server device for the capillary blood vessel imaging system described above. Effect of the Invention
[0046] The capillary blood vessel imaging system, the server device for the capillary blood vessel imaging system, and the capillary blood vessel imaging program configured as above can easily obtain capillary blood vessel images with a good degree of focus. [Brief description of the drawings]
[0047] [Figure 1] 1 is a block diagram showing an example of the configuration of a capillary blood vessel imaging system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is an external perspective view showing an example of the microscope camera shown in FIG. [Diagram 3] 2 is a flowchart showing an example of an operation of the imaging processing device shown in FIG. [Figure 4] 2 is a flowchart showing an example of an operation of the imaging processing device shown in FIG. [Diagram 5] 2 is a flowchart showing an example of the operation of the server device shown in FIG. [Figure 6] 2 is an explanatory diagram showing an example of a captured image displayed on the display shown in FIG. 1; [Figure 7] 2 is a screen diagram showing an example of a display screen of the display shown in FIG. 1. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0048] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In addition, components with the same reference numerals in each drawing are the same components, and the description thereof will be omitted. Fig. 1 is a block diagram showing an example of the configuration of a capillary blood vessel imaging system according to an embodiment of the present invention.
[0049] 1 includes a microscope camera 2 (imaging section), an imaging processing device 3, and a server device 4 (server device for the capillary imaging system). The imaging processing device 3 and the server device 4 are communicatively connected via a network 5 so as to be able to transmit and receive data.
[0050] The microscope camera 2 is a camera that captures images of capillaries in the human body using a microscope. Fig. 2 is an external perspective view showing an example of the microscope camera 2 shown in Fig. 1. The microscope camera 2 shown in Fig. 2 roughly comprises a roughly cylindrical microscope body 21, a roughly plate-shaped base 22, a connecting part 23 that connects the microscope body 21 and the base 22, and an adjustment handle 24 that adjusts the angle of the microscope body 21.
[0051] A guide section 28 for guiding a person's fingertip is provided on base 22. A recess 281 is formed in guide section 28 so as to be recessed to fit the shape of the fingertip. When the subject places his / her fingertip in recess 281, the fingertip is roughly positioned.
[0052] An objective lens 25 is provided at the bottom end of microscope body 21, and is adapted to capture an image of the subject's fingertip placed in recess 281. An illumination unit 26 is attached around objective lens 25 to illuminate the subject's fingertip.
[0053] The microscope body 21 contains an optical system of the microscope and an imaging element such as a CMOS (Complementary Metal Oxide Semiconductor) or a CCD (Charge Coupled Device) that captures the image obtained by the microscope. The microscope body 21 captures an image of the subject's fingertip, more specifically, an image of the capillaries in the fingertip. The microscope body 21 may capture multiple still images of the capillaries continuously, or may capture a video including each image as a frame. The microscope body 21 transmits image data representing the image captured by the imaging element to the imaging processing device 3 via a cable 29.
[0054] As the microscope camera 2, for example, a capillary scope SC-10 manufactured by Atto Co., Ltd. can be suitably used. The microscope camera 2 is not limited to one that captures images of fingertips as long as it can capture images of capillaries. The microscope camera 2 is not limited to one that captures images of human capillaries, and may be one that captures images of capillaries of living organisms such as animals.
[0055] Various formats, such as the NTSC (National Television System Committee) format, can be used as the format of the image data to be transmitted to the imaging processing device 3. The image data may also be transmitted to the imaging processing device 3 by wireless signals.
[0056] The microscope body 21 is provided with a focus adjustment handle 27. A user can adjust the focus by operating the focus adjustment handle 27.
[0057] The focal point is the distance at which an image from an optical system such as a lens can be seen clearly, and is also called the focal point or focus. When an image is in focus, it is called "in focus." When an image is not in focus, it is called "out of focus" or "out of focus." The distance range at which a lens can focus is called the depth of field, and the depth of field becomes particularly shallow (narrow) when the shooting distance is short or the magnification of the optical system is high (e.g., when the focal length is long), and generally the value of the depth of field is only about 100 μm. For this reason, when using a microscope, if the subject moves slightly (relative to the shooting distance), the image will be out of focus.
[0058] Blurring also includes subject blurring, equipment blurring, and camera shake. Subject blurring refers to the movement of the subject's position on a plane (imaging surface) where an image is captured by a lens optical system or the like, shifting two-dimensionally (vertically, horizontally, diagonally, etc.) along the time axis. Equipment blurring refers to the movement of the subject's position along the time axis due to the occurrence of vibrations on the imaging device side, such as a microscope. Camera shake refers to the movement of the subject's position due to the physical movement of the photographer when the photographer holds the imaging device, such as a microscope, (handheld, etc.). Even if the photographer is not holding the imaging device completely, camera shake can also occur by touching or operating the device, causing vibrations.
[0059] Since imaging (photography) is performed over a certain period of time (shutter speed) (for example, 1 / 10th to 1 / 100th of a second), if the above-mentioned blurring occurs, the subject will move even within the short period of time at that shutter speed, and the clarity of the captured image will decrease. This state is called a blurring state.
[0060] 1, the imaging processing device 3 is configured using an information processing device such as a personal computer, etc. The imaging processing device 3 includes a calculation unit 30, an external I / F circuit 31 (image acquisition unit), a display 32 (display unit), a keyboard 33, a mouse 34, and a communication I / F circuit 35.
[0061] The external I / F circuit 31 is an interface circuit capable of receiving image data showing multiple images from the microscope camera 2. The communication I / F circuit 35 is a communication interface circuit capable of communicating with the server device 4 via the network 5. The communication I / F circuit 35 may be a wired communication circuit or a wireless communication circuit. The display 32 displays images captured by the microscope camera 2 in real time.
[0062] The calculation unit 30 is configured to include, for example, a CPU (Central Processing Unit) that executes predetermined calculation processing, a RAM (Random Access Memory) that temporarily stores data, a non-volatile storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), a timer circuit, and peripheral circuits thereof, etc. The calculation unit 30 executes, for example, a first capillary imaging program stored in the above-mentioned storage device, thereby functioning as an evaluation target region receiving unit 301, a focus evaluation value calculation unit 302, a judgment threshold setting unit 303, a focus evaluation value display unit 304, an operation guide notification unit 305, a focused image selection unit 306, and a blur evaluation value calculation unit 307.
[0063] The evaluation target area receiving unit 301 receives a designation of an arbitrary area range within the imaging range of the microscope camera 2 as the evaluation target area.
[0064] The focus evaluation value calculation unit 302 calculates a focus evaluation value Fv indicating a numerical value of the degree of focus for each of the multiple images acquired by the external I / F circuit 31. The judgment threshold setting unit 303 executes a judgment threshold setting process to set the focus evaluation value Fv indicating the best focus among the multiple focus evaluation values Fv calculated by the focus evaluation value calculation unit 302 as a judgment threshold Jv.
[0065] The focus evaluation value display unit 304 displays the focus evaluation value Fv calculated by the focus evaluation value calculation unit 302 on the display 32 in real time during the imaging period by the microscope camera 2. The operation guide notification unit 305 notifies the user of operation guides that encourage the user to adjust the microscope camera 2. The operation guides may be displayed on the display 32, or may be audio that encourages the user to adjust the microscope camera 2.
[0066] The focused image selection unit 306 compares each focus evaluation value Fv with a determination threshold Jv for determining whether the degree of focus is good or not, and selects an image having a good degree of focus from among the multiple images as a focused image FP.
[0067] The blur evaluation value calculation unit 307 calculates a blur evaluation value Bv that indicates the degree of blur of each focused image FP. The focused image selection unit 306 transmits each focused image FP, the focus evaluation value Fv of each focused image FP, and the blur evaluation value Bv of each focused image FP to the server device 4 via the network 5.
[0068] The server device 4 includes a calculation unit 40 and a communication I / F circuit 44 (focused image receiving unit). The communication I / F circuit 44 is a communication interface circuit capable of communicating with the communication I / F circuit 35 via the network 5. The communication I / F circuit 44 may be a wired communication circuit or a wireless communication circuit. The communication I / F circuit 44 receives input of a plurality of focused images FP transmitted from the imaging processing device 3 to the server device 4.
[0069] The calculation unit 40 is configured to include, for example, a CPU that executes predetermined calculation processing, a RAM that temporarily stores data, a non-volatile storage device such as an HDD or SSD, a timer circuit, and peripheral circuits thereof, etc. The calculation unit 40 functions as an image capture evaluation value calculation unit 41, a best image selection unit 42, and a health condition evaluation unit 43 by executing, for example, a second capillary imaging program stored in the above-mentioned storage device.
[0070] The reflection evaluation value calculation unit 41 calculates a reflection evaluation value Rv that indicates the degree of reflection of the light source in each focused image FP transmitted from the image capturing and processing device 3.
[0071] The best image selection unit 42 selects one of the multiple focused images FP transmitted from the imaging processing device 3 as the best image BP based on at least one of the focus evaluation value Fv of each focused image FP, the blur evaluation value Bv indicating the degree of blur of each focused image FP, and the reflection evaluation value Rv indicating the degree of reflection of the light source in each focused image FP.
[0072] The health condition evaluation unit 43 evaluates the health condition of the subject based on the best image BP. As a method for evaluating the health condition by the health condition evaluation unit 43, various known methods can be used, such as the methods described in Japanese Patent No. 6186663 and Japanese Patent No. 6906188.
[0073] Next, a description will be given of the operation of the capillary blood vessel imaging system 1 configured as described above. Figures 3 and 4 are flowcharts showing an example of the operation of the imaging processing device 3 shown in Figure 1, which corresponds to an example of the processing of the first capillary blood vessel imaging program.
[0074] First, with the subject placing the tip of his / her finger on the guide portion 28 of the microscope camera 2, the microscope camera 2 starts capturing moving images of the capillaries (step S1), and the captured image is displayed on the display 32. Next, the evaluation target area receiving unit 301 receives the designation of an arbitrary area range within the imaging range of the microscope camera 2 as the evaluation target area (step S2).
[0075] Fig. 6 is an explanatory diagram showing an example of a captured image G displayed on the display 32 shown in Fig. 1. Six capillaries K are captured in the captured image G shown in Fig. 6. The evaluation target area receiving unit 301 receives, as the evaluation target area A, a rectangular area selected by the user by operating the mouse 34, such as the evaluation target area A shown in Fig. 6.
[0076] Next, the operation guide notifying unit 305 notifies the user by displaying an operation guide on the display 32 to encourage the user to adjust the focus (step S3). Fig. 7 is a screen diagram showing an example of the display screen DP of the display 32 shown in Fig. 1. On the display screen DP, a message such as "Please adjust the focus" is displayed next to the captured image G as operation guide M to encourage the user to adjust the focus. Note that the method of notifying is not limited to a display, and the operation guide to encourage the user to adjust the focus may be announced by a sound.
[0077] Next, the judgment threshold setting unit 303 starts measuring the elapsed time t1 (step S4). Next, the focus evaluation value calculation unit 302 calculates the focus evaluation value Fv of the frame image, which is the image of each frame in the moving image captured by the microscope camera 2, for the evaluation target area A in real time, that is, every time a frame image is obtained (step S5).
[0078] Specifically, the focus evaluation value calculation section 302 calculates the focus evaluation value Fv of each frame image by executing the following steps (1) to (4) for each frame image.
[0079] (1) The frame image is grayscaled, (2) the contours of the grayscaled frame image are extracted, (3) within the evaluation area A, the difference in luminance value between the pixel extracted as the contour and the pixel adjacent to that pixel on the outside of the contour is calculated as the luminance difference Ls, and (4) the sum of the calculated luminance differences Ls that exceed a predetermined reference value ref is calculated as the focus evaluation value Fv.
[0080] As the contour extraction method of (2) above, a known contour extraction method such as a Laplacian filter can be used. If the luminance difference Ls of (3) above is large, it means that the contour is clear, and it can be determined that the focus is good.
[0081] According to (4) above, the sum of the luminance differences Ls for pixels in the evaluation area A that are in good focus and exceed the degree of focus set by the reference value ref is calculated as the focus evaluation value Fv. As a result, the focus evaluation value Fv becomes an index for evaluating the degree of focus of the capillary blood vessel image overall over the entire evaluation area A.
[0082] In the above (4), the coordinates of the in-focus pixel P, which is a pixel for which a luminance difference Ls exceeding the reference value ref has been obtained, are stored for use as a blur evaluation value Bv, which will be described later.
[0083] The focus evaluation value Fv may be any value that indicates the degree of focus in a numerical value, and is not limited to the value calculated by the above (1) to (4). The focus evaluation value Fv calculated by the above (1) to (4) indicates that the degree of focus is better as the value is larger, but the focus evaluation value Fv may be an index in which the degree of focus is better as the value is smaller. Various known techniques for evaluating focus can be used as a method for calculating the focus evaluation value Fv.
[0084] The reference value ref corresponds to an example of a reference level set in advance for the degree of focus of a pixel, and the focused pixel P corresponds to an example of a pixel whose degree of focus exceeds the preset reference level. The focus evaluation value calculation unit 302 may display the focused pixel P on the display 32 in real time, for example, by highlighting it brightly, as shown in Fig. 6. This allows the user performing the focusing operation to know the part in focus.
[0085] The shape of the capillaries is winding, and even if a fingertip is placed on the guide portion 28 of the microscope camera 2, the entire capillary cannot be positioned equidistant from the objective lens of the microscope camera 2 within the imaging range of the microscope camera 2. Therefore, it is difficult to focus on the entire capillary.
[0086] Therefore, by providing the evaluation target area receiving unit 301 and making it possible to set the evaluation target area A, the focus evaluation value Fv can be used as an index representing the degree of focus of an area important for evaluating the health condition, such as the vicinity of the tip of a capillary blood vessel. As a result, the focused image selection unit 306 and the best image selection unit 42, which will be described later, can select an image in which the area important for evaluating the health condition is well focused.
[0087] In step S5, the frame image immediately preceding the frame image to be processed is stored so that the immediately preceding image, which will be described later, can be acquired.
[0088] Next, the focus evaluation value display unit 304 displays the focus evaluation value Fv calculated by the focus evaluation value calculation unit 302 on the display 32 in real time (step S6). The focus evaluation value display unit 304 may display the focus evaluation value Fv in a graph using the number of bars, as shown in Fig. 7, for example, or may display the focus evaluation value Fv directly as a numerical value. By displaying the focus evaluation value Fv in real time while the user is performing a focus operation during the imaging period by the microscope camera 2, the accuracy of focusing by the user can be improved.
[0089] Next, the judgment threshold setting unit 303 associates one frame image with the calculated focus evaluation value Fv and stores it as a focused image candidate, and each time a larger focus evaluation value Fv is calculated, replaces the focused image candidate with the image with the larger focus evaluation value Fv together with the calculated focus evaluation value Fv (step S7). As a result, the frame image with the largest focus evaluation value Fv is associated with the largest focus evaluation value Fv and stored as a focused image candidate.
[0090] Next, the blur evaluation value calculation unit 307 calculates a blur evaluation value Bv of the candidate focused image based on the frame immediately before the candidate focused image, i.e., the previous image captured immediately before the candidate focused image, and the candidate focused image, and stores the blur evaluation value Bv in association with the candidate focused image (step S8).
[0091] Specifically, for example, the blur evaluation value Bv is calculated based on the coordinates of the focused pixel P for which the luminance difference Ls exceeding the reference value ref, which was stored in (4) above. That is, the coordinates of the focused pixel P in the immediately preceding image are compared with the coordinates of the focused pixel P in the focused image candidate, and the sum of the coordinate differences between corresponding pixels in both images is calculated as the blur evaluation value Bv of the focused image candidate. As a result, when the blur evaluation value Bv is zero, the focused image candidate is not blurred, and the larger the blur evaluation value Bv, the larger the blur of the focused image candidate is.
[0092] When the microscope camera 2 captures images in an interlaced format such as NTSC, images of odd and even lines are alternately captured. In this case, the blur evaluation value Bv can be calculated based on the images of the odd and even lines before and after the images. When calculating the blur evaluation value Bv, the difference in the coordinates may be calculated after correcting the deviation in the coordinate positions of the odd and even lines.
[0093] The blur evaluation value Bv only needs to represent the degree of blur in the image, and is not limited to the example in which the blur evaluation value Bv increases as the blur increases, but may increase as the blur decreases.
[0094] In addition, the method is not limited to the example in which the blur evaluation value Bv is calculated by comparing the position of the coordinates of the focused pixel P in the immediately preceding image with the coordinates of the focused pixel P in the focused image candidate. As a method for calculating the blur evaluation value Bv, various known methods for performing correlation between two images (the immediately preceding image and the focused image candidate), so-called pattern matching, can be used, such as Pearson's product correlation coefficient, SAD (Sum of Absolute Difference), SSD (Sum of Squared Difference), NCC (Normalized Cross-Correlation), and ZNCC (Zero-mean Normalized Cross-Correlation).
[0095] Next, the judgment threshold setting unit 303 compares the elapsed time t1 with a preset spare time ts1 (step S9). The spare time ts1 may be set to a fixed time of, for example, about 10 seconds, or may be set to an arbitrary time. If the elapsed time t1 is equal to or less than the spare time ts1 (NO in step S9), steps S5 to S9 are repeated again, and if the elapsed time t1 exceeds the spare time ts1 (YES in step S9), the judgment threshold setting unit 303 sets the focus evaluation value Fv of the focused image candidate as the judgment threshold Jv (step S11).
[0096] As a result, the focus evaluation value Fv that indicates the best focus among all images captured during the preliminary time ts1 is set as the determination threshold value Jv. Steps S7 and S11 correspond to an example of a determination threshold setting process.
[0097] Next, the focused image selection unit 306 selects the image stored as the focused image candidate as the focused image FP (step S12). As a result, the image with the best focus among all the images captured during the preliminary time ts1 is selected as the focused image FP.
[0098] Next, the focused image selection unit 306 starts measuring the elapsed time t2 (step S13).
[0099] Next, similar to step S5, the focus evaluation value calculation unit 302 calculates the focus evaluation value Fv of the frame image, which is the image of each frame in the video image captured by the microscope camera 2, for the evaluation target area A in real time, i.e., each time a frame image is obtained (step S14).
[0100] Next, similar to step S6, the focus evaluation value display unit 304 displays the focus evaluation value Fv calculated by the focus evaluation value calculation unit 302 on the display 32 in real time (step S15).
[0101] Next, each time a focus evaluation value Fv equal to or greater than the determination threshold value Jv is calculated, the focused image selection unit 306 selects and stores the image for which the focus evaluation value Fv has been calculated as a focused image FP (step S16).
[0102] The number, shape, thickness, and other conditions of human capillaries vary from person to person. The same is true for capillaries of living organisms such as animals other than humans. Therefore, when calculating the focus evaluation value Fv using the above (1) to (4), the number of pixels extracted as the contour varies depending on the condition of the subject's capillaries. As a result, different focus evaluation values Fv are calculated depending on the subject, even for images with the same degree of focus. Therefore, if the judgment threshold Jv is set to a fixed value, there is a risk that an image with appropriate focus cannot be selected depending on the subject.
[0103] However, according to steps S4 to S11, it is possible to set the determination threshold Jv according to the state of the capillaries for each subject, which increases the reliability with which an image of the capillaries with a good degree of focus is selected as the focused image FP in step S16.
[0104] Next, the blur evaluation value calculation unit 307 calculates a blur evaluation value Bv of the focused image FP based on the frame immediately before the focused image FP, i.e., the immediately preceding image captured immediately before the focused image FP, and the focused image FP, and stores the blur evaluation value Bv in association with the focused image FP (step S17). The process of step S17 is the same as step S8, except that the focused image FP is used instead of the focused image candidate of step S8.
[0105] Next, the focused image selection unit 306 compares the elapsed time t2 with a preset actual shooting time ts2 (step S18). The actual shooting time ts2 may be set to a fixed time of, for example, about 20 to 40 seconds, or may be set to any time.
[0106] If the elapsed time t2 is less than or equal to the actual shooting time ts2 (NO in step S18), steps S14 to S18 are repeated again, and if the elapsed time t2 exceeds the actual shooting time ts2 (YES in step S18), the focused image selection unit 306 transmits all focused images FP and the focus evaluation value Fv and blur evaluation value Bv of each focused image FP to the server device 4 via the communication I / F circuit 35 (step S19).
[0107] As a result, all images whose focused images FP are determined to have a good degree of focus, that is, images whose focused images FP have a value equal to or greater than the determination threshold value Jv, as well as the focus evaluation value Fv and blur evaluation value Bv of each focused image FP are transmitted to the server device 4.
[0108] As described above, according to steps S1 to S19, a well-focused image FP having a focus evaluation value Fv equal to or greater than the determination threshold value Jv can be obtained, making it easy to obtain a well-focused image of the capillaries.
[0109] Next, the operation of the server device 4 will be described. Fig. 5 is a flow chart showing an example of the operation of the server device 4 shown in Fig. 1, which corresponds to an example of processing of the second capillary blood vessel imaging program. First, the communication I / F circuit 44 receives and acquires a plurality of focused images FP and the focus evaluation value Fv and blur evaluation value Bv of each focused image FP transmitted from the imaging processing device 3 (step S21).
[0110] Next, the reflection evaluation value calculation unit 41 calculates the reflection evaluation value Rv of each focused image FP received by the communication I / F circuit 44 (step S22). As shown in Fig. 2, the microscope camera 2 illuminates the subject's fingertip with the illumination unit 26 and captures the image. At that time, the illumination light from the illumination unit 26 may be reflected in the captured image G, for example, as shown in Fig. 6 as reflection R. The reflection R reduces image quality and reduces the evaluation accuracy of the health condition based on the captured image G.
[0111] Therefore, the reflection evaluation value calculation unit 41 calculates a reflection evaluation value Rv as one of the evaluation indexes for evaluating the image quality of each focused image FP.
[0112] For example, the reflection evaluation value calculation unit 41 calculates the reflection evaluation value Rv of each focused image FP by executing the following (A) to (D) for each focused image FP.
[0113] (A) The focused image FP is grayscaled, (B) the average brightness value Lave of each pixel of the grayscaled focused image FP is calculated, (C) pixels whose brightness is higher than the average brightness value Lave by at least a predetermined reflection judgment value Rref are designated as reflected pixels, and (D) the sum of the brightness values of the reflected pixels is designated as the reflection evaluation value Rv of the focused image FP.
[0114] According to (A) to (D), the larger the reflection evaluation value Rv, the more reflection in the focused image FP. The reflection determination value Rref may be appropriately set based on, for example, experimental results, standard deviation, or the like.
[0115] In addition, the reflection evaluation value Rv may be any index that represents the degree of reflection of the light source in the focused image FP, and is not limited to the example in which a larger reflection evaluation value Rv indicates that there is more reflection in the focused image FP, but may also indicate that a smaller reflection evaluation value Rv indicates that there is more reflection in the focused image FP.
[0116] Next, the best image selection unit 42 selects one of the multiple focused images FP as the best image BP based on the multiple focused images FP, the focus evaluation value Fv of each focused image FP, and the blur evaluation value Bv of each focused image FP received by the communication I / F circuit 44, and the reflection evaluation value Rv of each focused image FP calculated by the reflection evaluation value calculation unit 41 (step S23).
[0117] As a method for selecting the best image BP based on the focus evaluation value Fv, the blur evaluation value Bv, and the reflection evaluation value Rv, for example, the following image evaluation value Gv may be calculated for each focused image FP, and the focused image FP with the largest image evaluation value Gv may be selected as the best image BP.
[0118] Image evaluation value Gv = Fv - Bv - Rv (1)
[0119] The above formula (1) calculates an image evaluation value Gv, where a larger value indicates better image quality, by adding the focus evaluation value Fv, and subtracting the blur evaluation value Bv and the reflection evaluation value Rv, where a smaller value indicates better image quality.
[0120] As another method for calculating the image evaluation value Gv, for example, the following formula (2) can be used.
[0121] Image evaluation value Gv = aFv - bBv - cRv (2)
[0122] a, b, and c are coefficients.
[0123] According to formula (2), weighting can be performed according to the importance of the focus evaluation value Fv, the blur evaluation value Bv, and the reflection evaluation value Rv. For example, a=0.7, b=0.2, and c=0.1 can be used.
[0124] Note that the smaller the image evaluation value Gv, the higher the image quality may be. In this case, the focused image FP with the smallest image evaluation value Gv may be selected as the best image BP.
[0125] As described above, according to steps S21 to S23, it is possible to obtain a best image BP having good image quality from the viewpoints of focus, blur, and reflection of illumination light, out of the focused images FP having a good degree of focus. This makes it easy to obtain an image of the capillaries having good overall image quality, not just in terms of the degree of focus.
[0126] Next, the health condition evaluation unit 43 evaluates the subject's health condition based on the best image BP (step S24) and ends the process. The health condition evaluation unit 43 may display the evaluation result on a display (not shown) or may transmit the evaluation result to the imaging processing device 3 or the like via the communication I / F circuit 44. According to step S24, the subject's health condition can be evaluated based on the best image BP with good image quality, thereby improving the accuracy of the health condition evaluation.
[0127] Although an example has been shown in which the user adjusts the focus of the microscope camera 2 by operating the focus adjustment handle 27, the microscope camera 2 may be configured to adjust the focus by a known autofocus mechanism. The microscope camera 2 may be configured to capture a video or multiple still images while the focus is being adjusted by the autofocus mechanism. The capillary blood vessel imaging system 1 does not necessarily have to include the microscope camera 2.
[0128] Furthermore, the capillary imaging system 1 may not include the health condition evaluation unit 43 and may not execute step S24. Furthermore, the capillary imaging system 1 may not include the evaluation target region receiving unit 301 and may not execute step S2. In this case, the focus evaluation value calculation unit 302 may calculate the focus evaluation value Fv for the entire image, or may calculate the focus evaluation value Fv for a preset evaluation target region A.
[0129] Furthermore, the capillary blood vessel imaging system 1 does not need to include the focus evaluation value display unit 304 and does not need to execute step S15, and does not need to include the operation guide notifying unit 305 and does not need to execute step S3.
[0130] Furthermore, the capillary blood vessel imaging system 1 does not need to include the projection evaluation value calculation unit 41 and does not need to execute step S22. In this case, the best image selection unit 42 may select the best image BP based on the focus evaluation value Fv and the blur evaluation value Bv.
[0131] Furthermore, the capillary blood vessel imaging system 1 does not need to include the blur evaluation value calculation unit 307 and does not need to execute step S17. In this case, the best image selection unit 42 may select the best image BP based on the focus evaluation value Fv and the projection evaluation value Rv. Alternatively, the best image selection unit 42 may select the focused image FP with the maximum focus evaluation value Fv as the best image BP based only on the focus evaluation value Fv.
[0132] Furthermore, the best image selection unit 42 may select the best image BP based on at least one of the blur evaluation value Bv and the reflection evaluation value Rv, without using the focus evaluation value Fv.
[0133] Furthermore, the capillary blood vessel imaging system 1 does not need to include the best image selection unit 42, the projection evaluation value calculation unit 41, and the blur evaluation value calculation unit 307, and does not need to execute steps S23, S22, and S17. Even if the best image BP is not selected, the focused image FP itself is an image of the capillaries with a good degree of focus. In this case, the health condition evaluation unit 43 only needs to evaluate the health condition of the subject, etc., based on the focused image FP.
[0134] Furthermore, the focused image selection unit 306 may not execute step S12. Furthermore, the capillary blood vessel imaging system 1 may not include the determination threshold setting unit 303 and may not execute steps S5 to S12. In this case, the focused image selection unit 306 may select a focused image FP based on a preset determination threshold Jv.
[0135] Furthermore, the capillary imaging system 1 does not necessarily have to include the server device 4, and the capillary imaging system 1 may be configured only by the imaging processing device 3. Furthermore, the imaging processing device 3 may include at least one of the image capture evaluation value calculation unit 41, the best image selection unit 42, and the health condition evaluation unit 43. In this case, the first capillary imaging program and the second capillary imaging program may be configured as one capillary imaging program.
[0136] Furthermore, the server device 4 may include an evaluation target area receiving unit 301, a focus evaluation value calculation unit 302, a judgment threshold setting unit 303, a focus evaluation value display unit 304, an operation guide notification unit 305, a focused image selection unit 306, and a blur evaluation value calculation unit 307, and the capillary blood vessel imaging system 1 may not include the imaging processing device 3. The microscope camera 2 may be connected to the communication I / F circuit 44 of the server device 4 via the network 5. In this case, the communication I / F circuit 44 corresponds to an example of an image acquisition unit.
[0137] Also, an example has been shown in which the focus evaluation value calculation unit 302 calculates the focus evaluation value Fv every time a frame image is captured by the microscope camera 2, and the focused image selection unit 306 selects an image every time the focus evaluation value Fv is calculated, i.e., performs processing in real time, but the capillary blood imaging system 1 is not limited to an example in which processing is performed in real time. After a moving image or a plurality of still images are captured by the microscope camera 2, the capillary blood imaging system 1 may select a focused image FP and a best image BP from the moving image or the plurality of still images afterwards. [Explanation of symbols]
[0138] 1. Capillary Imaging System 2 Microscope camera (imaging unit) 3. Imaging Processing Device 4 Server device (server device for capillary blood vessel imaging system) 5. Network 21 Microscope body 22 Foundation 23 Connecting part 24 Adjustment handle 25 Objective Lens 26 Lighting Department 27 Focus adjustment handle 28 Guide section 29 Cable 30 Arithmetic section 31 External I / F circuit (image acquisition section) 32 Display (display unit) 33 Keyboard 34 Mouse 35 Communication I / F circuit 40 Arithmetic section 41 Reflection evaluation value calculation unit 42 Best Image Selection 43 Health Status Assessment Division 44 Communication I / F circuit (focused image receiving section) 281 Recess 301 Evaluation Area Reception Department 302 Focus evaluation value calculation unit 303 Judgment threshold setting unit 304 Focus evaluation value display section 305 Operation guide notification section 306 Focused Image Selection 307 Blur evaluation value calculation unit A. Evaluation Area BP Best Image Bv Blur evaluation value DP display screen FP Focused Image Fv Focus evaluation value G. Captured image Gv image evaluation value Jv Judgment Threshold K capillary Lave Average luminance value Ls Luminance difference M Operation Guide P Focused pixel R Reflection Rref Reflection judgment value Rv Reflection evaluation value ref Reference value t1,t2 Elapsed time ts1 Reserve time ts2 Main shooting time
Claims
1. An image acquisition unit that acquires a plurality of images of capillaries of an organism from an imaging unit that captures the plurality of images; a focus evaluation value calculation unit that calculates a focus evaluation value indicating a degree of focus by converting the degree of focus of each of the plurality of images into a numerical value; a determination threshold setting unit that executes a determination threshold setting process to set a focus evaluation value that indicates the best focus among the plurality of focus evaluation values calculated by the focus evaluation value calculation unit as a determination threshold for determining whether or not the degree of focus is good; a focused image selection unit that selects an image having a good degree of focus as a focused image from among the plurality of images by comparing the determination threshold value with each of the focus evaluation values, The capillary imaging system, wherein the focused image selection unit selects an image having a good degree of focus as the focused image from among a plurality of images newly acquired by the image acquisition unit after the judgment threshold setting process.
2. 2. The capillary blood vessel imaging system according to claim 1, wherein the in-focus image selection unit further selects, as the in-focus image, an image for which the in-focus evaluation value indicating the best focus is obtained in the determination threshold setting process.
3. An evaluation target area receiving unit that receives a designation of an arbitrary area range within the imaging range of the imaging unit as an evaluation target area, The capillary imaging system according to claim 1 , wherein the focus evaluation value calculation section calculates the focus evaluation values for the evaluation target regions in the plurality of images.
4. The imaging unit is capable of adjusting a focus by a user, 4. The capillary imaging system according to claim 1, further comprising a focus evaluation value display unit that displays the focus evaluation value calculated by the focus evaluation value calculation unit in real time during an imaging period by the imaging unit.
5. The imaging unit is capable of adjusting a focus by a user, an operation guide notifying unit that notifies the user of an operation guide that prompts the user to adjust the focus; 5. The capillary blood vessel imaging system according to claim 1, wherein the imaging section repeatedly captures the image during a period during which the operation guidance is being notified.
6. A display unit that displays an image captured by the imaging unit in real time is further provided. The capillary imaging system according to any one of claims 1 to 5, wherein the focus evaluation value calculation unit determines in real time the degree of focus of each pixel of the image captured by the imaging unit, and causes the display unit to highlight focused pixels whose degree of focus exceeds a preset reference level.
7. 7. The capillary imaging system according to claim 1, further comprising a best image selection unit that further selects one of the plurality of focused images as a best image based on a focus evaluation value of each of the focused images.
8. a blur evaluation value calculation unit that calculates a blur evaluation value representing a degree of blur of each of the focused images; 8. The capillary imaging system according to claim 1, further comprising a best image selection unit that further selects one of the plurality of focused images as a best image based on a blur evaluation value of each of the focused images.
9. a reflection evaluation value calculation unit that calculates a reflection evaluation value that indicates a degree of reflection of a light source in each of the focused images; The capillary imaging system according to any one of claims 1 to 8, further comprising a best image selection unit that further selects one of the plurality of focused images as a best image based on an imaging evaluation value of each of the focused images.
10. The capillary imaging system according to any one of claims 1 to 6, further comprising a best image selection unit that selects one of the multiple focused images as a best image based on at least one of a focus evaluation value of each of the focused images, a blur evaluation value representing a degree of blur of each of the focused images, and a reflection evaluation value representing a degree of reflection of a light source in each of the focused images.
11. The capillary imaging system according to any one of claims 1 to 10, further comprising a health condition assessment unit that assesses a health condition of the living organism based on the focused image.
12. 11. The capillary imaging system according to claim 7, further comprising a health status assessment unit for assessing a health status of the living organism based on the best image.
13. 13. The capillary blood vessel imaging system according to claim 1, wherein the imaging section captures the images as frames of a video.
14. the focus evaluation value calculation unit calculates the focus evaluation value each time the image is captured by the imaging unit, 14. The capillary imaging system according to claim 1, wherein the in-focus image selection section selects the image each time the in-focus evaluation value is calculated.
15. The capillary blood vessel imaging system according to claim 1 , further comprising the imaging unit.
16. a focused image receiving unit that receives input of a plurality of focused images, which are images of capillaries of an organism; a focus evaluation value calculation unit that calculates a focus evaluation value indicating a degree of focus by converting the degree of focus into a numerical value for each of the plurality of focused images; a determination threshold setting unit that executes a determination threshold setting process to set a focus evaluation value that indicates the best focus among the plurality of focus evaluation values calculated by the focus evaluation value calculation unit as a determination threshold for determining whether or not the degree of focus is good; a focused image selection unit that selects a focused image having a good degree of focus from among the plurality of focused images by comparing the determination threshold value with each of the focus evaluation values; a best image selection unit that selects one of the plurality of focused images selected by the focused image selection unit as a best image based on at least one of the focus evaluation value, a blur evaluation value that indicates a degree of blur of each of the focused images, and a reflection evaluation value that indicates a degree of reflection of a light source in each of the focused images, The in-focus image selection unit selects an image having a good degree of focus from among a plurality of in-focus images newly acquired by the image acquisition unit after the judgment threshold setting process.
17. A capillary imaging program that causes a computer to function as the capillary imaging system according to any one of claims 1 to 14.
18. A capillary blood vessel imaging program that causes a computer to function as the server device for the capillary blood vessel imaging system according to claim 16.
Citation Information
Patent Citations
Method for measuring dielectric breakdown strength between layers of coil
JP1986086663A
Storage device
JP1988079374A
Digital camera and focus control program
JP2008276115A
Electronic apparatus, imaging device, and reproducing device
JP2009089348A
Imaging device capable of focus display control
JP2012099888A
