Diagnostic support device, diagnostic support method, and program

Abstract

Based on the diagnostic results obtained from the images of the diagnostic area, a projected image related to the diagnostic area is displayed on the living body in a recognizable manner. [Solution] The diagnostic support device 100 includes a control unit 110 that detects a predetermined diagnostic area based on a biological image, acquires a diagnostic result related to the diagnostic area based on candidate lesion images related to the detected diagnostic area, and displays a projected image related to the diagnostic area on the living body in a recognizable manner based on the acquired diagnostic result.

Description

Technical Field

[0001] The present invention relates to a diagnostic support device, a diagnostic support method, and a program.

Background Art

[0002] Conventionally, a medical system that captures an image of the human body and displays diagnostic information on a display of an information device has been known. For example, Patent Document 1 discloses a skin disease analysis system that uses a machine learning model to accurately determine the types of skin diseases and skin tumors in a medical system that analyzes an imaged skin disease, and displays the determination result on a display such as a smartphone.

Prior Art Documents

[0007] According to the present invention, a projected image relating to a diagnostic region can be displayed on a living body in a recognizable manner based on the diagnostic results obtained from the image of the diagnostic region. [Brief explanation of the drawing]

[0008] [Figure 1] This diagram shows the configuration of the diagnostic support system according to the embodiment. [Figure 2] This is a diagram showing the functional configuration of diagnostic support devices included in a diagnostic support system. [Figure 3] This figure shows an example of an image database. [Figure 4] This is a flowchart of the diagnostic support process according to the embodiment. [Figure 5] This figure shows an example of a biological image. [Figure 6] This figure shows an example of a diagnostic domain database. [Figure 7] This figure shows an example of a past biological image. [Figure 8] This diagram illustrates the correspondence between past and current biological images. [Figure 9] This figure shows an example of correspondence data. [Figure 10] This figure shows an example of past correspondence data. [Figure 11] This figure shows an example of a diagnostic area selection screen. [Figure 12] This figure shows an example of a projection image in which all diagnostic areas suspected of being malignant are indicated by borders. [Figure 13]This figure shows an example of a projected image that outlines all diagnostic areas suspected of malignancy and displays risk scores near each diagnostic area. [Figure 14] This figure shows an example of a projection image, where selected diagnostic areas are indicated by solid lines, and unselected diagnostic areas suspected of malignancy are indicated by dashed lines. [Figure 15] This figure shows an example of an externally linked image database. [Modes for carrying out the invention]

[0009] The diagnostic support device and the like according to the embodiment will be described with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals. As shown in Figure 1, the diagnostic support device 100 according to this embodiment comprises a photographing device 210 that captures images of the subject 300 with a wide field of view and acquires these images as biological images, and a projection device 220 that projects a projection image related to the diagnosis of the biological images onto the surface of the subject 300's body, together with the diagnostic support system 200. Furthermore, the diagnostic support device 100 can be linked with an external device 230 that takes close-up photographs of the subject 300's skin, etc., as needed. The diagnostic support device 100 detects a diagnostic area from the biological image acquired by the imaging device 210, processes images of candidate lesions (e.g., moles) present in the diagnostic area (candidate lesion images) to obtain a diagnostic result (e.g., risk score), and projects an image based on the diagnostic result onto the living body using the projection device 220. The diagnostic area is, for example, a part of the skin where the color has changed, or any other area where a doctor is expected to make a diagnosis (the diagnostic site). Furthermore, potential lesions include not only parts that show biological changes (lesions) caused by disease, but also parts where it is unclear whether or not they are diseased. In other words, anything that is considered to require diagnosis, regardless of whether it is malignant or benign (for example, a mole), is a potential lesion.

[0010] As shown in FIG. 2, the diagnostic support system 200 includes a diagnostic support device 100, an imaging device 210, and a projection device 220. These may be divided into three devices as shown in FIG. 1, or may be configured as two or one device (for example, an information processing device incorporating a camera and a projector). Further, the diagnostic support device 100 may be one device, or may be divided into a plurality of devices (for example, an image processing device including a control unit 110 and a storage unit 120, and an operation terminal device including a display unit 130 and an operation input unit 140). As shown in FIG. 2, the diagnostic support device 100 includes a control unit 110, a storage unit 120, a display unit 130, and an operation input unit 140 as functional components. The control unit 110 is composed of a processor such as a CPU (Central Processing Unit). The control unit 110 executes diagnostic support processing and the like, which will be described later, according to a program stored in the storage unit 120. The storage unit 120 is composed of, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, etc., and stores programs executed by the control unit 110 and necessary data. The display unit 130 includes a display such as a liquid crystal display or an organic EL (Electro-Luminescence) display. The display unit 130 displays an image captured by the imaging device 210, a screen for selecting a diagnostic region, etc. However, when it is not necessary to display these, the diagnostic support device 100 may not include the display unit 130. The operation input unit 140 is a user interface such as a keyboard, a mouse, a touch panel, etc., and receives operation inputs from a user (for example, a doctor, a nurse, etc.). When the operation input unit 140 includes a touch panel, it may be a touch panel integrated with the display of the display unit 130. When it is not necessary to receive operation inputs from a user, the diagnostic support device 100 may not include the operation input unit 140.

[0011] The imaging device 210 includes an imaging element such as a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor. Then, the imaging device 210 captures an image of the skin of the subject 300, for example, in a wide field of view and acquires the image data. The control unit 110 acquires the biological image data of the subject 300 by the imaging device 210 based on, for example, a shooting instruction from the user, and stores the acquired biological image data together with the shooting date and time in the storage unit 120. As a result, in the storage unit 120, an image database 121 is constructed that stores, for example, the biological image data of the subject 300 identified by the patient ID (Identification) captured by the imaging device 210, associated with the patient ID, the image ID, and the shooting date and time. In the example shown in FIG. 3, it can be seen that the biological image data with the image ID = 004 was captured on October 1, 2023, and the biological image data with the image ID = 005 was captured on January 18, 2024, and each is stored in the image database 121. The projection device 220 is a projector. Since the projection target is the surface of the human body with unevenness, it is desirable to use a focus-free laser scanning projector for the projection device 220. The external device 230 is an imaging device capable of taking a contact image of the skin of the subject 300, for example, a dermocamera (registered trademark).

[0012] The functional configuration of the diagnostic support device 100 and the like has been described above. Next, the diagnostic support process executed by the control unit 110 of the diagnostic support device 100 will be described with reference to FIG. 4. This process is a process of projecting a video related to the diagnostic result onto the diagnostic area of the subject 300, and the execution is started by a user's instruction.

[0013] First, the control unit 110 performs a calibration process to establish a correspondence between the coordinates of the imaging device 210 and the projection device 220 (step S101). The method of calibration is arbitrary, but for example, the control unit 110 projects a checkerboard pattern image onto the subject 300 using the projection device 220, and then captures the projected checkerboard image with the imaging device 210 to obtain the captured image. The control unit 110 then obtains the correspondence between the coordinates of each intersection point in the captured image (camera coordinates) and the coordinates of each intersection point in the original checkerboard image (projector coordinates). By using this correspondence, the control unit 110 can determine what values ​​should be set as projector coordinates to project onto a certain area of ​​the subject 300's body (an area represented by camera coordinates). Furthermore, if the imaging device 210 and the projection device 220 are integrated, ensuring that the correspondence between camera coordinates and projector coordinates is always maintained, or if calibration processing has already been completed in past imaging, the process in step S101 may be omitted.

[0014] Next, the control unit 110 uses the imaging device 210 to photograph the subject 300 and acquire a biological image (step S102). In this step, the control unit 110 acquires image data of a biological image 310 (for example, the biological image with image ID=005 in the example shown in Figure 5), as shown in Figure 5. Then, the control unit 110 detects diagnostic regions 320, 321, and 322 from the biological image (step S103). The method for detecting the diagnostic regions 320, 321, and 322 is arbitrary, but several methods are described below. For example, one possible method is the following three-step approach. Step 1: Convert the biological image data (overall image) into a grayscale image, binarize it at a certain threshold, and detect closed curves based on the boundary of the values. Step 2: Repeat the process from Step 1, changing the threshold for binarization, and group closed curves with similar center coordinates together. Step 3: Determine the central position and median size of each group, and use these as the position and size of each diagnostic region. Another method involves using object detection models such as R-CNN (Region-Convolutional Neural Network) or YOLO (You Only Look Once). However, the above is merely one example, and the detection methods for diagnostic areas 320, 321, and 322 are not limited to the two methods mentioned here.

[0015] The control unit 110 then stores images of candidate lesions (candidate lesion images) present in the diagnostic area in the storage unit 120 (step S104). A candidate lesion image is an image extracted from a biological image of a predetermined geometric shape (e.g., a rectangle) that completely includes the diagnostic area (with a margin of several pixels or more outside the diagnostic area). Each candidate lesion image is assigned a diagnostic area ID so that it can be uniquely identified, and is stored in the storage unit 120 as a diagnostic area database 122, for example as shown in Figure 6, linked to the patient ID (ID of the subject 300), image ID (ID of the biological image in which the diagnostic area was detected), date and time of capture (date and time when the biological image was captured), position in the biological image (x and y coordinates of the top left and bottom right of the rectangular area containing the diagnostic area), and diagnostic area ID (in Figure 6, an ID obtained by adding the ID of the diagnostic area contained in the image to the image ID). In the example shown in Figure 6, it can be seen that diagnostic region 321 in the biological image data shown in Figure 5 is detected as the diagnostic region with diagnostic region ID=005-001, and diagnostic region 322 is detected as the diagnostic region with diagnostic region ID=005-002.

[0016] Next, the control unit 110 extracts corresponding candidate lesion images from past biological images (step S105). In this step, the control unit 110 first reads image data of past biological images (in this example, the most recent biological image with image ID = 004) of the same subject 300 (in this example, patient ID = 00101) from the image database 121, as shown in Figure 7, and performs a correspondence between the biological image taken this time (image data with image ID = 005) and the past biological image (image data with image ID = 004), as shown by the dashed line in Figure 8. The method of this correspondence is arbitrary, but as an example, one method is to associate those with similar positions (for example, the relative position of the center of gravity of the diagnostic area based on the position of the human body (back, etc.) in the biological image). If there are multiple diagnostic areas with similar positions, one method is to obtain the image feature quantities of each diagnostic area and associate those with similar distances between feature quantities. Furthermore, instead of performing the mapping in two steps as described above (mapping using the centroid position, then mapping using image features), it is also possible to use a method in which the coordinates of the centroid position are included in the image features as the features used for mapping, and features that are close in distance from each other are mapped. With this method, the result of mapping the position, color, size, etc., of the diagnostic region can be obtained in one step. In addition, to further improve the accuracy of the mapping, a method may be used to optimize a predetermined objective function that takes spatial consistency into consideration in addition to these features. This makes it possible to maintain the relative positional relationship of the mapped diagnostic region between the current image and past images. However, since automatically mapping diagnostic areas in this way may result in incorrect mappings, the control unit 110 may display the results of the automatic mapping on the display unit 130, for example as shown in Figure 8, allowing the user to confirm whether the mapping between past and present diagnostic areas is correct and to correct the mapping as needed.

[0017] The control unit 110 then stores the finally determined correspondence result as correspondence relationship data 123 in the storage unit 120, as shown in Figure 9. As shown in Figure 8, the current diagnostic area 1 (diagnostic area ID=005-001) is associated with the past diagnostic area 1 (diagnostic area ID=004-001), so in Figure 9, the current diagnostic area ID=005-001 is associated with the past diagnostic area ID=004-001. Note that the current diagnostic area ID=005-001 is associated not only with 004-001 but also with 003-001, etc. as past diagnostic area IDs, which were obtained using the results of past correspondences. For example, suppose that when the diagnostic region of the previous biological image (image data with image ID=004) was associated with past diagnostic regions, the correspondence data 123 shown in Figure 10 was obtained. In other words, at this point, the diagnostic region ID=004-001 was associated with the past diagnostic region ID=003-001. Then, the current diagnostic region ID=005-001 was associated with the past diagnostic region ID=004-001. Referring to Figure 10, since the diagnostic region ID=004-001 was associated with the diagnostic region ID=003-001, the current diagnostic region ID=005-001 will be associated not only with the diagnostic region ID=004-001 but also with the diagnostic region ID=003-001.

[0018] Therefore, when the diagnostic region of the current biological image (image data with image ID=005) is associated with the diagnostic region of the previous biological image, as shown in Figure 9, for diagnostic regions 1 to 4, the current diagnostic region ID and past diagnostic region ID shown in Figure 10 are stored in the past diagnostic region ID in Figure 9. Then, the diagnostic region of the current biological image that could not be associated with the diagnostic region of the past biological image is added to the correspondence relationship data 123 as a newly appearing diagnostic region (as shown in diagnostic region 5 in Figure 9). By referring to the correspondence data 123 generated in this way, the control unit 110 can extract corresponding candidate lesion images from past biological images for each diagnostic region in the current biological image. The "labels" shown in Figures 9 and 10 are used to indicate each diagnostic area to the user (for example, they are displayed near the image (candidate lesion image) of each diagnostic area when displaying a list of diagnostic areas on the display unit 130). In this case, they are automatically set as "Diagnostic Area 1," "Diagnostic Area 2," etc., in ascending order of the diagnostic area ID (excluding the image ID), but the user may be allowed to set them freely.

[0019] Next, the control unit 110 diagnoses each diagnostic region based on the candidate lesion images for each diagnostic region (step S106). Specifically, the control unit 110 diagnoses the diagnostic region based on the candidate lesion images for each diagnostic region extracted in step S105. Diagnosis includes estimating a risk score (a value from 0 (benign) to 1 (malignant) indicating the probability of malignancy based on the candidate lesion image), estimating the disease name, and estimating the attributes of the lesion (type and morphology). The method of diagnosis is arbitrary, but for example, one method is to calculate the deviation from the mean value for each diagnostic factor (size, color, shape, and texture features, etc.) within the diagnostic region or image features obtained by a machine learning model, and determine that an abnormality (high risk score) exists if the deviation is large. Alternatively, a large number of images of malignant lesions and benign lesions may be prepared, a benign / malignant classifier may be generated using deep learning, and the control unit 110 may use this benign / malignant classifier to calculate the risk score. Furthermore, the control unit 110 may derive risk scores and disease names using machine learning models or the like based on the size, color, shape, and texture characteristics of the diagnostic area. In this case, the control unit 110 may also refer to the correspondence data 123 and obtain diagnostic results based on multiple candidate lesion images related to the corresponding past diagnostic area. By using multiple candidate lesion images, the accuracy of the diagnosis can be improved.

[0020] The control unit 110 then determines whether or not to select a diagnostic area (step S107). This determination determines whether to operate the diagnostic support process in diagnostic area selection mode (a mode in which the user selects a diagnostic area and projects the location of the selected diagnostic area onto the subject 300 so that it can be recognized) or in diagnostic area non-selection mode (a mode in which the location of all diagnostic areas suspected of being malignant is projected onto the subject 300 so that it can be recognized). This step can be omitted if there is no need to change the mode (either mode can be fixed). The mode change may be made on a settings screen (not shown), or it may be set in advance by the user before the diagnostic support process is executed. If the control unit 110 does not select a diagnostic area (step S107; No), it proceeds to step S109. If a diagnostic area is selected (step S107; Yes), the control unit 110 prompts the user to select a diagnostic area on the diagnostic area selection screen, which displays a list of diagnostic areas on the display unit 130 as shown in Figure 11, and acquires the diagnostic area selected by the user on the operation input unit 140 (step S108). In the example of the diagnostic area selection screen shown in Figure 11, the diagnostic area list displays candidate lesion images and their risk scores for each diagnostic area in descending order of risk score, and shows the user selecting diagnostic area 1 with the cursor 131. By selecting the desired diagnostic area with the cursor 131 and clicking the OK button 132, the control unit 110 can acquire the selected diagnostic area (diagnostic area 1 in this example). The user can also return to the unselected diagnostic area mode by clicking the back button 133. The external device linkage button 134 will be described later. Note that in Figure 11, the diagnostic areas are listed in descending order of the risk score estimated by the diagnosis, but this is just one example. For example, the diagnostic domains could be listed by grouping or sorting them by the disease name estimated in the diagnosis, or by grouping or sorting them by the attributes (type or morphology) of the lesion estimated in the diagnosis.

[0021] The control unit 110 then uses the imaging device 210 to record video of the subject 300 and acquire frame images for tracking (biological images acquired in real time, one frame at a time, by recording video) (step S109). The control unit 110 then generates a projection image (tracked to the diagnostic area) based on the results of the diagnosis in step S106 and the frame images acquired in step S109 (step S110). The projection image may include a symbol indicating the location of the diagnostic area suspected of being malignant, as well as diagnostic information (risk score, disease name, etc.) and identification information (patient ID, image ID, diagnostic area ID, date and time of shooting, etc.). For example, in the non-selection of diagnostic areas mode (the determination in step S107 is No), and further in the risk score hiding mode, as shown in Figure 12, a projection image is generated showing the locations of all diagnostic areas suspected of malignancy, i.e., all diagnostic areas with a risk score of 0.5 or higher, in a frame 222. However, the subject 300, depicted with a dotted line in the projection image 221 in Figure 12, is not actually included in the projection image, but is depicted to clearly show that the projection image 221 is projected onto the surface of the subject 300's body (the same applies to Figures 13 and 14 described later). In the risk score display mode, as shown in Figure 13, a projection image 221 is generated showing the locations of all diagnostic areas suspected of malignancy in a frame 222, and also showing the risk score 223 in the vicinity. Furthermore, in diagnostic area selection mode (determined as Yes in step S107) and risk score hiding mode, as shown in Figure 14, a projection image 221 is generated that shows the location of the diagnostic area selected from the list displayed on the display unit 130 in step S108 (diagnostic area 1 in the example shown in Figure 14) with a selection frame 225 (solid frame in Figure 14), and the locations of other diagnostic areas suspected of being malignant with a non-selection frame 224 (dashed frame in Figure 14).

[0022] These frames 222, risk score 223, non-selection frame 224, and selection frame 225 are projected by the control unit 110 to detect the body movements of the subject 300 in the frame images acquired in step S109 and to be projected at positions that follow the movement of each diagnostic area on the surface of the subject 300's body (to track the diagnostic area). Note that the projected images 221 shown in Figures 12 to 14 are merely examples of projected images 221. Furthermore, the control unit 110 may not only include the risk score in the projected image 221, but may also include, for example, the predicted disease name in the projected image 221. In addition, the appearance (color, thickness, type (solid line, dotted line, dashed line, etc.), shape (rectangle, circle, etc.)) of the symbol (e.g., frame line) indicating the location of the diagnostic area may be changed depending on the range of the risk score value and the disease name. The control unit 110 may also generate a projected image 221 in which an arrow or the like that points to the diagnostic area is used as a symbol to indicate the location of the diagnostic area, either in place of or in conjunction with the frame line. The control unit 110 then projects the projected image 221 generated in step S110 onto the surface of the subject 300's body using the projection device 220 (step S111).

[0023] Next, the control unit 110 determines whether or not to acquire an image from the external device 230 (step S112). This determination method is arbitrary, but for example, if the external device link button 134 is pressed on the diagnostic area selection screen as shown in Figure 11, the control unit 110 determines to acquire an image from the external device 230. Alternatively, based on the diagnostic results, if, for example, the risk score is greater than a predetermined value (e.g., 0.6), the control unit 110 may determine that "it is necessary to photograph the affected area in close-up mode," display this on the display unit 130, and set the determination in step S112 to Yes. If you do not want to acquire an image from the external device 230 (step S112; No), proceed to step S114. If an image is to be acquired from the external device 230 (step S112; Yes), the control unit 110 acquires the image (external linked image) from the external device 230 and stores the external linked image in the storage unit 120, linking it to the diagnostic area acquired in step S108 (step S113). In this step, the control unit 110 stores an external linked image database 124 in the storage unit 120, for example, as shown in Figure 15. Figure 15 shows that an externally linked image taken on October 1, 2023 at 16:15:43 is associated with and stored in the diagnostic region with diagnostic region ID 004-001, and an externally linked image taken on January 18, 2024 at 15:43:21 and another taken on January 18, 2024 at 15:48:31 are associated with and stored in the diagnostic region with diagnostic region ID 005-001. It also shows that there are no externally linked images associated with the diagnostic region with diagnostic region ID 005-002 or 004-002. In this way, any number of externally linked images can be associated with each diagnostic region and stored in the externally linked image database 124.

[0024] The control unit 110 then determines whether or not to terminate the projection on the projection device 220 (step S114). This determination method is also arbitrary, but for example, if the user instructs the diagnostic support device 100 to terminate the process, the control unit 110 determines to terminate the projection. If the projection is not terminated (step S114; No), the process returns to step S107. If the projection is terminated (step S114; Yes), the control unit 110 terminates the diagnostic support process.

[0025] Through the diagnostic support processing described above, the diagnostic support device 100 can display a projected image related to the diagnostic area to the subject 300 (living body) in a recognizable manner based on the diagnostic results acquired based on the image of the diagnostic area. Moreover, since the projected image is tracked to the diagnostic area based on frame images (living body images) that are continuously acquired in real time even during projection, even if the subject 300 moves, the projected image can continue to be displayed in the correct position by following the movement. Furthermore, by using multiple candidate lesion images related to the diagnostic area when acquiring the diagnostic results, it is possible to make a more accurate diagnosis of the diagnostic area. In addition, by including a symbol indicating the diagnostic area in the projected image, the diagnostic support device 100 can make it easier to recognize where the diagnostic area is located on the subject 300. Furthermore, by representing this symbol in a manner (color, shape, type, etc.) according to the diagnostic result, the diagnostic result can be easily recognized at a glance. In addition, by not directly displaying the disease name, etc., but indicating it in the manner of a symbol, there is the advantage of not being subject to the restrictions of the Pharmaceutical Affairs Law. Furthermore, the diagnostic support device 100 projects the diagnostic area selected by the user onto the subject 300, making it easier to associate the diagnostic area displayed on the user's operating terminal (e.g., the display unit 130 of the diagnostic support device 100) with the symbol indicating the location of the diagnostic area projected onto the subject 300. In addition, since image data acquired by an external device can be linked to the diagnostic area and stored in the storage unit 120, it is possible to record a biological image that captures the overall image of the subject 300 with a wide field of view, and a detailed image of each diagnostic area (e.g., a dermoscopy (close-up) image) in association with each other.

[0026] In the diagnostic area selection screen shown in Figure 11, the control unit 110 sorted the diagnostic areas in descending order of risk score (danger level), but this is only one example of how to display a list of diagnostic areas. In the case of skin lesions, generally, the darker the color and the larger the size, the higher the risk. Therefore, when displaying a list of diagnostic areas, they may be sorted in descending order of color or in descending order of size. Furthermore, to show the time-series changes of candidate lesions in each diagnostic area, the control unit 110 may refer to the correspondence data 123 and arrange each diagnostic area in chronological order. In this case, the user will also be able to select past diagnostic areas. If a past diagnostic area is selected, the control unit 110 refers to the correspondence data 123 and projects a projection image onto the projection device 220 that points to (for example, enclosed in a selection frame 225) the diagnostic area currently projected onto the subject 300 that is associated with the selected past diagnostic area. In this way, it is possible to associate past diagnostic areas with current diagnostic areas, so even if the user finds a diagnostic area of ​​concern from past biological images, they can easily check the current state of that diagnostic area based on the projection image projected onto the subject 300. Furthermore, the display unit 130 may also display a wide-field image of the subject 300, providing a user interface (UI) that shows the correspondence between each diagnostic area and its position in the biological image. While the use of these diagnostic results is at the discretion of the individual, one possible way to utilize them is to display the distribution of each diagnostic factor (e.g., a histogram) and then show where each diagnostic area falls within that distribution, in order to obtain more useful diagnostic information.

[0027] Furthermore, although the above-described embodiment mainly described the diagnostic support for skin diseases, the scope of application of the diagnostic support device 100 is not limited to the diagnosis of skin diseases. For example, the diagnostic support device 100 can be applied to various parts of the body that can be projected onto the imaged area, such as the uterus or oral cavity.

[0028] Furthermore, in the above-described embodiment, the diagnostic support device 100, the imaging device 210, and the projection device 220 were described as separate devices constituting the diagnostic support system 200. However, some of these (for example, the diagnostic support device 100 and the imaging device 210) or all of them may be combined into a single device. Furthermore, the display unit 130 and the operation input unit 140 may exist as separate operation terminal devices connected (wirelessly or wired) to the diagnostic support device 100. Thus, the diagnostic support system 200 and the diagnostic support device 100 may have all their components integrated into a single housing, or any functional units (such as the display unit 130, operation input unit 140, imaging device 210, projection device 220, etc.) may exist as separate units as needed.

[0029] Furthermore, the diagnostic support device 100 can also be implemented using a computer such as a smartphone, tablet, or PC (Personal Computer). Specifically, in the above embodiment, it was described that the program for diagnostic support processing executed by the control unit 110 is pre-stored in the storage unit 120. However, the program may be stored and distributed on a non-temporary computer-readable recording medium such as a flexible disk, CD-ROM (Compact Disc Read Only Memory), DVD (Digital Versatile Disc), MO (Magneto-Optical disc), memory card, or USB memory, and a computer capable of executing the above-mentioned processing can be configured by loading and installing the program into the computer.

[0030] Furthermore, the program can be superimposed on a carrier wave and applied via a communication medium such as the Internet. For example, the program could be posted and distributed on a bulletin board system (BBS) on a communication network. This program could then be launched and executed under the control of the operating system (OS), just like any other application program, to perform the aforementioned processes. Furthermore, the control unit 110 may consist of any single processor, such as a single processor, multi-processor, or multi-core processor, or it may be configured by combining any of these processors with processing circuits such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field-Programmable Gate Array).

[0031] Although preferred embodiments of the present invention have been described above, the present invention is not limited to these specific embodiments, and the present invention includes the invention described in the claims and its equivalents. [Explanation of symbols]

[0032] 100...Diagnostic support device, 110...Control unit, 120...Storage unit, 130...Display unit, 140...Operation input unit, 200...Diagnostic support system, 210...Photography device, 220...Projection device

Claims

1. Based on biological images, a predetermined diagnostic area is detected, Based on the candidate lesion images related to the detected diagnostic region, a diagnostic result related to the diagnostic region is obtained. Based on the acquired diagnostic results, the projected image relating to the diagnostic area is displayed on the living body in a recognizable manner. Control unit, A diagnostic support device equipped with the following features.

2. The control unit, The aforementioned biological images are acquired in real time, Based on the acquired biological image, the projected image is tracked onto the diagnostic area of ​​the biological body. The diagnostic support device according to claim 1.

3. The control unit, The candidate lesion images for each of the aforementioned diagnostic areas are stored in the memory unit, linked to the date and time of acquisition, and are associated with past diagnostic areas. Based on the multiple candidate lesion images relating to the aforementioned associated diagnostic region, a diagnostic result relating to the diagnostic region is obtained. The diagnostic support device according to claim 1.

4. The control unit, The projected image includes a symbol indicating the diagnostic area. The diagnostic support device according to claim 1.

5. The control unit, The symbols to be included in the projected image are represented in a manner corresponding to the diagnostic result. The diagnostic support device according to claim 4.

6. The control unit, Based on the operation input entered from the operation input unit, the diagnostic area is selected. Obtain the diagnostic results related to the selected diagnostic area, Based on the acquired diagnostic results, the projected image relating to the diagnostic area is displayed on the living body in a recognizable manner. The diagnostic support device according to claim 1.

7. The control unit, Acquire image data from an external device, The acquired image data is linked to the selected diagnostic region and stored in the memory unit. The diagnostic support device according to claim 6.

8. The control unit, Based on biological images, a predetermined diagnostic area is detected, Based on the candidate lesion images related to the detected diagnostic region, a diagnostic result related to the diagnostic region is obtained. Based on the acquired diagnostic results, the projected image relating to the diagnostic area is displayed on the living body in a recognizable manner. Diagnostic support methods.

9. In the control unit, Based on biological images, a predetermined diagnostic area is detected, Based on the candidate lesion images related to the detected diagnostic region, a diagnostic result related to the diagnostic region is obtained. Based on the acquired diagnostic results, the projected image relating to the diagnostic area is displayed on the living body in a recognizable manner. A program that executes a process.

Images