Identification support apparatus, identification support method, and program

Abstract

To display information in a coordinate system having a plurality of axes on the basis of temporal change values of a plurality of pieces of predetermined diagnostic region information.SOLUTION: An identification support device 100 includes a control unit 110 that acquires a temporal change value of each of a plurality of feature amounts related to a diagnostic region on the basis of a plurality of diagnostic region images captured at different times, and displays a scatter diagram in which a temporal change value to be assigned to a first axis and a temporal change value to be assigned to a second axis are selected from the plurality of feature amounts.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an identification assistance device, an identification assistance method, and a program. [Background technology]

[0002] Conventionally, technologies for diagnosing whether biological tumors are benign or malignant have been developed. For example, a technology is known that detects tumor feature points from biological images and monitors the progression of the color and size of the biological tumor. However, when diagnosing a disease state, it is rare to diagnose the disease state using a single indicator such as the color or size of the biological tumor. Therefore, a method for diagnosing the progression and risk of a biological tumor based on more complex indicators is needed. For example, Patent Document 1 discloses a medical image processing device and the like that can improve the accuracy of identifying the malignant transformation of a tumor candidate by identifying the tumor candidate, calculating morphological information and functional information, and calculating malignant transformation feature amounts. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-147930 Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional technology such as that disclosed in Patent Document 1 does not combine multiple indicators to comprehensively visualize the risk of each tumor, and there was room for improvement in creating a user interface that displays information according to the doctor's needs (for example, a visually easy-to-understand display of the degree of change over the course of each tumor).

[0005] The present invention has been made in consideration of the above-mentioned circumstances, and aims to provide an identification assistance device, an identification assistance method, and a program that can display information in a coordinate system having multiple axes based on the time-varying values ​​of a predetermined number of diagnostic region information. [Means for solving the problem]

[0006] In order to achieve the above object, one aspect of the identification assistance device according to the present invention comprises: Obtaining time-varying values ​​of each of a plurality of feature amounts related to a diagnostic region based on a plurality of diagnostic region images taken at different times; displaying a scatter plot in which the time-varying values ​​to be assigned to a first axis and the time-varying values ​​to be assigned to a second axis are selected from the plurality of feature quantities; control unit, Equipped with. [Effects of the Invention]

[0007] According to the present invention, information can be displayed in a coordinate system having a plurality of axes based on time-varying values ​​of a plurality of pieces of predetermined diagnostic region information. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram showing a functional configuration of an identification assistance apparatus according to an embodiment; [Figure 2] FIG. 2 is a diagram illustrating an example of an image database. [Figure 3] 10 is a flowchart of an identification support process according to an embodiment. [Figure 4] FIG. 10 is a diagram illustrating an example of past image data. [Figure 5] FIG. 10 is a diagram illustrating an example of a diagnosis area database. [Figure 6] FIG. 10 is a diagram illustrating an example of current image data. [Figure 7] FIG. 2 is a diagram for explaining the correspondence between a past image and a current image. [Figure 8] FIG. 10 is a diagram illustrating an example of correspondence data. [Figure 9] FIG. 10 is a diagram illustrating correction of a past image based on a current image. [Figure 10] FIG. 10 is a diagram illustrating an example of a time-varying value database. [Figure 11]10A and 10B are diagrams showing an example of a scatter diagram in which dots are arranged as plot points and an example of an enlarged view of a diagnostic region image. [Figure 12] FIG. 10 is a diagram for explaining the magnitude of risk in a scatter diagram. [Figure 13] FIG. 10 is a diagram showing an example of a scatter diagram in which current diagnostic region images are arranged as plot points. [Figure 14] FIG. 10 is a diagram showing an example of a scatter diagram in which pairs of past diagnostic region images and current diagnostic region images are arranged as plot points. [Figure 15] FIG. 10 is a diagram showing an example of a display screen on which display axis setting and shooting date setting can be performed. DETAILED DESCRIPTION OF THE INVENTION

[0009] An identification support device and the like according to an embodiment will be described with reference to the drawings, in which the same or corresponding parts are designated by the same reference numerals. The identification support device 100 according to the embodiment visually displays the magnitude of change over time in an image showing a diagnostic region (e.g., an image of a skin tumor). The identification support device 100 captures, for example, dermoscopy images used in dermatological examinations at multiple times and visually displays the magnitude of change over time in diagnostic region information in the image. This allows users (doctors, etc.) to more easily understand the risk level, etc., of a patient's diagnostic region than ever before. The diagnostic region includes not only areas (e.g., skin flakes) that show biological changes (lesions) caused by disease, but also areas that show symptomatic changes before the onset of disease. In other words, the diagnostic region includes all areas that a doctor intends to diagnose (diagnostic locations), regardless of the progression of the disease, and even areas where it is unclear whether they are diseased. The diagnostic region information refers to various information that indicates the characteristics of the diagnostic region, such as the size, color, and malignancy (probability of malignancy) of the diagnostic region.

[0010] As shown in FIG. 1, the identification assistance device 100 includes, as its functional components, a control unit 110, a storage unit 120, an imaging unit 130, a display unit 140, and an input unit 150. The control unit 110 is configured with a processor such as a CPU (Central Processing Unit), etc. The control unit 110 executes a classification support process, which will be described later, using a program stored in the storage unit 120. The storage unit 120 is configured with, for example, a random access memory (RAM), a read only memory (ROM), a flash memory, etc., and stores the programs executed by the control unit 110 and necessary data.

[0011] The imaging unit 130 includes an imaging element such as a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor. The imaging unit 130 captures, for example, an image of the patient's skin and acquires the image data. The control unit 110 acquires image data of the patient's skin using the imaging unit 130 based on, for example, an image capture instruction from a user, and stores the acquired image data together with the image capture date and time in the storage unit 120. As a result, an image database 121 is constructed in the storage unit 120, as shown in FIG. 2, for example, in which image data of the skin of a patient identified by a patient ID (Identification) captured by the imaging unit 130 is stored together with the patient ID, image ID, and image capture date and time.

[0012] The display unit 140 includes a display such as a liquid crystal display, an organic EL (Electro-Luminescence) display, etc. The display unit 140 displays images captured by the imaging unit 130, scatter diagrams (described later), and the like. The input unit 150 is a user interface such as a keyboard, a mouse, a touch panel, etc., and receives operation input from a user. When the input unit 150 includes a touch panel, the touch panel may be integrated with the display of the display unit 140.

[0013] The functional configuration of the identification support device 100 has been described above. The control unit 110 detects a diagnostic area from image data captured by the imaging unit 130 and displays a scatter plot in which the values ​​of time-varying changes in multiple feature quantities (e.g., size, color, malignancy level, etc.) related to the diagnostic area are assigned to separate axes. The identification support process, which is the process performed to achieve this, will be described with reference to FIG. 3. This process is initiated by a user's instruction. For example, if the user wishes to display a scatter plot based on a diagnostic area image, the user executes this process. Note that, before the identification support process is executed, it is assumed that past image data of the patient's skin, etc., is stored in the storage unit 120 as the image database 121. For example, the patient has their skin photographed during a regular checkup, and the photographed image data is stored in the image database 121.

[0014] First, the control unit 110 acquires past image data from the storage unit 120 (step S101). In this step, the control unit 110 acquires image data such as that shown in FIG. 4. The control unit 110 then detects diagnostic regions 200, 201, and 202 from the past image data (step S102). The control unit 110 then stores the images of the detected diagnostic regions 200, 201, and 202 in the storage unit 120 as a diagnostic region database 122, as shown in FIG. 5, in which the images are linked with a patient ID, an image ID, a capture date and time, a position in the entire image (the upper left x-y coordinates and the lower right x-y coordinates of the rectangular region from which the diagnostic region is extracted), a diagnostic region ID (in FIG. 5, an ID obtained by adding the ID of the diagnostic region included in the image to the image ID), and feature values ​​(size, color value, malignancy level, etc., as described below). However, some or all of the feature values ​​may be obtained in step S102, or may be obtained in step S108 or later, as described below. In the example shown in Fig. 5, it can be seen that the diagnostic area 201 in the image data shown in Fig. 4 is detected as the diagnostic area with diagnostic area ID = 002-001, and the diagnostic area 202 is detected as the diagnostic area with diagnostic area ID = 002-002. Any method can be used to detect these diagnostic areas 200, 201, and 202, but several methods will be described below.

[0015] For example, the first method can be a method consisting of the following three steps. Step 1: Image data (whole image) is converted into a grayscale image, binarized using a certain threshold, and closed curves are detected based on the boundary between values. Step 2: The process of step 1 is repeated by changing the threshold value for binarization, and the closed curves with close center coordinates are grouped together. Step 3: The median of the center position and size of each group is calculated and used as the position and size of each diagnostic region. A second method is to use an object detection model such as R-CNN (Region-Convolutional Neural Network) or YOLO (You Only Look Once). However, the above is merely an example, and the detection method for the diagnostic regions 200, 201, and 202 is not limited to the two methods mentioned here.

[0016] Next, the control unit 110 acquires current image data from the storage unit 120 or the imaging unit 130 (step S103). In this step, the control unit 110 acquires current image data (for example, a current image as shown in FIG. 6) that is slightly different from past image data (for example, a past image as shown in FIG. 4). Then, the control unit 110 detects a diagnostic region 200 from the current image data (step S104). The method for detecting the diagnostic region 200 in step S104 is the same as the detection method in step S102, and information about the diagnostic region 200 detected in step S104 is also additionally recorded in the diagnostic region database 122.

[0017] The control unit 110 then detects correspondence between the diagnostic areas in the previous image and the current image, as indicated by the dashed line in FIG. 7 (step S105). While any method for detecting correspondence between diagnostic areas can be used, one example is to calculate image features for each diagnostic area and determine correspondence between diagnostic areas with similar distances between the features. For diagnostic areas for which correspondence cannot be determined using image features, a method can be used to determine correspondence between diagnostic areas with similar centroid positions. By determining correspondence between diagnostic areas with similar centroid positions, correspondence can be achieved even when the distance between image features is large, such as when the color or size of diagnostic areas changes significantly between the previous and current images. Furthermore, instead of performing correspondence in two steps (determining correspondence using image features and then determining correspondence using centroid positions) as described above, correspondence detection can be performed using features that include not only image features but also the coordinates of centroid positions, thereby determining correspondence between diagnostic areas with similar distances between features. This method increases the likelihood of determining correct correspondence in a single step, even when the color or size of diagnostic areas changes significantly.

[0018] In step S105, the control unit 110 recognizes a diagnostic area in the current image that could not be associated with a diagnostic area in the previous image as a newly emerged diagnostic area (new diagnostic area). For example, diagnostic area 6 in the current image in FIG. 7 is recognized as a new diagnostic area. The control unit 110 stores the correspondence detected in step S105 in the storage unit 120 as correspondence data 123 as shown in FIG. 8. The "labels" shown in FIG. 8 are used to indicate each diagnostic area to the user (for example, to be displayed near the plot point of each diagnostic area when a scatter plot is displayed on the display unit 140). Here, they are automatically set as "diagnostic area 1," "diagnostic area 2," ... in ascending order of the current diagnostic area ID (the portion excluding the image ID), but the user may set them freely. A diagnostic area that corresponds between past image data and current image data, such as diagnostic area 1 to diagnostic area 5 shown in Figure 7 (a diagnostic area in which both a past diagnostic area ID and a current diagnostic area ID exist in the correspondence data 123), is called a corresponding diagnostic area, and a diagnostic area that does not exist in past image data but exists in current image data, such as diagnostic area 6 shown in Figure 7 (a diagnostic area in which only a current diagnostic area ID exists in the correspondence data 123), is called a new diagnostic area.

[0019] Next, the control unit 110 corrects the past image data based on the positions of the corresponding diagnostic regions (step S106). In this step, the control unit 110 corrects (transforms) the past image data so that the positions of the diagnostic regions in the past image data match the positions of the corresponding diagnostic regions in the current image data, as shown in FIG. 9. Any method for correcting the image data may be used, for example, morphological transformation. Correcting the past image data in this manner makes it easier to detect changes in the size of the diagnostic regions compared to the current image data and to confirm whether the correspondence is correct. Note that the control unit 110 may detect changes in the size of the diagnostic regions based on the image database 121, diagnostic region database 122, correspondence data 123, etc. stored in the storage unit 120 without correcting the past image data. In this case, the control unit 110 does not need to perform the processing of step S106.

[0020] Next, the control unit 110 determines whether there is a corresponding diagnostic region (an unprocessed corresponding diagnostic region) for which the processes of steps S108 and S109, which will be described later, have not been performed (step S107). If there is no unprocessed corresponding diagnostic region (step S107; No), the process proceeds to step S110. If there is an unprocessed corresponding diagnostic region (step S107; Yes), the control unit 110 selects one of the unprocessed corresponding diagnostic regions and acquires the magnitude of size change (size change value) of the corresponding diagnostic region (step S108). The method for acquiring the size change value of the corresponding diagnostic region is arbitrary. For example, the size of the corresponding diagnostic region in the past image data and the size of the corresponding diagnostic region in the current image data are calculated, and the size change value of the corresponding diagnostic region is acquired by calculating the ratio (size of the current corresponding diagnostic region ÷ size of the past corresponding diagnostic region). Note that the control unit 110 may calculate the difference (size of the current corresponding diagnostic region - size of the past corresponding diagnostic region) instead of the ratio as the size change value of the corresponding diagnostic region. The size of the corresponding diagnostic region can be calculated by any method, for example, by counting the number of pixels whose pixel values ​​differ (enough to be determined as a diagnostic region) in an area including the diagnostic region of the image data.The control unit 110 then associates the calculated size with the image of the corresponding diagnostic region as one of the feature quantities of the corresponding diagnostic region and stores it in the diagnostic region database 122 of the storage unit 120, and stores the calculated size change value in the time-varying value database 124 as shown in FIG.

[0021] Next, the control unit 110 acquires the magnitude of the color change (color change value) of the corresponding diagnostic area (step S109) and returns to step S107. The method of acquiring the color change value of the corresponding diagnostic area may be arbitrary. For example, the color change value of the corresponding diagnostic area may be acquired by calculating the blackness of the corresponding diagnostic area of ​​the past image data and the blackness of the corresponding diagnostic area of ​​the current image data, and then calculating the difference (blackness of the current corresponding diagnostic area - blackness of the past corresponding diagnostic area). Note that the control unit 110 may also acquire the ratio (blackness of the current corresponding diagnostic area ÷ blackness of the past corresponding diagnostic area) instead of the difference as the color change value of the corresponding diagnostic area. The method for calculating the blackness can be arbitrary. One example is to calculate the luminance value Yl of the diagnostic area and the luminance value Ys of the skin from the median RGB (Red, Green, Blue) values ​​(Rl, Gl, Bl) of pixels belonging to the area considered to be the diagnostic area and the median RGB values ​​(Rs, Gs, Bs) of pixels of the skin considered not to be the diagnostic area, respectively, and calculate the blackness as (Ys - Yl) / Ys. Here, the luminance value Y can be calculated from the RGB values ​​as Y = 0.299 × R + 0.578 × G + 0.114 × B. The control unit 110 then associates the calculated blackness (color value) with the image of the corresponding diagnostic area as one of the feature quantities of the corresponding diagnostic area and stores it in the diagnostic area database 122 of the storage unit 120. The control unit 110 also stores the calculated color change value in the time-varying value database 124.

[0022] In step S110, the control unit 110 determines whether there is a new diagnostic region (an unprocessed new diagnostic region) that has not yet been processed in steps S111 and S112 (described later). If there is no unprocessed new diagnostic region (step S110; No), the process proceeds to step S113. If there is an unprocessed new diagnostic region (step S110; Yes), the control unit 110 selects one of the unprocessed new diagnostic regions and acquires the size of the new diagnostic region (step S111). The method for acquiring the size of the new diagnostic region is the same as the method for determining the size of the corresponding diagnostic region in step S108 described above. The control unit 110 then associates the acquired size with the image of the new diagnostic region as one of the feature parameters of the new diagnostic region and stores it in the diagnostic region database 122 of the storage unit 120. Note that, although it is not possible to obtain a size change value for a new diagnostic region in the strict sense, in the example shown in FIG. 10, the size of the past diagnostic region is considered to be 0, and the size change value is stored as +∞ in the time-varying value database 124.

[0023] Next, the control unit 110 acquires the color value of the new diagnostic region (step S112) and returns to step S110. The method for acquiring the color value of the new diagnostic region is the same as the method for calculating the blackness in step S109 described above. The control unit 110 then associates the acquired blackness (color value) with the image of the new diagnostic region as one of the feature quantities of the new diagnostic region and stores it in the diagnostic region database 122 of the storage unit 120. Note that, although it is not possible to calculate a color change value for the new diagnostic region in the strict sense, in the example shown in FIG. 10, the color value of the past diagnostic region is considered to be 0, and the color change value is stored as +∞ in the time-change value database 124. The control unit 110 may also estimate time-change values ​​such as size change values ​​and color change values ​​from only the new diagnostic region, for example, by using a deep neural network (DNN) trained on a huge amount of image data, and store the estimated time-change values ​​in the time-change value database 124.

[0024] In step S113, the control unit 110 displays each diagnostic region on a scatter diagram as shown in FIG. 11 by arranging plot points based on the previously acquired size change and color change values ​​of the corresponding diagnostic region and the size and color change values ​​of the new diagnostic region, and then terminates the identification support process. In the example shown in FIG. 11, the horizontal axis is assigned the size change value, and the vertical axis is assigned the color change value. However, for the new diagnostic region, the size change and color change values ​​are defined assuming that the previous size and color values ​​are zero. In this case, there is no problem when calculating the change value as a difference, but calculating it as a ratio results in division by zero. Therefore, when the change value is calculated as a ratio, the size change and color change values ​​of the new diagnostic region are set to the maximum value on the scatter diagram. Furthermore, in the example shown in FIG. 11, when the user selects one of the diagnostic regions plotted on the scatter diagram by clicking, the control unit 110 displays enlarged images of the previous and current images of the selected diagnostic region so that the previous and current images of the selected diagnostic region can be compared. FIG. 11 shows an example in which diagnostic region 3 is selected.

[0025] By performing the above-described identification support process, the identification support device 100 assigns time-varying values ​​of feature quantities related to diagnostic regions, such as size change values ​​and color change values, to the first and second axes of a scatter plot, respectively, and displays information in a coordinate system having multiple axes based on the time-varying values ​​of a predetermined number of diagnostic region information. This display makes it possible to visualize objective diagnostic results of biological tumors and present the risk.

[0026] It is believed that the larger the size and the darker the color of a diagnostic region (the darker the color), the higher the risk. Therefore, as shown in FIG. 12, the larger the size change value and color change value (those in the upper right corner of the scatter plot), the higher the risk level of that diagnostic region. When examining a diagnostic region, the higher the risk level, the more important it is. Therefore, a person checking a diagnostic region (such as a doctor) should prioritize checking the diagnostic region in the upper right corner of the scatter plot from among many diagnostic regions. Note that the risk level may be calculated by weighting the sum of the size change value and the color change value or the weighting multiplication of the size change value and the color change value, and the risk level value may be displayed near each plot point on the scatter plot.

[0027] 11 shows an example in which a diagnostic region image is displayed by selecting a plot point on a scatter plot. However, the control unit 110 may display a scatter plot in which each diagnostic region image is arranged as a plot point, as shown in FIG. 13. By arranging the diagnostic region images as plot points, the user can simultaneously grasp the image of the diagnostic region and its risk level, thereby enabling more efficient diagnosis. While the current diagnostic region image is arranged as a plot point in FIG. 13, a past diagnostic region image may also be arranged as a plot point. Alternatively, whether the image to be arranged as a plot point is the current diagnostic region image or a past diagnostic region image may be switched in response to a user instruction or at predetermined time intervals (e.g., one second).

[0028] Furthermore, the control unit 110 may arrange a pair of past and current diagnostic region images side by side as plot points, as shown in FIG. 14 . This allows the user to more easily grasp changes in the diagnostic region compared to when only the current or past diagnostic region images are arranged as plot points. Although the pair of past and current diagnostic region images is displayed side by side in FIG. 14 , they may also be displayed superimposed (overlapping the past and current diagnostic region images). This allows the user to more easily grasp changes in the size and shape of the diagnostic region. Furthermore, when superimposing, the transparency may be changed, for example, the greater the time difference between the past and current images (the longer the elapsed time from the reference time to the diagnostic time, the lighter the display of the past diagnostic region image). This allows the user to intuitively grasp the length of the period (elapsed time) from the reference time to the diagnostic time.

[0029] 13 and 14, the rectangular borders of the diagnostic region images are displayed in black. However, the rectangular borders of the diagnostic region images may be color-coded depending on the type and nature of the skin disease associated with the diagnostic region, the magnitude of change in the feature values ​​(e.g., color, size, malignancy, etc.). This allows the user to grasp various information associated with each diagnostic region that cannot be directly represented as a scatter plot from the color of the border. Note that the border color used here is merely an example, and the background color of the diagnostic region image may be color-coded instead of or in addition to the border color. Furthermore, instead of or in addition to the border color, the style of the border (solid line, dashed line, dotted line, dotted line, etc.) may be changed depending on the type and nature of the skin disease associated with the diagnostic region, the magnitude of change in the feature values ​​(e.g., color, size, malignancy, etc.).

[0030] In the above description of the classification support process (FIG. 3), size change values ​​and color change values ​​are assigned to each axis of the scatter plot as time-dependent change values ​​for each of multiple feature quantities related to the diagnostic region. However, the feature quantities related to the diagnostic region are not limited to size and color. The time-dependent change value of any feature quantity related to the diagnostic region can be assigned to any axis of the scatter plot, as long as the feature quantity is considered to indicate a higher risk level for the diagnostic region as the time-dependent change value increases. For example, the control unit 110 may calculate the malignancy level from an image of each diagnostic region, calculate a malignancy level change value based on the malignancy level of the diagnostic region in a previous image and the malignancy level of the current diagnostic region (e.g., "current malignancy level ÷ previous malignancy level" or "current malignancy level - previous malignancy level"), and assign the malignancy level change value to any axis of the scatter plot. The malignancy level can be calculated in any manner, including, for example, using a benign / malignant classifier trained using a deep learning model on a large number of malignant diagnostic region images and benign diagnostic region images. For example, it may be possible to set which feature change values ​​are to be assigned to the X-axis and Y-axis, as in the display axis setting 141 shown in Fig. 15. In this way, by generating a scatter plot by selecting multiple types of feature values ​​for which time-varying values ​​are to be calculated from among size, color, malignancy, etc., the user can determine the risk of the diagnostic region from a more multifaceted perspective.

[0031] Furthermore, although not shown, three axes, namely, X-axis, Y-axis, and Z-axis, may be prepared as the axes of the scatter diagram, and one of three types of time-varying values ​​of feature quantities (e.g., size change value, color change value, and malignancy change value) may be assigned to each axis to construct a three-dimensional scatter diagram. In this way, by using a three-dimensional scatter diagram with time-varying values ​​(e.g., size change value) assigned to the first axis, time-varying values ​​(e.g., color change value) assigned to the second axis, and time-varying values ​​(e.g., malignancy change value) assigned to the third axis, the user can grasp the time-varying values ​​of three types of feature quantities at once, thereby more efficiently making a multifaceted judgment of the risk of the diagnostic region.

[0032] In the above description, the two images used to calculate the time-change value were referred to as a past image and a current image. However, both the capture dates of the past image and the current image are past dates relative to the time point at which the scatter plot was displayed. The past image in the above description is the reference image used to calculate the magnitude of the time-change value, and the current image can be referred to as the image used for diagnosis. Therefore, the capture date of the past image will also be referred to as the reference date, and the capture date of the current image will also be referred to as the diagnostic date. The reference date, which is the capture date (date) of the past image, and the diagnostic date, which is the capture date of the current image, can be any date as long as they are different from each other. For example, as in the image capture date setting 142 shown in FIG. 15, two capture dates may be selected from the image data stored in the storage unit 120. In the example shown in FIG. 15, October 2022 and January 2024 are selected. In this case, of these two dates, October 2022, which is the earlier date, is set as the reference date, and January 2024, which is closer to the present date, is set as the diagnostic date. Then, a scatter diagram 143 is displayed based on the change values ​​(size change values ​​and color change values ​​in FIG. 15) of the feature amounts of the diagnostic area images of October 2022 and January 2024.

[0033] In the scatter diagram 143, plot points are arranged based on the magnitude of the change in the feature value over time set in the display axis setting 141. Therefore, if the reference time or diagnostic time is changed in the imaging date setting 142, the change in the feature value over time changes accordingly, and the positions at which the plot points are arranged also change. Therefore, the user can grasp the change in the risk level of each diagnostic area according to the time based on the change in the arrangement of each diagnostic area changed in the imaging date setting 142. Furthermore, by selecting any diagnostic area, diagnostic area images of that diagnostic area (diagnostic area 4 in FIG. 15) for each time stored in the storage unit 120 are displayed in a list in the display frame 144, so the user can set the reference time and diagnostic time while viewing the diagnostic area images for each time.

[0034] In the above description, the diagnostic region image is described as a diagnostic region image of a skin disease, but the diagnostic region image is not limited to a diagnostic region image of a skin disease. The identification support device 100 can be applied to a diagnostic region image of any disease as long as the magnitude of the change in the feature value over time related to the diagnostic region is considered to be related to the risk of the diagnostic region.

[0035] In the above-described embodiment, the identification assistance device 100 is described as including the imaging unit 130 and capable of capturing an image of a diagnostic area for a user's skin disease by itself. However, the imaging unit 130 may exist as a separate device from the identification assistance device 100. For example, in an identification assistance system including a camera including the imaging unit 130 and an identification assistance device configured by removing the imaging unit 130 from the identification assistance device 100 (camera-less identification assistance device), the control unit 110 may acquire a diagnostic area image, etc. from the imaging unit 130 of the camera and perform processing similar to that of the above-described identification assistance device 100.

[0036] Furthermore, the display unit 140 may be configured to exist as a separate display device connected (wirelessly or wired) to the identification assistance device 100. In this way, the identification assistance device 100 does not need to have all of its components in a single housing, and any functional unit (such as the imaging unit 130, display unit 140, or input unit 150) may exist as a separate unit as needed.

[0037] Furthermore, the identification assistance device 100 can also be realized by a computer such as a smartphone, a tablet, or a PC (Personal Computer). Specifically, in the above embodiment, it has been described that the program for the identification assistance process and the like executed by the control unit 110 is pre-stored in the storage unit 120. However, the program may be stored and distributed on a non-transitory computer-readable recording medium such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), a DVD (Digital Versatile Disc), an MO (Magneto-Optical disc), a memory card, or a USB memory, and the program may be read and installed on a computer to configure a computer capable of executing each of the above-described processes.

[0038] Furthermore, the program may be superimposed on a carrier wave and applied via a communication medium such as the Internet. For example, the program may be posted and distributed on a bulletin board system (BBS) on a communication network. The program may then be started and executed under the control of an operating system (OS) in the same way as other application programs, thereby enabling the above-described processes to be performed. In addition, the control unit 110 may be configured by any single processor such as a single processor, multiprocessor, or multi-core processor, or by combining any of these processors with processing circuits such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field-Programmable Gate Array).

[0039] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to such specific embodiments, and the present invention includes the inventions described in the claims and their equivalents. [Explanation of symbols]

[0040] 100... Identification assistance device, 110... Control unit, 120... Storage unit, 130... Imaging unit, 140... Display unit, 150... Input unit

Claims

1. Obtaining time-varying values ​​of each of a plurality of feature amounts related to a diagnostic region based on a plurality of diagnostic region images taken at different times; displaying a scatter plot in which the time-varying values ​​to be assigned to a first axis and the time-varying values ​​to be assigned to a second axis are selected from the plurality of feature quantities; control unit, An identification assistance device comprising:

2. The control unit Accept input of two periods from the periods when the diagnostic region images were taken. Of the two accepted time periods, the earlier time period is set as the reference time period, and the time period closer to the present time period is set as the diagnosis time period; acquiring a change value of the feature amount of the diagnostic region image from the reference time to the diagnostic time as the time-varying value; The identification assistance device according to claim 1 .

3. The control unit Accepts input of two features from color, size, and malignancy, assigning one of the two received feature amounts to the first axis and the other to the second axis; The identification assistance device according to claim 1 .

4. The control unit placing the diagnostic region images as plot points on the scatter plot; The identification assistance device according to claim 1 .

5. The control unit a pair of images consisting of the diagnostic region image at the reference time and the diagnostic region image at the diagnostic time are arranged on the scatter diagram as plot points; The identification assistance device according to claim 2 .

6. The control unit an image obtained by superimposing the diagnostic region image at the reference time on the diagnostic region image at the diagnostic time is arranged on the scatter diagram as plot points; The identification assistance device according to claim 2 .

7. The control unit changing transparency of the diagnostic region image at the reference time when the diagnostic region image at the reference time is superimposed on the diagnostic region image at the diagnostic time according to the length of time elapsed from the reference time to the diagnostic time; The identification assistance device according to claim 6.

8. The control unit further selecting the time-varying value to be assigned to a third axis from among the plurality of feature quantities; displaying a scatter plot of the time-varying values ​​assigned to a first axis, the time-varying values ​​assigned to a second axis, and the time-varying values ​​assigned to a third axis; The identification assistance device according to claim 1 .

9. The control unit Obtaining time-varying values ​​of each of a plurality of feature amounts related to a diagnostic region based on a plurality of diagnostic region images taken at different times; displaying a scatter plot in which the time-varying values ​​to be assigned to a first axis and the time-varying values ​​to be assigned to a second axis are selected from the plurality of feature quantities; Identification aid methods.

10. In the control section, Obtaining time-varying values ​​of each of a plurality of feature amounts related to a diagnostic region based on a plurality of diagnostic region images taken at different times; displaying a scatter plot in which the time-varying values ​​to be assigned to a first axis and the time-varying values ​​to be assigned to a second axis are selected from the plurality of feature quantities; A program that executes a process.

Images