Image recording device, information processing device, information processing method, and recording medium

Abstract

The medical device comprises: an input unit (11) for acquiring a medical image; an identification unit (121) for identifying the medical image acquired in the input unit and acquiring the identification result; a user action analysis unit (122) for acquiring the action analysis result by analyzing the action related to the user's interpretation of the medical image; a comparison unit (123) for comparing the identification result acquired in the identification unit with the action analysis result acquired in the user action analysis unit and acquiring the comparison result; and a recording unit (124) for storing the medical image and information about the comparison result acquired in the comparison unit.

Description

Technical Field

[0001] The present invention relates to an image recording device, an information processing device, an information processing method, and a recording medium. Background Art

[0002] In the medical field, diagnostic imaging has developed various systems for capturing medical images of various anatomical structures associated with individual patients in order to classify and evaluate disease conditions. Known imaging systems include endoscope systems, CT (computed tomography) systems, MRI (magnetic resonance imaging) systems, X-ray systems, ultrasound systems, and PET (positron emission tomography) systems.

[0003] In addition, Japanese Patent Application Publication No. 2007-528746 discloses a method for implementing a lesion detection function based on so-called computer-aided detection / diagnosis assistance, CADe / x (Computer-Aided Detection / Diagnosis), by performing machine learning on annotation data of medical personnel such as doctors.

[0004] However, improving the performance of machine learning-based identifiers like the one described above typically requires a large amount of data. Therefore, in systems where machine learning is required for the identifier, the amount of data to be processed is expected to increase. However, large amounts of data require significant storage capacity, and network lines are tied up during data transmission. Therefore, it is expected that "efficient data collection" will become increasingly necessary in the future. For example, one approach to "efficient data collection" is to collect only "medical images that contain objects that are difficult to identify through machine learning" as useful data.

[0005] Here, as a technology for selecting useful medical images from a large number of medical images, for example, Japanese Patent No. 5048286 discloses a technology for efficiently transmitting only medical images that capture a desired part from a plurality of medical images.

[0006] However, the technology described in Japanese Patent No. 5048286 is merely a technique for efficiently transmitting medical images capturing a desired area. Therefore, it is believed that it is not possible to use the technology described in Japanese Patent No. 5048286 to select "medical images difficult to recognize through machine learning" as described above. In other words, existing technologies make it difficult to collect only "medical images difficult to recognize through machine learning" as useful data.

[0007] The present invention has been completed in view of the above situation, and its purpose is to provide an image recording device, an information processing device, an information processing method and a recording medium that can accurately select medical images that are difficult to identify through machine learning. Summary of the Invention

[0008] Means for solving problems

[0009] One embodiment of the present invention is an information processing device comprising: an input unit that acquires a medical image; an identification unit that identifies the medical image acquired in the input unit and acquires an identification result; a user motion analysis unit that acquires a motion analysis result by analyzing motions related to the user's interpretation of the medical image; and a comparison unit that compares the recognition result acquired in the identification unit with the motion analysis result acquired in the user motion analysis unit to acquire a comparison result regarding whether the recognition result and the motion analysis result are consistent or inconsistent.

[0010] One embodiment of the present invention is an image recording device, comprising: an input unit that acquires a medical image; an identification unit that identifies the medical image acquired in the input unit and acquires an identification result; a user motion analysis unit that acquires a motion analysis result by analyzing motions related to the user's interpretation of the medical image; a comparison unit that compares the recognition result acquired in the identification unit with the motion analysis result acquired in the user motion analysis unit and acquires a comparison result of whether the recognition result and the motion analysis result are consistent or inconsistent; and a recording unit that stores the medical image and information about the comparison result acquired in the comparison unit.

[0011] An information processing method according to one embodiment of the present invention comprises the following steps: an input step for acquiring a medical image; a recognition step for recognizing the medical image acquired in the input step and acquiring a recognition result; a user motion analysis step for acquiring a motion analysis result by analyzing the motion related to the user's interpretation of the medical image; and a comparison step for comparing the recognition result acquired in the recognition step with the motion analysis result acquired in the user motion analysis step to acquire a comparison result regarding whether the recognition result and the motion analysis result are consistent or inconsistent.

[0012] One embodiment of the present invention is a recording medium, which records an information processing program, which enables a computer to execute the following steps: an input step, obtaining a medical image; an identification step, identifying the medical image obtained in the input step and obtaining an identification result; a user action analysis step, obtaining an action analysis result by analyzing actions related to the user's interpretation of the medical image; and a comparison step, comparing the recognition result obtained in the identification step with the action analysis result obtained in the user action analysis step, and obtaining a comparison result regarding whether the recognition result and the action analysis result are consistent or inconsistent. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a block diagram showing the configuration of a medical system including the image recording apparatus according to the first embodiment of the present invention.

[0014] Figure 2 This is an explanatory diagram schematically illustrating the operation of the image recording device according to the first embodiment.

[0015] Figure 3 This is a flowchart showing the operation of the image recording device according to the first embodiment.

[0016] Figure 4 This is a block diagram showing the configuration of a medical system including a modified example of the image recording apparatus according to the first embodiment.

[0017] Figure 5 This is a block diagram showing the configuration of a medical system including an image recording apparatus according to a second embodiment of the present invention.

[0018] Figure 6 This is a flowchart showing the operation of the image recording device according to the second embodiment.

[0019] Figure 7 This is a block diagram showing the configuration of a medical system including an image recording apparatus according to a third embodiment of the present invention.

[0020] Figure 8 This is an explanatory diagram schematically illustrating the operation of the image recording device according to the third embodiment.

[0021] Figure 9 This is a flowchart showing the operation of the image recording device according to the third embodiment.

[0022] Figure 10 This is a block diagram showing the configuration of a medical system including an image recording apparatus according to a fourth embodiment of the present invention.

[0023] Figure 11 This is an explanatory diagram schematically illustrating the operation of the image recording device according to the fourth embodiment.

[0024] Figure 12 This is a flowchart showing the operation of the image recording device according to the fourth embodiment.

[0025] Figure 13 This is a block diagram showing the configuration of a medical system including an image recording apparatus according to a fifth embodiment of the present invention.

[0026] Figure 14 This is an explanatory diagram schematically illustrating the operation of the image recording device according to the fifth embodiment.

[0027] Figure 15 This is a flowchart showing the operation of the image recording device according to the fifth embodiment.

[0028] Figure 16 This is a block diagram showing the configuration of a medical system including an image recording apparatus according to a sixth embodiment of the present invention.

[0029] Figure 17 This is a flowchart showing the operation of the image recording device according to the sixth embodiment.

[0030] Figure 18 This is a block diagram showing the configuration of a medical system including an image recording apparatus according to a seventh embodiment of the present invention. DETAILED DESCRIPTION

[0031] Hereinafter, embodiments of the present invention will be described using the drawings.

[0032] <First embodiment>

[0033] Figure 1 is a block diagram showing the configuration of a medical system including an image recording device (information processing device) according to a first embodiment of the present invention. Figure 2 This is an explanatory diagram schematically illustrating the operation of the image recording device according to the first embodiment. Figure 3 This is a flowchart showing the operation of the image recording device according to the first embodiment.

[0034] like Figure 1 As shown, the medical system 1 including the image recording device 110 of the first embodiment mainly includes the image recording device 110 for acquiring medical images and performing prescribed image processing, a storage unit 31 connected to the image recording device 110 and storing prescribed data, and a display unit 32 for displaying the medical images after image processing is performed in the image recording device 110.

[0035] In addition, the medical system 1 can be a system that includes various peripheral devices related to diagnosis and treatment in addition to an endoscope having a camera unit for capturing the medical image and a light source device for providing prescribed illumination light to the endoscope. In addition, it can also be a network system that is widely used in medical sites, etc., and is a system that constitutes a part of a network system for sharing patient information including medical images, etc. to conduct medical services.

[0036] In addition, the medical images in the present invention mainly refer to images (for example, endoscopic images, ultrasonic examination images, camera images, etc.) obtained by the doctor as the user himself using prescribed medical equipment (for example, various imaging devices such as medical endoscope devices and medical ultrasonic examination devices) in accordance with his own medical orders (treatment guidelines, etc.).

[0037] Alternatively, a medical image may be officially recognized image information obtained by a prescribed medical practitioner, etc., in accordance with prescribed settings and formats, upon receiving a doctor's order (commission or instruction, etc.). Furthermore, medical practitioners, etc., include various qualified individuals related to medical care (so-called medical personnel), including doctors, nurses, and various professional technicians.

[0038] The image recording device 110 is connected to, for example, the endoscope described above, and primarily includes an input unit 11 for acquiring medical images captured by the imaging unit of the endoscope; a control unit 12 for controlling the overall operation of the image recording device 110; and a computing unit 115 for performing various processes, described below, on the medical images acquired by the input unit 11. In this embodiment, the image recording device 110 includes, in addition to an image processing unit (not shown) for performing predetermined image processing on the acquired medical images, a memory (not shown) for storing various programs, and the like.

[0039] The input unit 11 acquires, for example, an endoscopic image (medical image) captured by an imaging unit of a medical endoscope. Furthermore, as described above, the medical image is not limited to an endoscopic image, and may be an image acquired using other medical equipment (e.g., various imaging devices such as an ultrasonic examination device).

[0040] The control unit 12 is implemented by hardware such as a CPU, and reads various programs stored in the above-mentioned memory. It instructs the various parts constituting the image recording device 110 and transmits data in accordance with the image data related to the medical image input from the input unit 11, the operation signal input from the specified input operation unit, etc., thereby uniformly controlling the overall operation of the image recording device 110.

[0041] The calculation unit 115 is configured to include various circuits representing the present invention, an identification unit (identification device) 121, a user action analysis unit 122, a comparison unit 123, a recording unit 124, etc., but the details will be described later.

[0042] The storage unit 31 is an external data storage unit connected to the image recording device 110 and is implemented by various types of memory such as flash memory that can update and record data, a hard disk, an SSD, or an information recording medium such as a CD-ROM, and a reading device thereof. Alternatively, the storage unit 31 may be a file server installed at a medical facility such as a hospital via an internal network (intra-hospital network) (not shown).

[0043] The display unit 32 is implemented by a display device such as an LCD or an EL display, and displays the medical image under the control of the control unit 12.

[0044] In this embodiment, the various components of the image recording device 110, such as the computing unit 115 and the control unit 12, may be configured as electronic circuits or as circuit blocks within an integrated circuit such as an FPGA (Field Programmable Gate Array). Furthermore, for example, the image recording device 110 may include one or more processors (CPUs, etc.).

[0045] <Calculation Unit 115 in First Embodiment>

[0046] Next, the detailed configuration of the calculation unit 115 in the first embodiment will be described.

[0047] The operation unit 115 includes: an identification unit 121, which identifies medical images such as endoscopic images obtained in the input unit 11 and obtains the identification result; a user motion analysis unit 122, which obtains the motion analysis result by analyzing the motion related to the doctor's (user's) interpretation of the medical image; a comparison unit 123, which compares the identification result obtained in the identification unit 121 and the motion analysis result obtained in the user motion analysis unit 122 to obtain the comparison result; and a recording unit 124, which stores the medical image and information about the comparison result obtained in the comparison unit 123.

[0048] The recognition unit 121 detects or classifies objects from a group of acquired medical images, such as endoscopic images, based on the purpose of the examination. Identifying Medical Images. In this embodiment, the recognition unit 121 includes either or both of the detection unit 21a and the classification unit 21b as identifiers of medical images corresponding to the purpose of the examination.

[0049] The detection unit 21a examines the medical image group such as the endoscopic image obtained in the input unit 11 as an inspection image group, and detects a specified abnormal area from these medical image groups. The abnormal area is, for example, an area where a specified lesion exists. For example, when the detection unit 21a detects the existence of a specified lesion, it sends the medical image containing the lesion as a recognition result to the subsequent comparison unit 123 (refer to Figure 2 ).

[0050] On the other hand, the recognition unit 121 may also include a classification unit 21b as a medical image identifier, or include both the detection unit 21a and the classification unit 21b as medical image identifiers, depending on the purpose of the examination. The classification unit 21b receives the input of the above-mentioned medical image group obtained by the input unit 11 and performs classification corresponding to the examination purpose. The classification unit 21b sends the classification result corresponding to the diagnostic index (for example, pathological diagnosis result, clinical diagnosis result, etc.) of the classified medical image as the recognition result to the subsequent comparison unit 123.

[0051] In this way, the recognition unit 121 identifies the medical image group such as endoscopic images obtained in the input unit 11 according to the purpose of inspection based on the detection results in the detection unit 21a or the classification results in the classification unit 21b, or both the detection results and the classification results, and sends the recognition result to the comparison unit 123.

[0052] In the first embodiment, the user action analysis unit 122 includes a discovery action analysis unit 22a that analyzes actions related to the doctor's discovery of a lesion. For example, when the input unit 11 acquires an endoscopic image as a medical image group, the discovery action analysis unit 22a analyzes the actions of the "endoscope insertion unit" related to the doctor's discovery of the lesion by making a judgment based on the status of a series of medical image groups or electrical signals that can be obtained from an endoscope-related device.

[0053] The detection action analysis unit 22a in this embodiment analyzes the action of a doctor (user) approaching and observing a region of interest (lesion) when inserting the endoscope insertion unit into a body cavity of a subject (patient). Specifically, the analysis is performed by acquiring a zoom operation of the endoscope device as signal information. Alternatively, the analysis is performed by determining the presence or absence of an icon or other information displayed when zooming in on a display screen output from the endoscopic imaging device.

[0054] Another analysis involves determining whether the insertion portion of the endoscope has been withdrawn while being operated. Specifically, a series of medical image sets are analyzed, and if a singular point (a feature point such as a strong edge or edge endpoint based on pixel information) in the image is detected for a predetermined period of time or longer, the withdrawal of the insertion portion is considered to have been stopped.

[0055] Then, the motion analysis unit 22a analyzes the doctor's behavior facing the lesion based on the information related to the motion of the endoscope insertion unit, and sends the analysis result to the subsequent comparison unit 123 (see Figure 2 ).

[0056] Furthermore, in this embodiment, the discovery action analysis unit 22a analyzes the doctor's action (behavior) based on whether or not there is any endoscope-related treatment after the lesion discovery action. That is, when the doctor discovers the lesion, it determines whether or not the prescribed treatment is performed (treatment or no treatment). Thus, it analyzes whether the doctor has performed treatment (treatment) or intentionally ignored (or ignored) the lesion after discovering it, and sends the result as the analysis result to the subsequent comparison unit 123 (refer to Figure 2 ).

[0057] Here, in the analysis of whether treatment has been performed, a treatment instrument is detected in an image. If a treatment instrument is in a specific state (for example, a snare, needle, or forceps is detected), the treatment instrument is considered to be detected and "treatment has been performed."

[0058] The comparison unit 123 compares the recognition result obtained by the recognition unit 121 with the motion analysis result obtained by the user motion analysis unit 122 , and sends the comparison result to the label assignment unit 24 a in the recording unit 124 .

[0059] That is, the comparison unit 123 obtains the recognition result (for example, data of medical images that are believed to have lesions (have lesions)) recognized in the recognition unit 121 (detection unit 21a or classification unit 21b), and obtains the motion analysis result (for example, data of medical images after the doctor performed the prescribed treatment after the lesion was discovered and data of medical images without treatment) analyzed in the user motion analysis unit 122 (in this embodiment, the discovery motion analysis unit 22a), and compares the recognition result and the motion analysis result, and sends the comparison result to the label assignment unit 24a in the recording unit 124.

[0060] Here, refer to Figure 4 A modified example of the comparison unit 123 will be described.

[0061] like Figure 4 As shown, the comparison unit 123 may also be configured as a comparison unit 123A comprising a comparison information acquisition unit 23a and a comparison result generation unit 23b. In this case, the comparison information acquisition unit 23a acquires the recognition result and the motion analysis result. Furthermore, the comparison result generation unit 23b compares the recognition result with the motion analysis result to generate the comparison result, and sends the comparison result to the label assignment unit 24a in the recording unit 124.

[0062] Return to Figure 1 , currently consider the case where the comparison unit 123 compares the recognition result of the recognition unit 121 for a certain medical image in the medical image group with the analysis result in the user action analysis unit 122 corresponding to the recognition result. At this time, for example, when the recognition result obtained from the recognition unit 121 related to the medical image is "lesion", and the analysis result obtained from the discovery action analysis unit 22a in the user action analysis unit 122 is "treatment", it is considered that the doctor has accurately treated the specified lesion, and the treatment corresponding to the recognition result and the doctor's action (behavior) are considered to be "consistent", and the comparison unit 123 sends the comparison result to the subsequent recording unit 124.

[0063] In contrast, when, for example, the recognition result obtained from the recognition unit 121 related to the medical image is "lesion present", and the analysis result obtained from the discovery action analysis unit 22a in the user action analysis unit 122 is "no treatment", the comparison unit 123 believes that the doctor has not accurately treated the prescribed lesion, or believes that the lesion does not need to be treated by the doctor, and regards the treatment corresponding to the recognition result and the doctor's action (behavior) as "inconsistent", and sends the comparison result to the subsequent recording unit 124.

[0064] On the other hand, the comparison result of the comparison unit 123 is not limited to the above-mentioned two-choice result of "consistency" or "inconsistency". For example, consider the case where the recognition result of the recognition unit 121 for a certain fixed group of medical image groups among the multiple inspection image groups and the analysis result of the user action analysis unit 122 corresponding to the certain fixed group of medical image groups are compared. In this case, for example, the degree of "consistency" or "inconsistency" related to the medical image group of the group can be used as information with a certain weight (for example, a weight of inconsistency), and the comparison result is sent to the subsequent recording unit 124.

[0065] The recording unit 124 includes a memory unit such as a flash memory in which records can be updated, and in the first embodiment, includes a label assigning unit 24 a and a classification unit 24 b .

[0066] The label assigning unit 24a generates a label corresponding to the comparison result obtained in the comparison unit 123 and assigns the label to the medical image related to the comparison result. In addition, the recording unit 124 stores the medical image after being labeled by the label assigning unit 24a in the above-mentioned predetermined memory unit.

[0067] Specifically, the label assigning unit 24a generates a "consistent label" indicating consistency based on the comparison result obtained in the comparison unit 123, for example, when the results of the recognition result (for example, the presence of a lesion) identified in the recognition unit 121 and the action analysis result (for example, treatment or no treatment) analyzed in the discovery action analysis unit 22a in the user action analysis unit 122 are consistent (in this case, as described above, it is "the presence of a lesion" and "treatment"), and assigns the "consistent label" to the medical image related to the comparison result.

[0068] On the other hand, when the recognition result obtained in the recognition unit 121 and the motion analysis result obtained in the discovery motion analysis unit 22a in the user motion analysis unit 122 are inconsistent (in this case, as described above, it is "lesion" and "no treatment"), the label assignment unit 24a generates an "inconsistent label" indicating the inconsistency and assigns the "inconsistent label" to the medical image related to the comparison result.

[0069] In addition, the label assigning unit 24a is not limited to generating and assigning the above-mentioned "consistent labels" or "inconsistent labels". For example, it can also generate a "weighted label" representing the weight of the inconsistency between the recognition result obtained in the recognition unit 121 and the motion analysis result obtained in the motion analysis unit 22a based on the comparison result information from the comparison unit 123, and assign the "weighted label" to the medical images related to the comparison result respectively.

[0070] Here, as an example of weighted calculation of inconsistency, the case where the "recognition result" is no lesion and the "motion analysis result" is treated is level 1, and on the other hand, the case where the "recognition result" is a lesion and the "motion analysis result" is no treatment is level 2, and differences in importance are also given to inconsistent labels.

[0071] In the first embodiment, the recording unit 124 further includes a classification unit 24b that classifies the corresponding medical image according to a predetermined standard based on the label generated in the label assigning unit 24a as described above.

[0072] The classification unit 24b classifies the corresponding medical images according to the labels generated in the label assigning unit 24a, for example, according to the above-mentioned categories of "consistent labels" and "inconsistent labels".

[0073] Here, when the "recognition result" and the "analysis result" for a certain medical image are consistent (in this case, the recognition result obtained from the recognition unit 121 for the medical image is "lesion present", and the analysis result obtained from the user action analysis unit 122 is "treatment present"), it is considered that the doctor has accurately treated the specified lesion, and the label assignment unit 24a generates a "consistent label" and assigns a "consistent label" to the medical image.

[0074] Then, the classification unit 24b, which receives the information of the "matching label" from the labeling unit 24a, classifies the corresponding medical image into, for example, "level 1" (see Figure 2 ) and stores the medical image together with the grading information "Level 1" in the specified memory unit.

[0075] On the other hand, when the "recognition result" and the "analysis result" for a certain medical image are inconsistent (in this case, the recognition result obtained from the recognition unit 121 for the medical image is "lesion", and the analysis result obtained from the user action analysis unit 122 is "no treatment"), it is considered that the doctor did not accurately treat the prescribed lesion, or that the lesion does not need to be treated by the doctor, and the label assignment unit 24a generates an "inconsistent label" and assigns the "inconsistent label" to the medical image.

[0076] Then, the classification unit 24b, which receives the information of the "inconsistent label" from the labeling unit 24a, classifies the corresponding medical image into, for example, "level 2" (refer to Figure 2 ) and stores the medical image together with the grading information "Level 2" in the specified memory unit.

[0077] In addition, the grading unit 24b may also perform a predetermined grading on the corresponding medical image based on the "weighted label" of the above-mentioned inconsistency generated in the label assigning unit 24a.

[0078] Then, in the first embodiment, the recording unit 124 stores each medical image after classification together with the label information in a predetermined memory unit. That is, the various labels as described above are generated in the label assigning unit 24a, and each medical image classified according to a predetermined reference in the classification unit 24b based on the generated labels is stored together with the label information (for example, the classification information described above) in a predetermined memory unit (refer to Figure 2 ).

[0079] <Function of the First Embodiment>

[0080] Next, refer to Figure 2 The illustration shown in Figure 3 The operation of the image recording apparatus according to the first embodiment will be described using a flowchart. Figure 2 This is an explanatory diagram schematically illustrating the operation of the image recording device according to the first embodiment. Figure 3 This is a flowchart showing the operation of the image recording device according to the first embodiment.

[0081] like Figure 3 As shown, the image recording device of the first embodiment first obtains a prescribed medical image such as an endoscopic image in the input unit 11 (step S11). In the present embodiment, the medical image is assumed to be a doctor 100 (refer to Figure 2 ) is an image (endoscopic image) obtained by the user using a prescribed medical device (a medical endoscope device in this embodiment) in accordance with his or her own medical instructions (diagnosis and treatment guidelines, etc.).

[0082] Next, the recognition unit 121 in the image recording device recognizes the medical image such as the endoscopic image acquired in the input unit 11 in a manner consistent with the examination purpose, and sends the specified medical image as the recognition result to the comparison unit 123 (step S12).

[0083] Specifically, in the detection unit 21a of the recognition unit 121, when a predetermined region of interest (for example, a region where a predetermined lesion exists) is detected in the medical image group such as the endoscopic image acquired from the input unit 11, the presence or absence of the region of interest is sent to the comparison unit 123 as a recognition result (refer to Figure 2 ).

[0084] On the other hand, the discovery action analysis unit 22a in the user action analysis unit 122 is in charge of the doctor 100 (refer to Figure 2 ) Pathology-related actions are analyzed and the analysis results are sent to the comparison unit 123 (step S13).

[0085] The discovery motion analysis unit 22a analyzes, for example, the motion used by a doctor (user) to approach and observe an area of ​​interest (lesion) when inserting the endoscope insertion unit into a body cavity of a subject (patient). Specifically, the zoom-in operation is analyzed by acquiring signal information from the endoscope device's zoom-in operation. Alternatively, as another method, the zoom-in operation is analyzed by determining the presence or absence of information such as an icon displayed when zoomed-in observation is performed on a display screen output from the endoscopic imaging device.

[0086] Another analysis involves determining whether the insertion portion of the endoscope has been withdrawn while being operated. Specifically, a series of medical image sets are analyzed, and if a singular point (a feature point such as a strong edge or edge endpoint based on pixel information) in the image is detected for a predetermined period of time or longer, the withdrawal of the insertion portion is considered to have been stopped.

[0087] Then, the motion analysis unit 22a analyzes the doctor's behavior facing the lesion based on the information related to the motion of the endoscope insertion unit, and sends the analysis result to the subsequent comparison unit 123 (see Figure 2 ).

[0088] Furthermore, in this embodiment, the discovery action analysis unit 22a analyzes the doctor's action (behavior) based on whether or not there is any endoscope-related treatment after the lesion discovery action. That is, when the doctor discovers a lesion, it determines whether or not a prescribed treatment has been performed. Thus, it analyzes whether the doctor has performed treatment (treatment) or intentionally ignored (or ignored) the lesion after discovering it, and sends the analysis result to the subsequent comparison unit 123 (refer to Figure 2 ).

[0089] Next, the comparison unit 123 in the image recording device obtains the recognition result (for example, data of a medical image considered to have a lesion (with a lesion)) recognized by the recognition unit 121 (detection unit 21a or classification unit 21b) (refer to Figure 2 ), and obtain the action analysis results obtained in the user action analysis unit 122 (for example, the data of medical images after the doctor has performed the prescribed treatment after the lesion is discovered and the data of medical images without treatment) (refer to Figure 2 ), and compares the recognition result and the action analysis result, and sends the comparison result to the label assigning unit 24a in the recording unit 124 (step S14).

[0090] Specifically, when the recognition result related to the medical image obtained from the recognition unit 121 is "lesion present", and the analysis result obtained from the discovery action analysis unit 22a in the user action analysis unit 122 is "treatment present", the comparison unit 123 believes that the doctor has accurately treated the specified lesion, and regards the treatment corresponding to the recognition result and the doctor's action (behavior) as "consistent", and sends the comparison result to the subsequent recording unit 124.

[0091] On the other hand, when the recognition result obtained from the recognition unit 121 related to the medical image is "lesion", and the analysis result obtained from the discovery action analysis unit 22a in the user action analysis unit 122 is "no treatment", the comparison unit 123 believes that the doctor has not accurately treated the prescribed lesion, or believes that the lesion does not need to be treated by the doctor, and regards the treatment corresponding to the recognition result and the doctor's action (behavior) as "inconsistent", and sends the comparison result to the subsequent recording unit 124.

[0092] Furthermore, the comparison unit 123 may compare the recognition results of the recognition unit 121 for a certain fixed group of medical image groups among the multiple inspection image groups with the analysis results of the user action analysis unit 122 corresponding to these certain fixed group of medical image groups, and use the degree of "consistency" or "inconsistency" related to the group of medical image groups as information with a certain weight (for example, inconsistency weight), and send the comparison result to the subsequent recording unit 124.

[0093] Next, the label assigning unit 24a in the recording unit 124 generates a label corresponding to the comparison result performed by the comparing unit 123, and assigns the label to the medical image related to the comparison result (step S15).

[0094] For example, in the comparison results of the comparison unit 123, when the recognition result identified in the recognition unit 121 (for example, the presence of a lesion) and the action analysis result analyzed in the discovery action analysis unit 22a in the user action analysis unit 122 (for example, treatment or no treatment) are consistent (in this case, there is a lesion and there is treatment), the label assignment unit 24a generates a "consistent label" indicating consistency and assigns the "consistent label" to the medical image related to the comparison result.

[0095] On the other hand, when the recognition result obtained in the recognition unit 121 and the motion analysis result obtained in the discovery motion analysis unit 22a in the user motion analysis unit 122 are inconsistent, the label assignment unit 24a generates an "inconsistent label" indicating the inconsistency and assigns the "inconsistent label" to the medical image related to the comparison result.

[0096] In addition, the label assigning unit 24a is not limited to generating and assigning the above-mentioned "consistent labels" or "inconsistent labels". For example, it can also generate a "weighted label" representing the weight of the inconsistency between the recognition result obtained in the recognition unit 121 and the motion analysis result obtained in the motion analysis unit 22a based on the comparison result obtained in the comparison unit 123, and assign the "weighted label" to the medical images related to the comparison result respectively.

[0097] Next, the grading unit 24b gradates the corresponding medical image according to the label generated in the label assigning unit 24a, for example, according to the above-mentioned categories of "consistent label" and "inconsistent label" (for example, as mentioned above, the medical image assigned with "consistent label" is graded as "level 1", and the medical image assigned with "inconsistent label" is graded as "level 2"), and stores the medical image together with the grading information in a specified memory unit (refer to Figure 2 ).

[0098] <Effects of the First Embodiment>

[0099] As described above, the image recording device according to the first embodiment has the effect of being able to record an object that a recognizer is not good at in a data format that can be easily extracted.

[0100] <Second embodiment>

[0101] Next, a second embodiment of the present invention will be described.

[0102] The image recording device of the second embodiment is characterized in that the analysis target of the doctor's movements in the user movement analysis unit is different from that of the first embodiment. Since the other structures are the same as those of the first embodiment, only the differences from the first embodiment will be described here, and the description of the common parts will be omitted.

[0103] Figure 5 is a block diagram showing the configuration of a medical system including an image recording apparatus according to a fourth embodiment of the present invention. Figure 6 This is a flowchart showing the operation of the image recording device according to the second embodiment.

[0104] like Figure 5 As shown, the medical system 1 including the image recording device 210 of the second embodiment is the same as the above-mentioned first embodiment, and mainly comprises the image recording device 210 for acquiring medical images and performing prescribed image processing, a storage unit 31 connected to the image recording device 210 and storing prescribed data, and a display unit 32 for displaying the medical images after image processing is performed in the image recording device 210.

[0105] The image recording device 210 in the second embodiment is also the same as the first embodiment, and mainly includes an input unit 11 for acquiring medical images captured by the camera unit in the endoscope, a control unit 12 for controlling the overall operation of the image recording device 210, and a calculation unit 215 for performing various processing described later on the medical images acquired in the input unit 11.

[0106] In the second embodiment, the calculation unit 215 is configured to include a recognition unit 221 , a user action analysis unit 222 , a comparison unit 223 , a recording unit 224 , and the like, which will be described in detail later.

[0107] In the second embodiment, each component of the image recording device 210, such as the computing unit 215 and the control unit 12, may be configured as an electronic circuit or as a circuit block in an integrated circuit such as an FPGA (Field Programmable Gate Array). Furthermore, for example, the image recording device 210 may include one or more processors (CPUs, etc.).

[0108] <Calculation Unit 215 in Second Embodiment>

[0109] Next, the detailed structure of the calculation unit 215 in the second embodiment will be described.

[0110] The operation unit 215 includes: an identification unit 221, which identifies medical images such as endoscopic images obtained in the input unit 11 and obtains the identification result; a user action analysis unit 222, which obtains the action analysis result by analyzing the action related to the doctor's (user's) interpretation of the medical image; a comparison unit 223, which compares the identification result obtained in the identification unit 221 and the action analysis result obtained in the user action analysis unit 222 to obtain the comparison result; and a recording unit 224, which stores the medical image and information about the comparison result obtained in the comparison unit 223.

[0111] The recognition unit 221 has the same structure and function as the recognition unit 121 in the first embodiment, so the description is omitted here. The recognition unit 221 recognizes medical images such as endoscopic images obtained in the input unit 11 in a manner consistent with the inspection purpose based on the detection results in the detection unit 21a or the classification results in the classification unit 21b, and sends the specified medical image as the recognition result to the comparison unit 223.

[0112] Here, in the above-mentioned first embodiment, there is a discovery action analysis unit 22a, which analyzes the actions related to the doctor's discovery of lesions. In contrast, the user action analysis unit 222 in this second embodiment has a diagnosis action analysis unit 22b, which analyzes the actions related to the lesion diagnosis performed by the doctor as the user.

[0113] After the doctor recognizes the recognition result of the recognition unit 221, the diagnostic action analysis unit 22b analyzes the action related to the doctor's interpretation of the medical image and obtains the action analysis result.

[0114] Specifically, after the doctor recognizes the recognition result of the recognition unit 221, the diagnosis operation analysis unit 22b sends information related to the diagnosis time required for the doctor to diagnose the lesion as the analysis result to the subsequent comparison unit 223 (see Figure 2 ) In the comparison unit 223, the length of the diagnosis time is compared with a predetermined reference value, for example, to determine whether the doctor is confused about the diagnosis, and the determination result is sent to the subsequent stage.

[0115] The diagnostic action analysis unit 22b sends the information related to the diagnosis input by the doctor as an analysis result to the subsequent comparison unit 223 (see Figure 2 ). In the comparison unit 223, the recognition result and the analysis result of the diagnosis are compared to determine whether the object is difficult for the doctor to diagnose, and the determination result is sent to the subsequent stage.

[0116] Furthermore, the diagnostic action analysis unit 22b receives the doctor's input of the diagnosis using an index different from the identification index of the identification unit, and sends information related to the diagnosis as an analysis result to the subsequent comparison unit 223 (see Figure 2 ). In the comparison unit 223, the recognition result is compared with the diagnostic information input with an index different from the recognition result to determine whether it is an object that is difficult for the doctor to diagnose, and the determination result is sent to the subsequent stage.

[0117] Here, the term "diagnosis based on different indicators" refers to diagnosis performed at each stage of diagnosis in clinical diagnosis, such as non-amplification diagnosis, amplification diagnosis, staining diagnosis, and the like.

[0118] Furthermore, the diagnostic action analysis unit 22b analyzes the information on the diagnostic result that matches the index identified by the identification unit with respect to the pathological information input after the examination, and sends the analysis result to the comparison unit 223 (see Figure 2 ) In the comparison unit 223, the recognition result of the recognition unit is compared with the diagnosis result of the pathology to determine whether it is an object that is difficult for the doctor to diagnose, and the determination result is sent to the subsequent stage.

[0119] Furthermore, the diagnostic action analysis unit 22b analyzes the doctor's treatment actions related to the doctor's diagnosis of the lesion, and sends the analysis result as to whether the prescribed treatment was performed (treatment or no treatment) to the subsequent comparison unit 223 (refer to Figure 2 ).

[0120] In the second embodiment, the structures and functions of the comparison unit 223 and the recording unit 224 are the same as those of the comparison unit 123 and the recording unit 124 in the first embodiment, respectively, and therefore detailed descriptions thereof are omitted here.

[0121] <Operation of the Second Embodiment>

[0122] Next, refer to Figure 2 The illustration shown in Figure 6 The operation of the image recording apparatus according to the second embodiment will be described using a flowchart. Figure 6 This is a flowchart showing the operation of the image recording device according to the second embodiment.

[0123] like Figure 6 As shown, the image recording device of the second embodiment is the same as the first embodiment. First, a prescribed medical image such as an endoscopic image is acquired in the input unit 11 (step S21). In this embodiment, the medical image is also assumed to be a doctor 100 as a user (refer to Figure 2) is an image (endoscopic image) obtained by the user using a prescribed medical device (a medical endoscope device in this embodiment) in accordance with his or her own medical instructions (diagnosis and treatment guidelines, etc.).

[0124] Next, the recognition unit 221 in the image recording device recognizes the medical image such as the endoscopic image acquired in the input unit 11 in a manner consistent with the examination purpose, and sends the specified medical image as the recognition result to the comparison unit 223 (step S22).

[0125] On the other hand, the diagnostic action analysis unit 22b of the user action analysis unit 222 analyzes the doctor 100 (see Figure 2 ) analyzes the action after interpreting the medical image and sends the analysis result to the comparison unit 223 (step S23).

[0126] Next, the comparison unit 223 in the image recording device obtains the recognition result (for example, data of a medical image considered to have a lesion (lesion)) recognized by the recognition unit 221 (refer to Figure 2 ), and obtain the action analysis results obtained in the user action analysis unit 222 (for example, the data of the medical image after the doctor performs the prescribed treatment after the interpretation of the medical image and the data of the medical image without treatment) (refer to Figure 2 ), and compares the recognition result with the action analysis result, and sends the comparison result to the label assigning unit 24a in the recording unit 224 (step S24).

[0127] Specifically, when the recognition result related to the medical image obtained from the recognition unit 221 is "lesion present", and the analysis result obtained from the diagnostic action analysis unit 22b in the user action analysis unit 222 is "treatment present", the comparison unit 223 believes that the doctor has accurately treated the specified lesion, and regards the treatment corresponding to the recognition result and the doctor's action (behavior) as "consistent", and sends the comparison result to the subsequent recording unit 224.

[0128] On the other hand, when the recognition result related to the medical image obtained from the recognition unit 221 is "lesion", and the analysis result obtained from the diagnostic action analysis unit 22b in the user action analysis unit 222 is "no treatment", the comparison unit 223 believes that the doctor did not accurately treat the specified lesion, and regards the treatment corresponding to the recognition result and the doctor's action (behavior) as "inconsistent", and sends the comparison result to the subsequent recording unit 224.

[0129] Next, the label assigning unit 24a in the recording unit 224 generates a label corresponding to the comparison result performed in the comparing unit 223, and assigns the label to the medical image related to the comparison result (step S25).

[0130] The functions of the label applying unit 24 a and the classification unit 24 b in the recording unit 224 are the same as those in the first embodiment, and therefore detailed description thereof will be omitted here.

[0131] <Effects of the Second Embodiment>

[0132] As described above, the image recording device according to the second embodiment also has the effect of being able to record an object that a recognizer is not good at in a data format that can be easily extracted, similar to the first embodiment.

[0133] <Third embodiment>

[0134] Next, a third embodiment of the present invention will be described.

[0135] In the image recording apparatus of the first and second embodiments, the recording unit 124 (224) is characterized in that it has a label assigning unit 24a and a grading unit 24b, and grades and records the medical image based on the label information generated in the label assigning unit 24a.

[0136] In contrast, the image recording device of the third embodiment is characterized in that the recording unit changes the image quality of the medical image based on the aforementioned tag information, adjusts the image data volume, and records the image. Since the rest of the structure is the same as the second embodiment, only the differences from the first and second embodiments will be described here, and the description of the common parts will be omitted.

[0137] Figure 7 is a block diagram showing the configuration of a medical system including an image recording apparatus according to a third embodiment of the present invention. Figure 8 This is an explanatory diagram for schematically explaining the operation of the image recording device according to the third embodiment. Figure 9 This is a flowchart showing the operation of the image recording device according to the third embodiment.

[0138] like Figure 7 As shown, the medical system 1 including the image recording device 310 of the third embodiment is the same as the above-mentioned first embodiment, and mainly comprises the image recording device 310 for acquiring medical images and performing prescribed image processing, a storage unit 31 connected to the image recording device 310 and storing prescribed data, and a display unit 32 for displaying the medical images after image processing is performed in the image recording device 310.

[0139] The image recording device 310 in the third embodiment is also the same as the first and second embodiments, and mainly includes an input unit 11 for acquiring medical images captured by the camera unit in the endoscope, a control unit 12 for controlling the overall operation of the image recording device 310, and a calculation unit 315 for performing various processing described later on the medical images acquired in the input unit 11.

[0140] In the third embodiment, the calculation unit 315 is configured to include a recognition unit 321 , a user action analysis unit 322 , a comparison unit 323 , a recording unit 324 , and the like, and the details will be described later.

[0141] Furthermore, in the third embodiment, each component of the image recording device 310, such as the computing unit 315 and the control unit 12, may be configured as an electronic circuit or as a circuit block within an integrated circuit such as an FPGA (Field Programmable Gate Array). Furthermore, for example, the image recording device 310 may be configured to include one or more processors (CPUs, etc.).

[0142] <Calculation Unit 315 in Third Embodiment>

[0143] Next, the detailed structure of the calculation unit 315 in the third embodiment will be described.

[0144] The operation unit 315 includes: an identification unit 321, which identifies medical images such as endoscopic images obtained in the input unit 11 and obtains the identification result; a user action analysis unit 322, which obtains the action analysis result by analyzing the action related to the doctor's (user's) interpretation of the medical image; a comparison unit 323, which compares the identification result obtained in the identification unit 321 and the action analysis result obtained in the user action analysis unit 322 to obtain the comparison result; and a recording unit 324, which stores the medical image and information about the comparison result obtained in the comparison unit 323.

[0145] The recognition unit 321 has the same structure and function as the recognition unit 221 in the second embodiment, so the description is omitted here. The recognition unit 321 recognizes medical images such as endoscopic images obtained in the input unit 11 in a manner consistent with the inspection purpose based on the detection results in the detection unit 21a or the classification results in the classification unit 21b, and sends the specified medical image as the recognition result to the comparison unit 323.

[0146] The user action analysis unit 322 is the same as in the second embodiment and includes a diagnostic action analysis unit 22b that analyzes actions related to lesion diagnosis performed by the user, the doctor. The structure and effects of the diagnostic action analysis unit 22b are the same as in the second embodiment, so their description is omitted here.

[0147] In the third embodiment, the configuration and operational effects of the comparison unit 323 are the same as those of the comparison unit 223 in the second embodiment, and therefore detailed descriptions thereof are omitted here.

[0148] In the third embodiment, the recording unit 324 includes a memory unit such as a flash memory capable of updating data, similar to the first and second embodiments. However, in addition to the label assigning unit 24a and the classification unit 24b, it also includes an image quality changing unit 24c. Furthermore, the label assigning unit 24a, the classification unit 24b, and the image quality changing unit 24c are all implemented as circuit blocks within an integrated circuit such as an FPGA (Field Programmable Gate Array).

[0149] In this third embodiment, the label assigning unit 24a, similar to the first and second embodiments, generates a label corresponding to the comparison result obtained by the comparison unit 323 and assigns the label to the medical image associated with the comparison result. Furthermore, the grading unit 24b grades the corresponding medical image according to a predetermined benchmark based on the label generated by the label assigning unit 24a. Furthermore, the recording unit 324 stores the medical image, after being labeled by the label assigning unit 24a, in the aforementioned predetermined memory unit.

[0150] That is, in this third embodiment, the various labels as described above are also generated in the label assigning unit 24a, and each medical image that has been graded according to a prescribed benchmark in the grading unit 24b based on the generated labels is stored in a prescribed memory unit together with the label information (for example, the above-mentioned grading information).

[0151] In the third embodiment, the image quality changing unit 24c changes the image quality associated with the medical image based on the label generated by the label assigning unit 24a. Specifically, the image quality changing unit 24c adjusts the image quality when the label generated by the label assigning unit 24a is, for example, the aforementioned "consistent label," or adjusts the image quality based on an "inconsistent label."

[0152] For example, when the recognition result obtained from the recognition unit 321 related to a certain medical image is "lesion", and the analysis result obtained from the user action analysis unit 322 is "treatment", it is considered that the doctor has performed accurate treatment, and the treatment corresponding to the recognition result and the doctor's action (behavior) are considered to be "consistent". The label assignment unit 24a accepts the result, generates the above-mentioned "consistent label", and assigns the "consistent label" to the corresponding medical image.

[0153] At this time, the image quality changing unit 24c changes the image quality of the medical image assigned the "matching label," specifically, to a low image quality. The recording unit 324 then stores the medical image, whose image quality has been changed to a low quality by the image quality changing unit 24c and whose recording capacity and transmission volume have been reduced, in a predetermined memory unit.

[0154] On the other hand, when the recognition result obtained from the recognition unit 321 related to the medical image is "lesion", and the analysis result obtained from the user action analysis unit 322 is "no treatment", it is considered that the doctor did not perform accurate treatment, and the treatment corresponding to the recognition result and the doctor's action (behavior) are considered to be "inconsistent". The label assignment unit 24a accepts the result, generates the above-mentioned "inconsistent label", and assigns the "inconsistent label" to the corresponding medical image.

[0155] In this case, the image quality changing unit 24c does not change the image quality of the medical image assigned the "inconsistency label", specifically, maintains the high image quality. Then, the recording unit 324 stores the medical image with maintained high image quality in a predetermined memory unit.

[0156] In addition, in the above example, based on the "consistent label" and "inconsistent label" generated by the label assigning unit 24a, especially when the generated label is a "consistent label", the image quality of the original high-quality medical image is changed to a low-quality image, which can achieve the suppression of the recording capacity and transmission volume related to the medical image, but the conditions for changing the image quality are not limited to this.

[0157] For example, the image quality changing unit 24c may have a function of setting a RAW image when certain conditions are met, and only medical images corresponding to specified labels (such as consistent labels) may be set as RAW images and recorded.

[0158] That is, for example, in cases where a doctor does not perform appropriate treatment on a medical image that deserves attention (an image showing a characteristic lesion), or where recording a medical image as a delicate image is advantageous, sometimes recording with high image quality is required, even acknowledging the disadvantage of increased data capacity. In such cases, it is possible to configure the system to record the medical image as a high-quality RAW image, while significantly reducing the image quality of other medical images to reduce recording capacity, thereby reducing overall recording capacity.

[0159] Furthermore, in addition to adjusting the image quality, the recording capacity can also be reduced by adjusting the screen size based on the tag information.

[0160] Furthermore, it may be configured to appropriately perform image processing on the corresponding medical image according to pre-set parameters based on the tag information, thereby reducing the recording capacity.

[0161] Thus, in the third embodiment, based on the comparison information between the treatment corresponding to the recognition result and the analysis result of the treatment after the doctor's interpretation, the label assignment unit 24a generates label information of the above-mentioned category and assigns the label to the medical image. On the other hand, the image quality changing unit 24c changes the image quality of the medical image that does not necessarily require high image quality to a low image quality based on the label information, thereby achieving the reduction of recording capacity and transmission amount (refer to Figure 8 ).

[0162] In addition, in this third embodiment, the image quality changing unit 24c changes the image quality of the corresponding medical image according to the label information generated in the label assigning unit 24a, but is not limited to this. For example, the image quality of the corresponding medical image can also be changed according to the level of the medical image after being graded in the grading unit 24b.

[0163] <Function of the Third Embodiment>

[0164] Next, refer to Figure 8 The illustration shown in Figure 9 The operation of the image recording apparatus according to the third embodiment will be described using a flowchart. Figure 9 This is a flowchart showing the operation of the image recording device according to the third embodiment.

[0165] like Figure 9 As shown, the image recording device of the third embodiment is the same as the first and second embodiments, and first obtains a prescribed medical image such as an endoscopic image in the input unit 11 (step S31). In this embodiment, the medical image is also assumed to be a doctor 100 as a user (refer to Figure 8) is an image (endoscopic image) obtained by the user using a prescribed medical device (a medical endoscope device in this embodiment) in accordance with his or her own medical instructions (diagnosis and treatment guidelines, etc.).

[0166] Next, the recognition unit 321 in the image recording device recognizes the medical image such as the endoscopic image acquired in the input unit 11 in a manner consistent with the examination purpose, and sends the specified medical image as the recognition result to the comparison unit 323 (step S32).

[0167] On the other hand, the diagnostic motion analysis unit 22b in the user motion analysis unit 322 analyzes the doctor 100 (see Figure 8 ) analyzes the action after interpreting the medical image and sends the analysis result to the comparison unit 323 (step S33).

[0168] Next, the comparison unit 323 in the image recording device obtains the recognition result recognized by the recognition unit 321 (see Figure 8 ), and obtain the action analysis results obtained in the user action analysis unit 322 (for example, the data of the medical image after the doctor has performed the prescribed treatment after the interpretation of the medical image and the data of the medical image without the treatment) (refer to Figure 8 ), and compares the recognition result with the action analysis result, and sends the comparison result to the label assignment unit 24a in the recording unit 324 (step S34).

[0169] Then, based on the comparison information between the treatment corresponding to the recognition result and the analysis result of the treatment after the doctor's interpretation, label information of the category as described above is generated in the label assignment unit 24a, and the label is assigned to the medical image. On the other hand, the image quality change unit 24c changes the image quality of the medical image that does not necessarily require high image quality to a low image quality based on the label information, and stores it in the specified memory unit (step S35).

[0170] <Effects of the Third Embodiment>

[0171] As described above, according to the image recording device of the third embodiment, a rich amount of information can be secured for images related to objects that the recognizer is not good at, while reducing the storage capacity and transmission amount of the recording unit.

[0172] <Fourth embodiment>

[0173] Next, a fourth embodiment of the present invention will be described.

[0174] In the image recording apparatus of the first to third embodiments, in the recording unit, classification is performed based on the label information generated in the label assigning unit 24a, or image quality is changed to adjust the recording capacity and transmission volume, but in principle all medical images are recorded.

[0175] In contrast, the image recording device of the present fourth embodiment is characterized in that, in the recording unit, it is set to ensure a rich amount of information for images related to objects that the identifier is not good at based on the above-mentioned label information, and on the other hand, based on the comparison information between the treatment corresponding to the recognition result and the analysis result of the treatment after the doctor's interpretation, it is set not to save medical images that may not need to be recorded, thereby further reducing the recording capacity and transmission volume.

[0176] Since the other structures are the same as those of the first and second embodiments, only the differences from the first and second embodiments will be described here, and the description of the common parts will be omitted.

[0177] Figure 10 is a block diagram showing the configuration of a medical system including an image recording apparatus according to a fourth embodiment of the present invention. Figure 11 1 is an explanatory diagram schematically illustrating the operation of the image recording device according to the fourth embodiment. Figure 12 This is a flowchart showing the operation of the image recording device according to the fourth embodiment.

[0178] like Figure 10 As shown, the medical system 1 including the image recording device 410 of the present fourth embodiment is the same as the above-mentioned first embodiment, and mainly comprises the image recording device 410 for acquiring medical images and performing prescribed image processing, a storage unit 31 connected to the image recording device 410 and storing prescribed data, and a display unit 32 for displaying the medical images after image processing is performed in the image recording device 410.

[0179] The image recording device 410 in the fourth embodiment is also the same as the first and second embodiments, and mainly includes an input unit 11 for acquiring medical images captured by the camera unit in the endoscope, a control unit 12 for controlling the overall operation of the image recording device 410, and a calculation unit 415 for performing various processing described later on the medical images acquired in the input unit 11.

[0180] In the fourth embodiment, the calculation unit 415 is configured to include a recognition unit 421 , a user action analysis unit 422 , a comparison unit 423 , a recording unit 424 , and the like, which will be described in detail later.

[0181] <Calculation Unit 415 in Fourth Embodiment>

[0182] Next, the detailed structure of the calculation unit 415 in the fourth embodiment will be described.

[0183] The operation unit 415 includes: an identification unit 421, which identifies medical images such as endoscopic images obtained in the input unit 11 and obtains the identification result; a user action analysis unit 422, which obtains the action analysis result by analyzing the action related to the doctor's (user's) interpretation of the medical image; a comparison unit 423, which compares the identification result obtained in the identification unit 421 and the action analysis result obtained in the user action analysis unit 422 to obtain the comparison result; and a recording unit 424, which stores the medical image and information about the comparison result obtained in the comparison unit 423.

[0184] The recognition unit 421 has the same structure and function as the recognition unit 121 in the first embodiment, so the description is omitted here, but the recognition unit 421 recognizes medical images such as endoscopic images obtained in the input unit 11 in a manner consistent with the inspection purpose based on the detection results in the detection unit 21a or the classification results in the classification unit 21b, and sends the specified medical image as the recognition result to the comparison unit 423.

[0185] The user motion analysis unit 422 is the same as the second embodiment and includes a diagnostic motion analysis unit 22b that analyzes motions related to lesion diagnosis performed by the user's doctor. The structure and effects of the diagnostic motion analysis unit 22b are the same as those of the second embodiment, so their description is omitted here.

[0186] In addition, in this fourth embodiment, the comparison unit 423 is the same as the comparison unit 223 in the second embodiment, and compares the recognition result obtained from the recognition unit 421 and the analysis result obtained from the user action analysis unit 422, and sends the information on whether the treatment corresponding to the recognition result and the interpreted doctor's action (behavior) are "consistent" or "inconsistent" as the comparison result to the subsequent recording unit 424.

[0187] In the fourth embodiment, the recording unit 424 includes a memory unit such as a flash memory that can update records, similar to the first and second embodiments. However, in addition to the label assignment unit 24a, it also includes an inconsistency processing unit 24d. Furthermore, both the label assignment unit 24a and the inconsistency processing unit 24d are implemented as circuit blocks in an integrated circuit such as an FPGA (Field Programmable Gate Array).

[0188] In this fourth embodiment, the label assigning unit 24a also generates a label corresponding to the comparison result output from the comparison unit 423, similar to the first and second embodiments, and assigns the label to the medical image related to the comparison result.

[0189] For example, if the treatment corresponding to the recognition result and the interpreted doctor's action (behavior) are "consistent", the label assigning unit 24a generates a "consistent label" and assigns the "consistent label" to the corresponding medical image. On the other hand, if they are "inconsistent", an "inconsistent label" is generated and assigned to the corresponding medical image.

[0190] In the fourth embodiment, the inconsistency processing unit 24d selects and processes the medical images stored in the memory unit based on the label information generated in the label assigning unit 24a. Specifically, the inconsistency processing unit 24d performs processing in the following manner: the medical images assigned the "consistent label" are not saved, and only the medical images assigned the "inconsistent label" are stored in the memory unit for storage.

[0191] Thus, in the fourth embodiment, based on the comparison information between the treatment corresponding to the recognition result and the analysis result of the treatment after the doctor's interpretation, the label information of the above-mentioned category is generated in the label assigning unit 24a, and the label is assigned to the medical image, and only the medical image assigned with the "inconsistent label" is processed and saved in the memory unit in the inconsistency processing unit 24d (refer to Figure 11 ).

[0192] <Function of the Fourth Embodiment>

[0193] Next, refer to Figure 11 The illustration shown in Figure 12 The operation of the image recording apparatus according to the fourth embodiment will be described using a flowchart. Figure 12 This is a flowchart showing the operation of the image recording device according to the fourth embodiment.

[0194] like Figure 12 As shown, the image recording device of the fourth embodiment is the same as the first and second embodiments, and first obtains a prescribed medical image such as an endoscopic image in the input unit 11 (step S41). In this embodiment, the medical image is also assumed to be a doctor 100 as a user (refer to Figure 11 ) The image (endoscopic image) obtained by the medical device itself using a prescribed medical device (in this embodiment, a medical endoscope device) in accordance with its own medical instructions (diagnosis and treatment guidelines, etc.).

[0195] Next, the recognition unit 421 in the image recording device recognizes the medical image such as the endoscopic image acquired in the input unit 11 in a manner consistent with the examination purpose, and sends the specified medical image as the recognition result to the comparison unit 423 (step S42).

[0196] On the other hand, the diagnostic action analysis unit 22b in the user action analysis unit 422 analyzes the doctor 100 (see Figure 11 ) analyzes the action after interpreting the medical image and sends the analysis result to the comparison unit 423 (step S43).

[0197] Next, the comparison unit 423 in the image recording device obtains the recognition result recognized by the recognition unit 421 (see Figure 11 ), and obtain the action analysis results obtained in the user action analysis unit 422 (for example, the data of the medical image after the doctor has performed the prescribed treatment after interpreting the medical image and the data of the medical image without the treatment) (refer to Figure 11 ), and compares the recognition result with the action analysis result, and sends the comparison result to the label assigning unit 24a in the recording unit 424 (step S44).

[0198] Then, based on the comparison information between the treatment corresponding to the recognition result and the analysis result of the treatment after the doctor's interpretation, label information of the category as described above is generated in the label assignment unit 24a, and the label is assigned to the medical image. On the other hand, the inconsistency processing unit 24d does not save the medical images that do not need to be recorded (images assigned with consistent labels) to the memory unit based on the label information, but only saves the medical images assigned with "inconsistent labels" to the memory unit (step S45).

[0199] <Effects of the Fourth Embodiment>

[0200] As described above, according to the image recording device of the fourth embodiment, a rich amount of information can be secured for images related to objects that the recognizer is not good at, while suppressing the storage capacity and transmission amount of the recording unit.

[0201] <Fifth embodiment>

[0202] Next, a fifth embodiment of the present invention will be described.

[0203] The image recording device of the fifth embodiment is characterized in that the treatment corresponding to the recognition result and the analysis results of the treatment after the doctor's interpretation are listed in the recording unit and stored together with the corresponding medical image. Since the other structures are the same as the first and second embodiments, only the differences from the first and second embodiments are described here, and the description of the common parts is omitted.

[0204] Figure 13 is a block diagram showing the configuration of a medical system including an image recording apparatus according to a fifth embodiment of the present invention. Figure 14 1 is an explanatory diagram schematically illustrating the operation of the image recording device according to the fifth embodiment. Figure 15This is a flowchart showing the operation of the image recording device according to the fifth embodiment.

[0205] like Figure 13 As shown, the medical system 1 including the image recording device 510 of the fifth embodiment is the same as the first embodiment described above, and mainly comprises the image recording device 510 for acquiring medical images and performing prescribed image processing, a storage unit 31 connected to the image recording device 510 and storing prescribed data, and a display unit 32 for displaying the medical images after image processing is performed in the image recording device 510.

[0206] The image recording device 510 in the fifth embodiment is also the same as the first and second embodiments, and mainly includes an input unit 11 for acquiring medical images captured by the camera unit in the endoscope, a control unit 12 for controlling the overall operation of the image recording device 510, and a calculation unit 515 for performing various processing described later on the medical images acquired in the input unit 11.

[0207] In the fifth embodiment, the calculation unit 515 is configured to include a recognition unit 521 , a user action analysis unit 522 , a comparison unit 523 , a recording unit 524 , and the like, which will be described in detail later.

[0208] <Calculation Unit 515 in Fifth Embodiment>

[0209] Next, the detailed structure of the calculation unit 515 in the fifth embodiment will be described.

[0210] The operation unit 515 includes: an identification unit 521, which identifies medical images such as endoscopic images obtained in the input unit 11 and obtains the identification result; a user action analysis unit 522, which obtains the action analysis result by analyzing the action related to the doctor's (user's) interpretation of the medical image; a comparison unit 523, which compares the identification result obtained in the identification unit 521 and the action analysis result obtained in the user action analysis unit 522 to obtain the comparison result; and a recording unit 524, which stores the medical image and information about the comparison result obtained in the comparison unit 523.

[0211] The recognition unit 521 has the same structure and function as the recognition unit 121 in the first embodiment, so the description is omitted here, but the recognition unit 521 recognizes medical images such as endoscopic images obtained in the input unit 11 in a manner consistent with the inspection purpose based on the detection result in the detection unit 21a or the classification result in the classification unit 21b, and sends the specified medical image as the recognition result to the comparison unit 523.

[0212] The user action analysis unit 522 is the same as in the second embodiment and includes a diagnostic action analysis unit 22b that analyzes actions related to lesion diagnosis performed by the user, the doctor. The structure and effects of the diagnostic action analysis unit 22b are the same as in the second embodiment, so their description is omitted here.

[0213] In addition, in this fifth embodiment, the comparison unit 523 is the same as the comparison unit 223 in the second embodiment, and compares the recognition result obtained from the recognition unit 521 and the analysis result obtained from the user action analysis unit 522, and sends the information of whether the treatment corresponding to the recognition result and the interpreted doctor's action (behavior) are "consistent" or "inconsistent" as the comparison result to the subsequent recording unit 524.

[0214] In the fifth embodiment, the recording unit 524 has a storage unit such as a flash memory capable of updating records, similar to the first and second embodiments, but further includes a list generating unit 24e in addition to the label assigning unit 24a.

[0215] In the fifth embodiment, the label assigning unit 24a also generates a label corresponding to the comparison result output from the comparison unit 523, similar to the first and second embodiments, and assigns the label to the medical image related to the comparison result.

[0216] For example, if the treatment corresponding to the recognition result and the interpreted doctor's action (behavior) are "consistent", the label assigning unit 24a generates a "consistent label" and assigns the "consistent label" to the corresponding medical image. On the other hand, if they are "inconsistent", an "inconsistent label" is generated and assigned to the corresponding medical image.

[0217] In the fifth embodiment, the list creation unit 24e creates a list associated with the analysis result based on the recognition result obtained from the recognition unit 521 and the analysis result obtained from the user action analysis unit 522, and saves it together with the medical image.

[0218] Thus, in the fifth embodiment, based on the comparison information between the treatment corresponding to the recognition result and the analysis result of the treatment after the doctor's interpretation, the label assigning unit 24a generates predetermined label information, assigns the label to the medical image, and creates and saves a list associated with the analysis result in the list generating unit 24e (refer to Figure 14 ).

[0219] <Function of the Fifth Embodiment>

[0220] Next, refer to Figure 14 The illustration shown in Figure 15The operation of the image recording apparatus according to the fifth embodiment will be described using a flowchart. Figure 15 This is a flowchart showing the operation of the image recording device according to the fifth embodiment.

[0221] like Figure 15 As shown, the image recording device of the fifth embodiment is the same as the first and second embodiments, and first obtains a prescribed medical image such as an endoscopic image in the input unit 11 (step S51). In this embodiment, the medical image is also assumed to be a doctor 100 as a user (refer to Figure 14 ) is an image (endoscopic image) obtained by the user using a prescribed medical device (a medical endoscope device in this embodiment) in accordance with his or her own medical instructions (diagnosis and treatment guidelines, etc.).

[0222] Next, the recognition unit 521 in the image recording device recognizes the medical image such as the endoscopic image acquired in the input unit 11 in a manner consistent with the examination purpose, and sends the specified medical image as the recognition result to the comparison unit 523 (step S52).

[0223] On the other hand, the diagnostic action analysis unit 22b in the user action analysis unit 522 analyzes the doctor 100 (see Figure 14 ) analyzes the action after interpreting the medical image and sends the analysis result to the comparison unit 523 (step S53).

[0224] Next, the comparison unit 523 in the image recording device obtains the recognition result recognized by the recognition unit 521 (see Figure 14 ), and obtain the action analysis results obtained in the user action analysis unit 522 (for example, the data of the medical image after the doctor has performed the prescribed treatment after interpreting the medical image and the data of the medical image without the treatment) (refer to Figure 14 ), and compares the recognition result with the action analysis result, and sends the comparison result to the label assigning unit 24a in the recording unit 524 (step S54).

[0225] Then, based on the comparison information between the treatment corresponding to the recognition result and the analysis result of the treatment after interpretation by the doctor, the specified label information is generated in the label assignment unit 24a, and the label is assigned to the medical image. On the other hand, a list associated with the analysis result is generated and saved in the list creation unit 24e (step S55).

[0226] <Effects of the Fifth Embodiment>

[0227] As described above, according to the image recording device of the fifth embodiment, it is possible to record an object that a recognizer is not good at in a data format that can be easily extracted.

[0228] <Sixth embodiment>

[0229] Next, a sixth embodiment of the present invention will be described.

[0230] The image recording device of the sixth embodiment is characterized in that it further includes an additional learning unit having a learning object setting unit that sets medical images to be additionally learned based on the comparison results of the comparison unit, and the additional learning unit performs additional learning of the learning network in the recognition unit only for medical images that have been assigned learning object labels. Since the other structures are the same as those of the first embodiment, only the differences from the first embodiment will be described here, and the description of the common parts will be omitted.

[0231] Figure 16 is a block diagram showing the configuration of a medical system including an image recording apparatus according to a sixth embodiment of the present invention. Figure 17 This is a flowchart showing the operation of the image recording device according to the sixth embodiment.

[0232] like Figure 16 As shown, the medical system 1 including the image recording device 610 of the sixth embodiment is the same as the first embodiment described above, and mainly comprises the image recording device 610 for acquiring medical images and performing prescribed image processing, a storage unit 31 connected to the image recording device 610 and storing prescribed data, and a display unit 32 for displaying the medical images after image processing is performed in the image recording device 610.

[0233] The image recording device 610 in the sixth embodiment is also the same as the first embodiment, and mainly includes an input unit 11 for acquiring medical images captured by the camera unit in the endoscope, a control unit 12 for controlling the overall operation of the image recording device 610, and a calculation unit 615 for performing various processing described later on the medical images acquired in the input unit 11.

[0234] In the sixth embodiment, the calculation unit 615 is configured to include a recognition unit 621 , a user action analysis unit 622 , a comparison unit 623 , a recording unit 624 , and the like, which will be described in detail later.

[0235] <Calculation Unit 615 in Sixth Embodiment>

[0236] Next, the detailed structure of the calculation unit 615 in the sixth embodiment will be described.

[0237] The operation unit 615 includes: an identification unit 621, which identifies medical images such as endoscopic images obtained in the input unit 11 and obtains the identification result; a user action analysis unit 622, which obtains the action analysis result by analyzing the action related to the doctor's (user's) interpretation of the medical image; a comparison unit 623, which compares the identification result obtained in the identification unit 621 and the action analysis result obtained in the user action analysis unit 622 to obtain the comparison result; and a recording unit 624, which stores the medical image and information about the comparison result obtained in the comparison unit 623.

[0238] The recognition unit 621 has the same structure and operational effects as the recognition unit 121 in the first embodiment, and therefore description thereof will be omitted here.

[0239] The user action analysis unit 622 is the same as in the first embodiment and includes a discovery action analysis unit 22a that analyzes actions related to lesion diagnosis performed by the user, the doctor. The structure and effects of the discovery action analysis unit 22a are the same as in the first embodiment, so their description is omitted here.

[0240] In addition, in this sixth embodiment, the comparison unit 623 is the same as the comparison unit 123 in the first embodiment, and compares the recognition result obtained from the recognition unit 621 and the analysis result obtained from the user action analysis unit 622, and sends the information on whether the treatment corresponding to the recognition result and the doctor's action (behavior) are "consistent" or "inconsistent" as the comparison result to the subsequent recording unit 624.

[0241] In the sixth embodiment, the recording unit 624 has a memory unit such as a flash memory capable of updating records, similar to the first embodiment, but further includes a learning object setting unit 24f in addition to the label assigning unit 24a.

[0242] In the sixth embodiment, the learning object setting unit 24 f sets the medical image as the object of additional learning based on the comparison result acquired by the comparison unit 623 .

[0243] In the sixth embodiment, the label assigning unit 24a also generates a label corresponding to the comparison result output from the comparison unit 623, and assigns the label to the medical image related to the comparison result, similar to the first embodiment. Furthermore, the label assigning unit 24a generates a learning object label to be assigned to the medical image set in the learning object setting unit 24f, and assigns the learning object label to the medical image.

[0244] Furthermore, the operation unit 615 in the sixth embodiment includes an additional learning unit 625 that performs additional learning of the learning network in the recognition unit 623 only in the medical images to which the labels for the learning objects are assigned.

[0245] <Function of the Sixth Embodiment>

[0246] Then, in Figure 17 The operation of the image recording device according to the sixth embodiment will be described using a flowchart. Figure 17 This is a flowchart showing the operation of the image recording device according to the sixth embodiment.

[0247] like Figure 17 As shown, the image recording device of the sixth embodiment is the same as that of the first embodiment. First, a predetermined medical image such as an endoscopic image is acquired from the input unit 11 (step S61). In the present embodiment, the medical image is also assumed to be an image (endoscopic image) acquired by the user, the doctor, using a predetermined medical device (in the present embodiment, a medical endoscope device) in accordance with his or her own medical instructions (diagnosis and treatment guidelines, etc.).

[0248] Next, the recognition unit 621 recognizes the medical image such as the endoscopic image acquired in the input unit 11 in a manner consistent with the purpose of the examination, and sends the specified medical image as the recognition result to the comparison unit 623 (step S62).

[0249] On the other hand, the discovery motion analysis unit 22a in the user motion analysis unit 622 analyzes the doctor's motion and sends the analysis result to the comparison unit 623 (step S63).

[0250] Next, the comparison unit 623 obtains the recognition result recognized in the recognition unit 621 and the action analysis result obtained in the user action analysis unit 622, compares the recognition result and the action analysis result, and sends the comparison result to the label assignment unit 24a in the recording unit 624 (step S64).

[0251] Then, the learning object setting unit 24f sets the medical image as the object of additional learning based on the comparison result obtained in the comparison unit 623 (using pre-set labels such as inconsistent labels as the object of additional learning), and the label assigning unit 24a generates a learning object label to be assigned to the medical image set in the learning object setting unit 24f, and assigns the learning object label to the medical image (step S65).

[0252] Then, the additional learning unit 625 in the operation unit 615 performs additional learning of the learning network in the recognition unit 623 only on the medical images to which the learning object labels are assigned by the label assigning unit 24a (step S66).

[0253] <Effects of the Sixth Embodiment>

[0254] As described above, the image recording device according to the sixth embodiment, in addition to the effects of the first embodiment, selects medical images that become objects of additional learning based on the comparison results between the treatment corresponding to the recognition result and the analysis results of the doctor's treatment, and performs additional learning of the learning network in the recognition unit only on the selected medical images, thereby enabling learning to be performed with higher accuracy.

[0255] <Seventh embodiment>

[0256] Next, a seventh embodiment of the present invention will be described.

[0257] The image recording device of the seventh embodiment is characterized in that, as in the first to sixth embodiments, it includes a transmission unit that transmits the medical image stored in the memory unit of the recording unit within the image recording device and information regarding the comparison result accompanying the medical image to an external recording device. Since the other structures are the same as those of the first embodiment, only the differences from the first embodiment will be described here, and the description of the common parts will be omitted.

[0258] Figure 18 This is a block diagram showing the configuration of a medical system including an image recording apparatus according to a seventh embodiment of the present invention.

[0259] like Figure 18 As shown, the medical system 1 including the image recording device 710 of the present seventh embodiment is the same as the above-mentioned first embodiment, and mainly comprises the image recording device 710 for acquiring medical images and performing prescribed image processing, a storage unit 31 connected to the image recording device 710 and storing prescribed data, and a display unit 32 for displaying the medical images after image processing is performed in the image recording device 710.

[0260] Here, the storage unit 31 is an external data storage unit connected to the image recording device 710. In this seventh embodiment, it has the following function: saving the medical image stored in the memory unit of the recording unit 724 in the image recording device 710 and information about the comparison results attached to the medical image.

[0261] In addition, the storage unit 31 is implemented by various memories such as flash memory that can update records, information recording media such as hard disks, SSDs or CD-ROMs and their reading devices, or it can also be a file server set up in a medical base such as the hospital via an internal network (intra-hospital network) not shown in the figure.

[0262] The image recording device 710 in the seventh embodiment is also the same as the first embodiment, and mainly includes an input unit 11 for acquiring medical images captured by the camera unit in the endoscope, a control unit 12 for controlling the overall operation of the image recording device 710, and a calculation unit 715 for performing various processing described later on the medical images acquired in the input unit 11.

[0263] In the seventh embodiment, the calculation unit 715 is configured to include a recognition unit 721 , a user action analysis unit 722 , a comparison unit 723 , a recording unit 724 , and the like, and the details will be described later.

[0264] <Calculation Unit 715 in Seventh Embodiment>

[0265] Next, the detailed structure of the calculation unit 715 in the seventh embodiment will be described.

[0266] The operation unit 715 includes: an identification unit 721, which identifies medical images such as endoscopic images obtained in the input unit 11 and obtains the identification result; a user action analysis unit 722, which obtains the action analysis result by analyzing the action related to the doctor's (user's) interpretation of the medical image; a comparison unit 723, which compares the identification result obtained in the identification unit 721 and the action analysis result obtained in the user action analysis unit 722 to obtain the comparison result; and a recording unit 724, which stores the medical image and information about the comparison result obtained in the comparison unit 723.

[0267] The recognition unit 721 has the same structure and operational effects as the recognition unit 121 in the first embodiment, and therefore description thereof will be omitted here.

[0268] The user action analysis unit 722 is the same as in the first embodiment and includes a discovery action analysis unit 22a that analyzes actions related to lesion diagnosis performed by the user, the doctor. The structure and effects of the discovery action analysis unit 22a are the same as in the first embodiment, so their description is omitted here.

[0269] In addition, in this seventh embodiment, the comparison unit 723 is the same as the comparison unit 123 in the first embodiment, and compares the recognition result obtained from the recognition unit 721 and the analysis result obtained from the user action analysis unit 722, and sends the information on whether the treatment corresponding to the recognition result and the doctor's action (behavior) are "consistent" or "inconsistent" as the comparison result to the subsequent recording unit 724.

[0270] In this seventh embodiment, the recording unit 724 is the same as the first embodiment, for example, it has a memory unit composed of a flash memory or other memory that can update records, and has a label assignment unit 24a and a classification unit 24b that perform the same functions as the first embodiment.

[0271] In this seventh embodiment, the operation unit 715 has a transmission unit 726 for transmitting information related to the medical image stored in the memory unit of the recording unit 724 to an external recording device such as the above-mentioned storage unit 31.

[0272] As described above, in the memory section of the recording section 724, medical images recorded in a set recording method are stored together with the accompanying information. For example, medical images selected using labels based on the comparison results between the treatment corresponding to the recognition result and the doctor's actions (behaviors), the transmission section 726 in this seventh embodiment has the function of transmitting data related to these medical images to an external device such as the above-mentioned storage section 31.

[0273] <Effects of the Seventh Embodiment>

[0274] As described above, the image recording device according to this seventh embodiment, in addition to the effects of the first embodiment, can also accurately transmit the medical image data selected and stored through prescribed labels based on the comparison results between the treatment corresponding to the recognition result and the analysis results of the doctor's treatment to an external recording device together with the accompanying information.

[0275] The present invention is not limited to the above-described embodiment, and various changes and modifications can be made without departing from the spirit of the present invention.

Claims

1. An information processing device, characterized in that The information processing device has: an input unit for acquiring a medical image; a recognition unit configured to recognize the medical image acquired in the input unit and obtain a recognition result; a user motion analysis unit that obtains motion analysis results by analyzing the user's motion related to the interpretation of the medical image; and a comparing unit that compares the recognition result obtained in the recognition unit with the action analysis result obtained in the user action analysis unit to obtain a comparison result regarding whether the recognition result and the action analysis result are consistent or inconsistent, If the recognition result indicates that there is a lesion and the action analysis result indicates that there is treatment, the recognition result is deemed to be consistent with the action analysis result. When the recognition result indicates that there is a lesion and the motion analysis result indicates that no treatment is required, it is considered that the recognition result and the motion analysis result are inconsistent.

2. The information processing device according to claim 1, wherein The recognition unit further includes a detection unit that detects a specified abnormal area based on the medical image acquired in the input unit.

3. The information processing device according to claim 1, wherein The recognition unit further includes a classification unit that classifies the medical image acquired in the input unit.

4. The information processing device according to claim 1, wherein The user motion analysis unit includes a discovery motion analysis unit that analyzes the user's motion related to discovery of a lesion and acquires the motion analysis result.

5. The information processing device according to claim 4, wherein The discovery action analysis unit analyzes the user's action based on the action of the endoscope insertion portion related to the lesion discovery action.

6. The information processing device according to claim 4, wherein The discovery action analysis unit analyzes the user's action based on the operation of the endoscope operating unit related to the lesion discovery action.

7. The information processing device according to claim 4, wherein The discovery action analysis unit analyzes the user's action based on whether or not the lesion discovery action involves any treatment related to the endoscope.

8. The information processing device according to claim 1, wherein The user motion analysis unit includes a diagnostic motion analysis unit that analyzes motion related to lesion diagnosis performed by the user and acquires the motion analysis result.

9. The information processing device according to claim 8, wherein The diagnostic action analysis unit analyzes the diagnostic time required for the user to diagnose the lesion.

10. The information processing device according to claim 8, wherein The diagnostic action analysis unit analyzes information related to the diagnosis input by the user.

11. The information processing device according to claim 8, wherein The diagnostic action analysis unit analyzes the diagnostic result based on an index different from the recognition result recognized by the recognition unit.

12. The information processing device according to claim 8, wherein The comparing unit analyzes the information by comparing the pathological information related to the user's diagnosis of the lesion with the result of the recognition by the recognizing unit.

13. The information processing device according to claim 8, wherein The diagnostic action analysis unit analyzes the user's treatment action related to the user's diagnosis of the lesion.

14. The information processing device according to claim 1, wherein After the user recognizes the recognition result of the recognition unit, the user action analysis unit analyzes the user's actions related to the interpretation of the medical image and obtains the action analysis result.

15. The information processing device according to claim 1, wherein The comparison section acquires information on the degree of agreement or disagreement between the recognition result and the motion analysis result as the comparison result.

16. The information processing device according to claim 2, wherein: When the detection unit detects a predetermined abnormal area based on the medical image acquired in the input unit, the recognition unit acquires a recognition result that the predetermined abnormal area is detected.

17. The information processing device according to claim 3, wherein: The classification unit classifies the medical image acquired in the input unit according to the examination purpose, and obtains a classification result corresponding to the diagnostic index for the classified medical image as a recognition result.

18. The information processing device according to claim 5, wherein The discovery action analysis unit determines the action of the endoscope insertion portion related to the user's discovery of the lesion based on the status of a series of medical image groups or electrical signals that can be obtained from the endoscope-related device, and performs analysis.

19. The information processing device according to claim 6, wherein When the endoscope insertion portion is operated, a series of medical image groups are analyzed, and when it is determined that a singular point in the image is continuously captured for a predetermined time period or longer, it is analyzed that the removal of the insertion portion is stopped.

20. The information processing device according to claim 19, wherein The singular points in an image are feature points such as strong edges or edge endpoints based on pixel information.

21. The information processing device according to claim 7, wherein The discovery action analysis unit analyzes the user's action based on whether or not an endoscope-related treatment is performed when the recognition unit discovers a lesion based on the medical image acquired by the input unit.

22. The information processing device according to claim 21, wherein: When the recognition unit discovers a lesion based on the medical image acquired in the input unit, the discovery action analysis unit determines whether a prescribed treatment is performed thereafter, thereby analyzing whether the user has treated the lesion after discovering it or has intentionally ignored it.

23. The information processing device according to claim 7, wherein: Regarding the analysis of whether or not treatment is performed, a treatment tool is detected in an image. If the treatment tool is in a specific state, it is considered that the treatment tool is detected and "treatment is performed".

24. An image recording device comprising: an input unit for acquiring a medical image; a recognition unit configured to recognize the medical image acquired in the input unit and obtain a recognition result; a user action analysis unit, which obtains action analysis results by analyzing the user's actions related to the interpretation of the medical image; a comparing unit that compares the recognition result obtained in the recognition unit with the motion analysis result obtained in the user motion analysis unit to obtain a comparison result regarding whether the recognition result and the motion analysis result are consistent or inconsistent; and a recording section that stores the medical image and information about the comparison result obtained in the comparing section, If the recognition result indicates that there is a lesion and the action analysis result indicates that there is treatment, the recognition result is deemed to be consistent with the action analysis result. When the recognition result indicates that there is a lesion and the motion analysis result indicates that no treatment is required, it is considered that the recognition result and the motion analysis result are inconsistent.

25. The image recording device according to claim 24, wherein The recording unit has a label assigning unit, which generates a label corresponding to the comparison result obtained in the comparison unit, assigns the label to the medical image related to the comparison result, and the recording unit records the medical image after being labeled by the label assigning unit.

26. The image recording device according to claim 25, wherein The label assigning unit generates a consistent label indicating consistency based on the comparison result obtained in the comparison unit, and assigns the consistent label to the medical image related to the comparison result when the recognition result obtained in the recognition unit and the motion analysis result obtained in the user motion analysis unit are consistent. On the other hand, when the two results are inconsistent, the label assigning unit generates an inconsistent label indicating inconsistency, and assigns the inconsistent label to the medical image related to the comparison result.

27. The image recording device according to claim 25, wherein The label assignment unit generates a weighted label based on the comparison result obtained in the comparison unit and assigns the weighted label to the medical images related to the comparison result respectively. The weighted label represents the weight of the inconsistency between the recognition result obtained in the recognition unit and the motion analysis result obtained in the user motion analysis unit.

28. The image recording device according to any one of claims 25 to 27, characterized in that The recording unit further includes a classification unit that classifies the labels generated by the label applying unit.

29. The image recording device according to claim 25, wherein The recording unit further includes an image quality changing unit that changes the image quality associated with the medical image based on the label generated in the label assigning unit, and the recording unit records the medical image after the change by the image quality changing unit.

30. The image recording device according to claim 28, wherein The recording unit further includes an image quality changing unit that changes the image quality associated with the medical image based on the label classified in the grading unit, and the recording unit records the medical image after the change by the image quality changing unit.

31. The image recording device according to claim 25, wherein The recording unit records only the medical images to which predetermined labels are assigned based on the labels generated by the label assigning unit.

32. The image recording device according to claim 26, wherein The recording unit further includes an inconsistency processing unit that selects only the medical images to which the inconsistency label is assigned by the label assigning unit, and the recording unit records only the medical images selected by the inconsistency processing unit.

33. The image recording device according to claim 27, wherein The recording unit further includes an inconsistency processing unit that selects only medical images to be recorded based on the weighted labels generated in the label assigning unit, and the recording unit records only the medical images selected in the inconsistency processing unit.

34. The image recording device according to claim 24, wherein The recording unit also has a list making unit, which makes a list that associates the recognition results obtained in the recognition unit and the motion analysis results obtained in the user motion analysis unit, and the recording unit records the list made in the list making unit together with the corresponding medical image.

35. The image recording device according to claim 25, wherein The recording unit does not record only the medical images to which a predetermined label is assigned, based on the label generated by the label assigning unit.

36. The image recording device according to claim 25, wherein The recording unit further includes a learning object setting unit that sets a medical image as an object of additional learning based on the comparison result obtained by the comparing unit. The label assigning unit generates a learning object label to be assigned to the medical image set in the learning object setting unit, and assigns the learning object label to the medical image.

37. The image recording device according to claim 36, wherein The image recording device further includes an additional learning unit that performs additional learning of the learning network in the recognition unit only in the medical image to which the learning object label is assigned.

38. The image recording device according to claim 24, wherein The image recording device includes a learning object setting unit that sets a medical image as an object of additional learning based on the comparison result obtained in the comparison unit. The image recording device further includes an additional learning unit that performs additional learning of the learning network in the recognition unit only in the medical image that is the target of additional learning.

39. The image recording device according to claim 24, wherein The image recording device further includes a transmitting unit that transmits the medical image recorded in the recording unit in accordance with a predetermined condition to an external recording device.

40. The image recording device according to claim 39, wherein The image recording device further includes a transmitting section that transmits the medical image stored in the recording section and information on the comparison result acquired in the comparing section to an external recording device.

41. An information processing method, characterized in that The information processing method has the following steps: Input step, obtaining medical images; a recognition step of recognizing the medical image acquired in the input step and obtaining a recognition result; a user action analysis step of obtaining action analysis results by analyzing the user's actions related to the interpretation of the medical image; as well as a comparison step of comparing the recognition result obtained in the recognition step with the action analysis result obtained in the user action analysis step to obtain a comparison result on whether the recognition result and the action analysis result are consistent or inconsistent, If the recognition result indicates that there is a lesion and the action analysis result indicates that there is treatment, the recognition result is deemed to be consistent with the action analysis result. When the recognition result indicates that there is a lesion and the motion analysis result indicates that no treatment is required, it is considered that the recognition result and the motion analysis result are inconsistent.

42. An image recording method, characterized in that: The image recording method has the following steps: Input step, obtaining medical images; a recognition step of recognizing the medical image acquired in the input step and obtaining a recognition result; a user action analysis step of obtaining action analysis results by analyzing the user's actions related to the interpretation of the medical image; a comparing step of comparing the recognition result obtained in the recognition step with the action analysis result obtained in the user action analysis step to obtain a comparison result as to whether the recognition result and the action analysis result are consistent or inconsistent; as well as a recording step of storing the medical image and information about the comparison result obtained in the comparing step, If the recognition result indicates that there is a lesion and the action analysis result indicates that there is treatment, the recognition result is deemed to be consistent with the action analysis result. When the recognition result indicates that there is a lesion and the motion analysis result indicates that no treatment is required, it is considered that the recognition result and the motion analysis result are inconsistent.

43. A recording medium having an information processing program recorded thereon, characterized in that: The information processing program is used to cause a computer to execute the following steps: Input step, obtaining medical images; a recognition step of recognizing the medical image acquired in the input step and obtaining a recognition result; a user action analysis step of obtaining action analysis results by analyzing the user's actions related to the interpretation of the medical image; as well as a comparison step of comparing the recognition result obtained in the recognition step with the action analysis result obtained in the user action analysis step to obtain a comparison result on whether the recognition result and the action analysis result are consistent or inconsistent, If the recognition result indicates that there is a lesion and the action analysis result indicates that there is treatment, the recognition result is deemed to be consistent with the action analysis result. When the recognition result indicates that there is a lesion and the motion analysis result indicates that no treatment is required, it is considered that the recognition result and the motion analysis result are inconsistent.

44. A recording medium having an image recording program recorded thereon, wherein: The image recording program is used to cause the computer to execute the following steps: Input step, obtaining medical images; a recognition step of recognizing the medical image acquired in the input step and obtaining a recognition result; a user action analysis step of obtaining action analysis results by analyzing the user's actions related to the interpretation of the medical image; a comparing step of comparing the recognition result obtained in the recognition step with the action analysis result obtained in the user action analysis step to obtain a comparison result as to whether the recognition result and the action analysis result are consistent or inconsistent; as well as a recording step of storing the medical image and information about the comparison result obtained in the comparing step, If the recognition result indicates that there is a lesion and the action analysis result indicates that there is treatment, the recognition result is deemed to be consistent with the action analysis result. When the recognition result indicates that there is a lesion and the motion analysis result indicates that no treatment is required, it is considered that the recognition result and the motion analysis result are inconsistent.

45. An information processing device comprising a processor and a recording medium having an information processing program recorded thereon, wherein: When the information processing program is executed by the processor, the following steps are performed: Input step, obtaining medical images; a recognition step of recognizing the medical image acquired in the input step and obtaining a recognition result; a user action analysis step of obtaining action analysis results by analyzing the user's actions related to the interpretation of the medical image; as well as a comparison step of comparing the recognition result obtained in the recognition step with the action analysis result obtained in the user action analysis step to obtain a comparison result on whether the recognition result and the action analysis result are consistent or inconsistent, If the recognition result indicates that there is a lesion and the action analysis result indicates that there is treatment, the recognition result is deemed to be consistent with the action analysis result. When the recognition result indicates that there is a lesion and the motion analysis result indicates that no treatment is required, it is considered that the recognition result and the motion analysis result are inconsistent.

46. ​​An image recording device comprising a processor and a recording medium having an image recording program recorded thereon, wherein: When the image recording program is executed by the processor, the following steps are performed: Input step, obtaining medical images; a recognition step of recognizing the medical image acquired in the input step and obtaining a recognition result; a user action analysis step of obtaining action analysis results by analyzing the user's actions related to the interpretation of the medical image; a comparing step of comparing the recognition result obtained in the recognition step with the action analysis result obtained in the user action analysis step to obtain a comparison result as to whether the recognition result and the action analysis result are consistent or inconsistent; as well as a recording step of storing the medical image and information about the comparison result obtained in the comparing step, If the recognition result indicates that there is a lesion and the action analysis result indicates that there is treatment, the recognition result is deemed to be consistent with the action analysis result. When the recognition result indicates that there is a lesion and the motion analysis result indicates that no treatment is required, it is considered that the recognition result and the motion analysis result are inconsistent.

Images