Medical image processing apparatus, method, and program

Abstract

The present application provides a kind of medical image processing device, method and program capable of automatically performing time measurement related to the examination state of time series medical image.In the medical image processing device with processor, the processor performs: sequentially obtaining the medical image acquisition processing of time series medical image;Examination state recognition processing based on the medical image obtained sequentially identifies examination state;Time measurement processing for time measurement related to examination state;According to the identification result of examination state, the behavior of time measurement control processing is controlled;And the information related to the state of time measurement is notified to the user by the notification processing using the notification unit.

Description

Technical Field

[0001] This invention relates to a medical image processing apparatus, method, and program, and particularly to a technique for reporting useful information from the observation of medical images to a user. Background Technology

[0002] Currently, an endoscopic examination data recording system has been proposed, which records the type of irradiation light used in the endoscopic examination, the distribution state of the drug in the patient's body, the implementation status of the treatment performed by the doctor, and the imaging status of the images taken by the endoscope into the examination data storage unit (Patent Document 1).

[0003] In addition, the endoscopic examination data recording system described in Patent Document 1 calculates the usage time of each irradiation light, the number of times each irradiation light is used, the observation time of each drug, the number of times each drug is used, the implementation time of each treatment, and the number of times each treatment is performed during the endoscopic examination, and records the calculated data in the examination data storage unit. Furthermore, it calculates the usage time of each irradiation light, the number of times each irradiation light is used, the observation time of each drug, the number of times each drug is used, the implementation time of each treatment, and the number of times each treatment is performed between two designated points during the endoscopic examination, and records the calculated data in the examination data storage unit.

[0004] In addition, the two locations for the start and end times of the observation light use are specified by the observation light button, the two locations for the start and end times of the drug use observation are specified by the drug button, and the two locations for the start and end times of the selected treatment are specified by the treatment button.

[0005] In addition, the endoscopic examination data recording system described in Patent Document 1 includes a display control unit that displays the screen of the display unit on the progression of data on the type of irradiation light, drug distribution data, and treatment implementation data acquired during the endoscopic examination.

[0006] Previous technical documents

[0007] Patent documents

[0008] Patent Document 1: Japanese Patent Application Publication No. 2017-12666 Summary of the Invention

[0009] The technical problem to be solved by the invention

[0010] According to the endoscopic examination data recording system described in Patent Document 1, the progress of each state of the endoscopic examination is automatically recorded in time series, thereby enabling quantitative and objective analysis of the endoscopic examination.

[0011] However, in order to measure the usage time of each irradiation light, the observation time of each drug, and the implementation time of the treatment, it is necessary to operate the observation light button, the drug button, and the treatment button separately to specify two locations: the start time and the end time of the observation light usage, the start time and the end time of the drug observation, and the start time and the end time of the treatment implementation. This places a burden on doctors to operate the buttons during endoscopic examinations.

[0012] Preferably, such button operations should be minimized in endoscopic examinations that require complex techniques.

[0013] The present invention was made in view of this situation, and its object is to provide a medical image processing apparatus, method and program capable of automatically performing time measurements related to the examination status of time-series medical images.

[0014] means for solving technical problems

[0015] To achieve the above objectives, the invention of the first aspect is a medical image processing apparatus comprising a processor, wherein the processor performs: medical image acquisition processing of sequentially acquiring time-series medical images; examination state recognition processing of recognizing examination states based on the sequentially acquired medical images; time measurement processing of performing time measurements related to the examination state; time measurement control processing of controlling the action of time measurement based on the recognition result of the examination state; and notification processing of notifying the user of information related to the state of time measurement using a notification unit.

[0016] According to a first aspect of the present invention, the examination status is automatically identified based on sequentially acquired medical images, and the actions for time measurement related to the examination status are controlled according to the identification result. Therefore, in the examination of time-series-based medical images, the burden on the user for time measurement can be reduced. In addition, since the information related to the time measurement status is reported to the user by a notification unit, the user can be notified, for example, in the case of automatic identification of an incorrect examination status, or in the case of automatic identification of an examination status at a time different from the user's expected timing.

[0017] In the medical image processing apparatus according to the second aspect of the present invention, it is preferable that the notification process only notifies the user of information related to the state of time measurement during a certain period based on the timing of a change in the state of time measurement. This is because if information related to the state of time measurement is notified for a certain period, it may sometimes hinder the inspection, or the information related to the state of time measurement (e.g., information indicating "measurement started") may differ significantly from the current inspection state (e.g., "measurement in progress").

[0018] In the medical image processing apparatus according to the third aspect of the present invention, it is preferable that the notification processing causes a first display area of ​​the display unit, which functions as a notification unit, to display information related to the measurement time generated by the time measurement processing in real time, and causes a second display area of ​​the display unit, different from the first display area, to display information related to the state of the time measurement, or to notify the information related to the state of the time measurement as sound information from a speaker, which functions as a notification unit. The information related to the measurement time may be the measurement time itself, or it may be a bar graph with a length corresponding to the measurement time, etc.

[0019] In the medical image processing apparatus according to the fourth aspect of the present invention, preferably, the information related to the state of time measurement includes one or more of the information of start, end, temporary stop, and restart of time measurement, and the time measurement control processing controls one or more of the start, end, temporary stop, or restart of time measurement in the time measurement processing.

[0020] In the medical image processing apparatus according to the fifth aspect of the present invention, preferably, the processor performs a receiving process to receive information related to the state of time measurement from the user operation unit. In the time measurement control process, when information related to the state of time measurement is received from the user operation unit, the received information related to the state of time measurement is used preferentially, instead of the recognition result in the check state recognition process, to control the time measurement action. Therefore, in cases such as incorrect automatic recognition of the check state, correct time measurement can be performed by the user manually indicating the information related to the state of time measurement.

[0021] In the medical image processing apparatus according to the sixth aspect of the present invention, it is preferable that the processor performs a process to save the time measurement result of the time measurement process in the storage unit. This allows for subsequent confirmation of the time required for examinations, etc. Furthermore, the time required for the examination becomes an indicator of the screening quality.

[0022] In the medical image processing apparatus according to the seventh aspect of the present invention, it is preferable that the processor performs the process of saving the time measurement result of the time measurement processing to the storage unit, and when the user operation of the time measurement control processing is controlled by the user operation unit, the method of saving the time measurement result to the storage unit is changed according to the state of the controlled action. Therefore, it is possible to distinguish whether the time measurement result is the result obtained solely through automatic identification of the inspection state, or whether an error exists in the automatic identification of the inspection state and is corrected through user operation.

[0023] In the medical image processing apparatus according to the eighth aspect of the present invention, it is preferable that the examination status recognition processing is based on sequentially acquired medical images to determine whether the examination status is in progress. This allows for the measurement of the processing time.

[0024] In the medical image processing apparatus according to the ninth aspect of the present invention, it is preferred that the examination status recognition processing is performed on the medical images acquired sequentially to detect the treatment device, and the treatment device detection result is used to identify whether the treatment is in progress.

[0025] In the medical image processing apparatus according to the tenth aspect of the present invention, it is preferred that the medical image is an endoscopic image.

[0026] In the medical image processing apparatus according to the eleventh aspect of the present invention, it is preferred that the examination status recognition processing is based on the sequential acquisition of medical image recognition endoscopes during insertion, removal, or external use.

[0027] In the medical image processing apparatus according to the twelfth aspect of the present invention, it is preferred that the examination status recognition processing performs landmark detection processing on landmarks within the lumen based on the sequentially acquired medical images, and identifies the examination status based on the landmark detection results.

[0028] In the medical image processing apparatus according to the thirteenth aspect of the present invention, it is preferable that the examination state recognition processing performs activity detection processing based on sequentially acquired medical images to detect the activity of the endoscope observer, and identifies the examination state based on the detection result of the endoscope observer's activity. As long as the activity state of the endoscope observer is identified, it can be determined that insertion is occurring when the endoscope is moving forward, and withdrawal is occurring when it is retracting. Therefore, the timing for the transition from insertion to withdrawal can be determined as the start of withdrawal.

[0029] The invention involved in the fourteenth aspect is a medical image processing method, including: a step of sequentially acquiring time-series medical images; an examination state recognition step of recognizing an examination state based on the sequentially acquired medical images; a time measurement step of performing time measurement related to the examination state; a time measurement control step of controlling the action of time measurement according to the recognition result of the examination state; and a notification step of notifying the user of information related to the state of time measurement using a notification unit, wherein the processor executes the processing of each step.

[0030] In the medical image processing method according to the fifteenth aspect of the present invention, it is preferable that, in the notification step, information related to the state measured by time is notified to the user only during a certain period based on the timing at which the state measured by time changes.

[0031] In the medical image processing method according to the sixteenth aspect of the present invention, it is preferable that, in the notification step, the first display area of ​​the display unit, which functions as a notification unit, displays information related to the measurement time in the time measurement step in real time, and the second display area of ​​the display unit, which is different from the first display area, displays information related to the state of the time measurement, or the information related to the state of the time measurement is notified as sound information from the speaker, which functions as a notification unit.

[0032] In the medical image processing method according to the seventeenth aspect of the present invention, it is preferred to include a step of receiving state-related information of time measurement from a user operation unit. In the time measurement control step, when state-related information of time measurement is received from the user operation unit, the received state-related information of time measurement is used preferentially instead of checking the recognition result in the state recognition step to control the action of time measurement.

[0033] The invention involved in the eighteenth aspect is a medical image processing program that causes a processor to perform the processing steps of the medical image processing method of any one of the fourteenth to seventeenth aspects of the present invention.

[0034] The invention involved in the nineteenth aspect is a recording medium that is non-transitory and computer-readable, recording the program of the eighteenth aspect of the present invention.

[0035] Invention Effects

[0036] According to the present invention, time measurements related to the examination status of time-series medical images can be automatically performed, which can reduce the burden on users during examinations. Attached Figure Description

[0037] Figure 1 This is a schematic diagram showing the overall structure of an endoscope system including the medical image processing apparatus involved in this invention.

[0038] Figure 2 This is a block diagram illustrating an implementation of a medical image processing device.

[0039] Figure 3 This is a diagram representing the first notification example made by the display.

[0040] Figure 4 This is a diagram illustrating a second notification example made by the display.

[0041] Figure 5 This is an example of what is displayed on the monitor when the measurement results are saved.

[0042] Figure 6 This is a flowchart illustrating one embodiment of the medical image processing method involved in the present invention. Detailed Implementation

[0043] The preferred embodiments of the medical image processing apparatus, method, and program involved in the present invention will now be described with reference to the accompanying drawings.

[0044] [The overall structure of an endoscope system, including medical image processing equipment]

[0045] Figure 1 This is a schematic diagram showing the overall structure of an endoscope system including the medical image processing apparatus involved in this invention.

[0046] like Figure 1 As shown, the endoscope system 9 includes an endoscope observer 10 as an electronic endoscope, a light source device 11, an endoscope processor device 12, a display device 13, a medical image processing device 14, an operation unit 15, and a display 16.

[0047] The endoscope observer 10 is a device for capturing time-series medical images, including images of the subject, such as an observer for the lower digestive tract. The endoscope observer 10 has an insertion part 20 that is inserted into the subject (in this example, the large intestine, which is a luminal organ) and has a tip and a base; a hand-operated part 21 that is connected to the base side of the insertion part 20 and is gripped by the surgeon to perform various operations; and a universal tether 22 that is connected to the hand-operated part 21.

[0048] The insertion part 20 is formed into a long strip with a narrow diameter. The insertion part 20 is composed of a flexible soft part 25, a bending part 26 that can be bent by the operation of the hand operation part 21, and a top part 27 that houses a camera optical system (objective lens) and camera element 28 (not shown) connected sequentially from its base end to its top end.

[0049] The imaging element 28 is a CMOS (component metal oxide semiconductor) or CCD (charge coupled device) type imaging element. The image light of the observed area is incident on the imaging surface of the imaging element 28 through an observation window (not shown) that is opened on the top surface of the top portion 27 and an objective lens (not shown) disposed behind the observation window. The imaging element 28 captures (converts) the image light of the observed area incident on the imaging surface and outputs an imaging signal.

[0050] The handheld operating section 21 is equipped with various operating components operated by the doctor (user). Specifically, the handheld operating section 21 includes two bending operation knobs 29 for bending the bending section 26, an air / water supply button 30 for air / water supply operation, and a suction button 31 for suction operation. In addition, the handheld operating section 21 is equipped with a still image capturing indicator 32 and a treatment instrument inlet 33. The still image capturing indicator 32 is used to indicate the capture of a still image 39 of the observed area, and the treatment instrument inlet 33 is used to insert a treatment instrument (not shown) into the treatment instrument insertion path (not shown) within the through insertion section 20.

[0051] The universal plug 22 is a connecting plug used to connect the endoscope observer 10 to the light source device 11. The universal plug 22 contains a light guide 35, a signal cable 36, and a fluid tube (not shown) that passes through the insertion part 20. Additionally, at one end of the universal plug 22 are a connector 37a that connects to the light source device 11 and a connector 37b that branches off from the connector 37a and connects to the endoscope processor device 12.

[0052] By connecting connector 37a to the light source device 11, light guide 35 and fluid tube (not shown) are inserted into the light source device 11. Thus, necessary illumination light, water, and gas are supplied to the endoscope observer 10 from the light source device 11 via light guide 35 and fluid tube (not shown). As a result, illumination light is shone from the illumination window (not shown) on the top surface of the tip 27 toward the observed area. Furthermore, by pressing the aforementioned air / water supply button 30, air or water is sprayed from the air / water supply nozzle (not shown) on the top surface of the tip 27 toward the observation window (not shown) on the top surface.

[0053] By connecting connector 37b to endoscope processor device 12, signal cable 36 and endoscope processor device 12 are electrically connected. Thus, via signal cable 36, image signal of the observed part is output from camera element 28 of endoscope observer 10 to endoscope processor device 12, and control signal is output from endoscope processor device 12 to endoscope observer 10.

[0054] The light source device 11 supplies illumination light to the light guide 35 of the endoscope observer 10 via connector 37a. The illumination light is selected from various wavelength bands corresponding to the observation purpose, such as white light (light in the white wavelength band or light in multiple wavelength bands), light in one or more specific wavelength bands, or combinations thereof.

[0055] The endoscope processor device 12 controls the operation of the endoscope observer 10 via connector 37b and signal cable 36. Furthermore, based on the imaging signal acquired from the imaging element 28 of the endoscope observer 10 via connector 37b and signal cable 36, the endoscope processor device 12 generates an image (also referred to as "moving image 38") consisting of a time-series frame image 38a including an image of the subject. Additionally, when the endoscope processor device 12 operates the still image capturing instruction unit 32 using the handheld operation unit 21 of the endoscope observer 10, it simultaneously generates the moving image 38, setting one frame image from the moving image 38 as a still image 39 corresponding to the timing of the capturing instruction.

[0056] The dynamic image 38 and the still image 39 are medical images obtained by capturing images inside the patient's body, i.e., inside a biological organism. Furthermore, if the dynamic image 38 and the still image 39 are images obtained using light (special light) in the aforementioned specific wavelength band, then both are special light images. Moreover, the endoscope processor device 12 outputs the generated dynamic image 38 and still image 39 to the display device 13 and the medical image processing device 14, respectively.

[0057] Furthermore, the endoscope processor device 12 can also generate (acquire) a special light image with information having the specific wavelength band described above, based on the ordinary light image obtained from the white light described above. In this case, the endoscope processor device 12 functions as a special light image acquisition unit. Moreover, the endoscope processor device 12 obtains a signal with a specific wavelength band by performing calculations based on the color information of red, green, and blue [RGB(Red, Green, Blue)] or cyan, magenta, and yellow [CMY(Cyan, Magenta, Yellow)] contained in the ordinary light image.

[0058] Furthermore, the endoscope processor device 12 can also generate known characteristic images such as oxygen saturation images based on at least one of a normal light image obtained from the aforementioned white light and a special light image obtained from light of the aforementioned specific wavelength band (special light). In this case, the endoscope processor device 12 functions as a characteristic image generation unit. Moreover, the dynamic images 38 or static images 39, including the aforementioned images of the living organism, normal light images, special light images, and characteristic images, are all medical images taken of the human body or visualized from measurement results for the purpose of image-based diagnosis and examination.

[0059] The display device 13 is connected to the endoscope processor device 12 and functions as a display unit that displays dynamic images 38 and static images 39 input from the endoscope processor device 12. While checking the dynamic images 38 displayed on the display device 13, the doctor (user) performs operations such as advancing and retracting the insertion unit 20. If a lesion is found in the observed area, the doctor operates the static image acquisition indicator 32 to acquire a static image of the observed area. In addition, the doctor performs procedures such as diagnosis and biopsy.

[0060] [Medical Image Processing Device]

[0061] The medical image processing device 14 recognizes the examination status based on time-series medical images, controls actions related to time measurements based on the recognition results of the examination status, and informs the user of information related to the time-measured status. In this embodiment, a personal computer is used, for example. In addition, the operation unit 15 includes a keyboard and mouse that are wired or wirelessly connected to the personal computer, as well as buttons located on the hand-held operation unit 21 of the endoscope observer 10. The display (display unit) 16 uses a liquid crystal monitor or other monitor that can be connected to a personal computer.

[0062] <Implementation of Medical Image Processing Device 14>

[0063] Figure 2 This is a block diagram illustrating an implementation of a medical image processing device.

[0064] Figure 2 The medical image processing device 14 shown consists of a medical image acquisition unit 40, a CPU (Central Processing Unit) 41, an examination status recognition processing unit 42, a time measurement processing unit 43, a time measurement control processing unit 44, a notification processing unit 45, a display control unit 46, a sound information reproduction processing unit 47, and a memory 48. The processing of each unit is implemented by one or more processors.

[0065] The CPU 41 operates based on the operating system stored in the memory 48 and various programs including the medical image processing program involved in this invention. It includes the control unit for the medical image acquisition unit 40, the examination status recognition processing unit 42, the time measurement processing unit 43, the time measurement control processing unit 44, the notification processing unit 45, the display control unit 46, and the sound information reproduction processing unit 47. In addition, it also functions as a part of these units.

[0066] The medical image acquisition unit 40 is used with the endoscope processor device 12 ( Figure 1A medical image acquisition and processing unit (not shown) acquires frame images 38a, including a time sequence of subject images, sequentially from the endoscope processor device 12 via a wired or wireless image input / output interface. In this example, it acquires the dynamic image 38 captured by the endoscope observer 10. Furthermore, when a still image 39, as described above, is captured midway through the capture of the dynamic image 38 using the endoscope observer 10, the medical image acquisition unit 40 acquires both the dynamic image 38 and the still image 39 from the endoscope processor device 12.

[0067] The examination status recognition processing unit 42 is the part that recognizes the examination status based on the dynamic image 38 acquired by the medical image acquisition unit 40.

[0068] As an example of identifying inspection status, the inspection status identification processing unit 42 performs landmark detection processing on the landmarks inside the detection cavity based on the dynamic image 38.

[0069] This example illustrates an implementation method for performing an endoscopic examination of the large intestine by inserting the endoscope 10 into the large intestine. During the endoscopic examination of the large intestine, the time required for the examination, i.e., the withdrawal time (the time required for the instrument to return from the cecum to the rectum after insertion), becomes an indicator of screening quality. Therefore, the withdrawal time is measured and the results are recorded.

[0070] When performing an endoscopic examination of the large intestine, the inspection status recognition processing unit 42 detects the Boan's valve at the ileocecal junction, the boundary between the large and small intestines, as a landmark within the lumen. Based on the detection result of this landmark (Boan's valve), it identifies whether the tip of the insertion part 20 of the endoscope observer 10 has reached the start position (examination status) for the endoscopic examination of the large intestine. The detection of landmarks within the lumen based on medical images can be performed, for example, using known AI (Artificial Intelligence) technology.

[0071] In addition, as another example of identifying the inspection status, the inspection status identification processing unit 42 identifies whether the endoscope observer 10 is inserted, withdrawn, or external based on the dynamic image 38.

[0072] When screening is performed while the endoscope observer 10 is being withdrawn, the timing for switching the endoscope observer 10 from insertion to withdrawal can be set to the start of withdrawal (the start of the withdrawal time measurement). Additionally, the withdrawal time measurement can be set to end when the endoscope observer is withdrawn from the body.

[0073] Whether the endoscope observer 10 is being inserted or withdrawn can be identified, for example, by the change in the size or position of a portion of the image within the frame image 38a constituting the dynamic image 38 between frames. If it is determined that the tip of the endoscope observer 10 is advancing, it is identified as being inserted; if it is determined that it is returning, it is identified as being withdrawn.

[0074] Furthermore, whether the tip of the insertion part 20 of the endoscope observer 10 is inside the lumen or outside the body can be identified based on image features, or based on significant changes in the subject or color tone in the frame image 38a. Moreover, when the identification result changes from inside the lumen to outside the body, the inspection status identification processing unit 42 can determine that the removal is complete (inspection is complete).

[0075] Furthermore, as another example of identifying the inspection status, the inspection status identification processing unit 42 identifies whether the inspection status is in progress based on the dynamic image 38. For example, the inspection status identification processing unit 42 performs a processing device detection process based on the dynamic image 38, and identifies whether the processing is in progress based on the detection result of the processing device. That is, when a processing device is detected from the dynamic image 38, the detection period can be identified as being in progress.

[0076] In addition, the timing when the treatment device is first detected in the dynamic image 38 can be identified as the start of treatment, and the timing when the treatment device is not detected in the dynamic image 38 after the treatment device is detected can be identified as the end of treatment.

[0077] In addition, the end of the procedure can also be identified based on the sequence of procedures. For example, if the procedure instrument is a biopsy forceps, and the timing of the forceps closing, EMR (Endoscopic mucosal resection), or ESD (Endoscopic Submucosal Dissection) is used, the timing of the clamping procedure being completed can also be identified as the end of the procedure.

[0078] The time measurement processing unit 43 is a unit that performs time measurements related to the inspection status, measuring specific times during the inspection.

[0079] The specific time measured by the time measurement processing unit 43 can be exemplified by the following two types.

[0080] 1. Removal time during colonoscopy

[0081] If the time from removal of the appendix to the end of the examination is short, it is reported as an increase in missed lesions. Therefore, the removal time has become an indicator of screening quality.

[0082] 2. Treatment time, etc.

[0083] The time required for treatment becomes a factor in the burden on the patient or the administration of sedatives.

[0084] The time measurement control processing unit 44 controls the time measurement actions performed by the time measurement processing unit 43 based on the identification result of the inspection status identified by the inspection status identification processing unit 42. Information related to the status of the time measurement performed by the time measurement processing unit 43 includes one or more of the following: start, end, temporary stop, and restart of the time measurement. The time measurement control processing unit 44 controls the start and end of the time measurement performed by the time measurement processing unit 43.

[0085] When the time measurement processing unit 43 measures the withdrawal time of a colonoscopy, the time measurement control processing unit 44 uses the examination status recognition processing unit 42 to detect a predetermined landmark (Bowen's valve). Then, the time measurement performed by the time measurement processing unit 43 is started when Bowen's valve is not detected, and the time measurement performed by the time measurement processing unit 43 is ended when the time is initially identified as external by the examination status recognition processing unit 42.

[0086] Alternatively, when the inspection status recognition processing unit 42 identifies it as being inserted and then identifies it as being being pulled out, the time measurement control processing unit 44 may switch the timing from being inserted to being pulled out as the timing for the start of pulling out and start the time measurement by the time measurement processing unit 43.

[0087] When the time measurement processing unit 43 measures the treatment time, the treatment device is detected by the inspection status recognition processing unit 42. When the treatment is detected, the time measurement control processing unit 44 takes the timing of the first detection of the treatment device as the timing of the start of the treatment and starts the time measurement performed by the time measurement processing unit 43. When the treatment is detected, the time measurement performed by the time measurement processing unit 43 ends when the timing of the first failure to detect the treatment device by the inspection status recognition processing unit 42 is taken as the timing of the end of the treatment.

[0088] Alternatively, preferably, the CPU 41 receives information related to the state of time measurement from the user operation unit, i.e., the operation unit 15. When the time measurement control processing unit 44 receives information related to the state of time measurement from the operation unit 15, it prioritizes using the received information related to the state of time measurement instead of checking the recognition result in the state recognition processing unit 42, and controls the action of time measurement.

[0089] When the status recognition processing unit 42 identifies information related to the status of time measurement, if an identification error occurs, the action of time measurement cannot be properly controlled (time measurement cannot be performed correctly). Therefore, when the status-related information of time measurement is received through user operation, the time measurement control processing unit 44 prioritizes the identification result of the status recognition processing unit 42 and uses the status-related information of time measurement through user operation.

[0090] The notification processing unit 45 is a part that notifies the user of information related to the time measurement status of the time measurement processing unit 43 through at least one of the display 16 and the speaker 17, which function as notification units. Further details of the notification of the information related to the time measurement status notified by the notification processing unit 45 will be described later.

[0091] The display control unit 46 includes an image display control unit 46A and an information display control unit 46B. The image display control unit 46A generates image data for display based on the medical image (moving image 38) acquired by the medical image acquisition unit 40 and outputs the data to the display 16. The information display control unit 46B generates image data for display representing information related to the state of time measurement based on the information reported by the notification processing unit 45 and outputs the data to the display 16.

[0092] The display control unit 46 combines the image data of the dynamic image 38 output from the image display control unit 46A and the image data representing information related to the state of time measurement output from the information display control unit 46B, and outputs the combined image data to the display 16, thereby displaying (notifying) the information related to the state of time measurement together with the medical image on the display 16.

[0093] The sound information reproduction processing unit 47 reproduces (announces) sound information that indicates the state of time measurement based on the information input from the notification processing unit 45, using the speaker 17. Furthermore, the sound information reproduction processing unit 47 can pre-store multiple sound information items based on the content of the information related to the state of time measurement, select the sound information corresponding to the information input from the notification processing unit 45, and output that sound information to the speaker 17.

[0094] The memory 48 includes flash memory, ROM (Read-only Memory), RAM (Random Access Memory), and a hard disk drive. The flash memory, ROM, and hard disk drive are non-volatile memories that store measurement times (removal time, treatment time) measured by the time measurement processing unit 43, as well as captured still images 39, in association with the operating system, various programs such as the medical image processing program involved in this invention, diagnostic reports, etc. The RAM, on the other hand, is a volatile memory capable of high-speed data read / write operations, serving as a temporary storage area for various programs stored in the non-volatile memory and as a working area of ​​the CPU 41.

[0095] [Notification of information related to the status of time measurement]

[0096] Next, the information related to the status of time measurement that is reported to the user by the notification processing unit 45 via the display 16 and the like will be explained.

[0097] <First Example of Notification>

[0098] Figure 3 This is a diagram representing the first notification example from the display.

[0099] exist Figure 3 On the screen 50 of the display 16 shown, a medical image 52 with dynamic imagery is displayed, along with text information 54 indicating the measurement time (e.g., the current removal time) as "03:40" in the first display area of ​​the display 16. In this example, the first display area is adjacent to the display area of ​​the medical image 52. Figure 3 The upper left area of ​​the image above.

[0100] When in a non-measurement state (before measurement begins), the notification processing unit 45 may display text information 54 such as "00:00", or it may not display it, or the entire text information 54 may be grayed out. This allows the user to monitor the measurement status during inspection.

[0101] In addition, the notification processing unit 45 displays the measurement time performed by the time measurement processing unit 43 in real time. The user can understand whether the inspection is in progress and the inspection time from the start of the inspection to the present based on the text information 54 of the real-time changing measurement time.

[0102] In addition, by observing whether the text information 54 changes from "00:00" or whether the inspection time is displayed from being undisplayed or grayed out, users can understand whether the device automatically recognizes the start of the inspection and begins measuring the time.

[0103] In addition, by stopping the change of text information 54, or by not displaying text information 54 or turning it gray, the user can understand that the device automatically recognizes the end of the inspection and ends the time measurement.

[0104] <Second Reporting Example>

[0105] Figure 4 This is a diagram illustrating a second notification example from the display. Figure 4 (A) shows the screen displayed at the start of the measurement. Figure 4 (B) indicates the screen displayed during the measurement.

[0106] exist Figure 4 The display 16 shown on screen 50 displays a medical image 52 with dynamic images and text information 54 indicating the measurement time, similar to the first notification example.

[0107] The notification processing unit 45 notifies the user of information related to the time-measured state only during a certain period of time when the state measured by time changes further, through the display 16 which serves as the notification unit.

[0108] exist Figure 4 In the second notification example shown in (A), when the inspection status recognition processing unit 42 recognizes a change in the measurement status, such as from non-measurement to measurement start, the notification processing unit 45 displays text information such as "start measurement" 56 on the display 16 as information related to the time measurement status only for a certain period of time from the measurement start time.

[0109] Furthermore, the notification processing unit 45 displays the text information 56 in a second display area that is different from the first display area of ​​the text information 54 that displays the measurement time (in this example, the display area adjacent to the lower side of the display area of ​​the medical image 52).

[0110] Additionally, the period for which the text message 56 is displayed, such as "Start Measurement," is approximately a few seconds, preferably a period that the user can visually recognize. Alternatively, the period can be set by the user.

[0111] Users need to carefully observe the medical image 52 during the examination, and therefore tend to overlook areas outside the medical image 52.

[0112] Therefore, it is preferable that the text information 56 is displayed larger than the text information 54 announcing the measurement time, so as to easily attract the user's attention. On the other hand, when the text information 56 is announced in a conspicuous manner, it may become an obstacle to inspection, so when a certain period of time has elapsed from the start of the measurement, such as... Figure 4 (B) indicates the end text message 56.

[0113] This allows for the suppression of adverse effects on the inspection process and enables the reporting of information related to the status of time measurements in a more prominent manner. As a more prominent display method, it can be achieved through methods such as... Figure 4The status change is indicated by text message 56, or by symbols, etc. Additionally, it can be displayed along with the measurement time, or not.

[0114] Furthermore, the notification processing unit 45 can notify the user, as sound information, via the sound information reproduction processing unit 47 and the speaker 17, only during a certain period based on the timing of the change in the state measured by time. For example, it can replace the text information 56 such as "Start measurement" or emit a sound such as "Start measurement" together with the text information 56 such as "Start measurement".

[0115] Furthermore, the sound information is not limited to sounds indicating "start measurement," but can also be a buzzing sound that announces a change in the state of time measurement.

[0116] The removal time or treatment time of the colon endoscopy measured by the time measurement and processing unit 43 needs to be retained as an examination record. Therefore, the CPU 41 saves the removal time or treatment time in the storage unit (memory 48) in association with the diagnostic report, etc.

[0117] At this time, the CPU41 preferably saves the time measurement results measured solely based on the recognition result of the automatic recognition by the inspection status recognition processing unit 42 and the time measurement results corrected by the user operation (e.g., the measurement results where the user performed the measurement start operation because the inspection status recognition processing unit 42 misjudged the measurement start timing) in a form that can identify them (changing the method of saving time measurement results).

[0118] In addition, Figure 3 and Figure 4 In the first and second reporting examples shown, the information related to the time measurement status includes, but is not limited to, the inspection time (removal time), inspection start time, and inspection end time. For example, it includes, for instance, the treatment time, treatment start time, and treatment end time during treatment using a treatment device.

[0119] Figure 5 This is an example of a screen displayed on the monitor when the measurement results are saved.

[0120] Specifically, when recording time information (X min Y sec) as a measurement result, the measurement result is assigned the information of "no user correction" or "with user correction". That is, when the user operation of the operation unit 15 controls the time measurement control process (such as the start time of the time measurement), the method of saving the time measurement result to the storage unit is changed according to the controlled action status.

[0121] Therefore, users can confirm the measurement results of "no user correction" after the inspection is completed, and then check whether there is a measurement error.

[0122] In addition, when recording with "no user correction", it is preferable to display a prompt for confirmation of the measurement results before recording (right after the check is completed).

[0123] Figure 5 This is a diagram showing an example of prompting confirmation of measurement results when recorded with "no user correction".

[0124] exist Figure 5 The example shown displays a prompt for confirmation such as "Unplugged time is 7m20s. Please confirm." It also displays the start or end time of the measurement and a still image at that moment. This allows the user to easily verify whether the automatic measurement results are incorrect.

[0125] In addition, when the "OK" button 57 and the "NG" button 58 are displayed and the "OK" button 57 is selected, the measurement result of the automatic measurement is saved with the information "no user correction" appended. When the "NG" button 58 is operated, after the user corrects the measurement result, the measurement result is saved with the information "user correction" appended.

[0126] For example, users can correct measurement results by temporarily saving the time-stamped dynamic image 38 to memory 48, reproducing the saved dynamic image 38, and selecting to check the frame image at the beginning or the frame image at the end to correct the measurement results.

[0127] Furthermore, when the calibrated measurement results are saved in memory 48, the dynamic image 38 temporarily stored in memory 48 can be deleted.

[0128] [Medical Image Processing Methods]

[0129] Figure 6 This is a flowchart illustrating one embodiment of the medical image processing method according to the present invention, and... Figure 2 The processing order of each part of the medical image processing apparatus 14 shown is illustrated in relation to each other.

[0130] exist Figure 6 In step S10, the CPU 41 first sets the flag F indicating whether an examination is in progress to F=0, indicating that an examination is not in progress. Then, the medical image acquisition unit 40 acquires the medical image of the current time point as the object of processing from the endoscope processor device 12.

[0131] The examination status recognition processing unit 42 identifies the examination status based on the medical images acquired in step S12 (including medical images acquired before the current time point) (step S14, examination status recognition step). For example, in the case of an endoscopy, it identifies whether the endoscope is being inserted, withdrawn, or removed from the body, or whether it is a procedure based on a treatment device.

[0132] In step S16, CPU41 determines whether the flag F is F=1 (in the process of checking). If it is not F=1 (in the case of "No"), it proceeds to step S18, where it determines whether the check has started. The determination of the check start point can be based on the identification result of the check status in step S14.

[0133] In step S18, when it is determined that it is the start time of the inspection ("Yes"), the flag F is set to F=1 (in inspection) (step S20), the time measurement control processing unit 44 starts the time measurement performed by the time measurement processing unit 43, and the time measurement processing unit 43 performs the inspection time measurement (step S22, time measurement control step, time measurement step).

[0134] Next, the notification processing unit 45 notifies the display 16 and / or speaker 17 of the start of the inspection only for a certain period of time based on the timing of the inspection start, and then proceeds to step S12 (step S24, notification step). Figure 4 In the notification example shown, display 16 displays text information such as "Start Measurement" 56. Figure 4 (A)) After a certain period of time, the text information 56 is cleared and the text information 54 of the measurement time is displayed again.

[0135] On the other hand, when the inspection starts, the flag F is set to F=1 (in inspection). Therefore, in step S16, it is determined that the inspection is in progress and the process proceeds to step S26.

[0136] In step S26, it is determined whether the inspection has ended. The determination of the inspection end point can be based on the identification result of the inspection status in step S14.

[0137] In step S26, when it is determined that it is not the end point of the inspection (the case of "No"), the display 16 displays the measurement time during the inspection (measurement) and returns to step S12 (step S28).

[0138] On the other hand, in step S26, when it is determined that the inspection is at its end (in the case of "Yes"), the time measurement control processing unit 44 ends the time measurement performed by the time measurement processing unit 43 (step S30). In addition, the notification processing unit 45 only announces the information indicating the end of the inspection through the display 16 and / or the speaker 17 for a certain period of time based on the timing of the inspection's end (step S32, notification step).

[0139] Then, the examination time (withdrawal time) or treatment time of the endoscopy, measured by the time measurement and processing unit 43, is displayed on the display 16 (see reference). Figure 5 The data is stored in memory 48 in association with diagnostic reports, etc. (step S34).

[0140] also, Figure 6 The determination of the inspection start point in step S18 and the determination of the inspection end point in step S26 are performed based on the inspection status recognition result of the inspection status recognition processing unit 42. However, the inspection start point and inspection end point can also be determined based on user instructions through user operation of the operation unit 15. Preferably, when the inspection start point or inspection end point is indicated by the user, the user-indicated inspection start point or inspection end point is used preferentially instead of the recognition result of the inspection status recognition processing unit 42 to control the start and end of the inspection time measurement.

[0141] [other]

[0142] In the above embodiments, the endoscope processor and the medical image processing device are provided separately, but they can also be integrated into one unit. That is, the endoscope processor can also be configured to function as a medical image processing device.

[0143] In addition, the measured examination time or treatment time is stored in the memory of the medical image processing device in association with the diagnostic report, but it is not limited to this; it can also be stored in an external memory (storage unit) connected to the medical image processing device.

[0144] In addition, medical images are not limited to endoscopic images taken by an endoscopic observer; for example, they can also be time-series images acquired by other medical devices such as ultrasound diagnostic devices.

[0145] Furthermore, the hardware structure for various controls of the medical image processing apparatus implementing the above embodiments consists of various processors as shown below. These processors include general-purpose processors (CPUs) that execute software (programs) and function as various control units; processors with customizable circuit structures, such as FPGAs (Field Programmable Gate Arrays), which are programmable logic devices (PLDs); and processors with circuit structures specifically designed for performing specific processes, such as ASICs (Application Specific Integrated Circuits), which are dedicated circuits.

[0146] A processing unit can be composed of one of these various processors, or it can be composed of two or more processors of the same or different types (e.g., multiple FPGAs, or a combination of CPU and FPGA). Alternatively, a single processor can constitute multiple control units. As examples of a single processor constituting multiple control units, firstly, there are configurations such as client or server computers, where a combination of one or more CPUs and software is used to construct a single processor, which functions as multiple control units. Secondly, there are configurations such as System-on-Chip (SoC), where a single IC (Integrated Circuit) chip implements the overall system functionality including multiple control units. In this way, one or more of the aforementioned processors are used as hardware structures to construct various control units.

[0147] In addition, the present invention includes a medical image processing program that enables a computer to function as a medical image processing device according to the present invention by being installed on a computer, and a non-volatile storage medium on which the medical image processing program is recorded.

[0148] Furthermore, the present invention is not limited to the embodiments described above, and various modifications can be made without departing from the spirit of the present invention.

[0149] Symbol Explanation

[0150] 9. Endoscopic System

[0151] 10 Endoscopic Observation Device

[0152] 11 Light source device

[0153] 12 Endoscopic Processor Device

[0154] 13 Display devices

[0155] 14 Medical Image Processing Device

[0156] 15 Operations Department

[0157] 16 monitors

[0158] 17 speakers

[0159] 20 Insertion section

[0160] 21 Hands-on Operations Department

[0161] 22 Universal Rope

[0162] 25. Soft parts

[0163] 26. Bend

[0164] 27. Top part

[0165] 28 camera elements

[0166] 29. Bend the operating knob

[0167] 30 Gas and water supply buttons

[0168] 31. Attraction Button

[0169] 32 Still Image Capture Instruction Unit

[0170] 33 Treatment device inlet

[0171] 35 Optical Guide

[0172] 36 Signal Cable

[0173] 37a and 37b connectors

[0174] 38. Animated Images

[0175] 38a frame image

[0176] 39 Still Images

[0177] 40 Medical Image Acquisition Department

[0178] 41 CPU

[0179] 42. Check Status Identification and Processing Unit

[0180] 43 Time Measurement and Processing Department

[0181] 44 Time Measurement and Control Processing Unit

[0182] 45 Notification Processing Department

[0183] 46 Display Control Unit

[0184] 46A Image Display Control Unit

[0185] 46B Information Display Control Unit

[0186] 47. Sound Information Reproduction Processing Unit

[0187] 48 Memory

[0188] 50 screens

[0189] 52 Medical Images

[0190] Text information 54, 56

[0191] 57. "OK" button

[0192] 58 “NG” button

[0193] F mark

[0194] Steps S10 to S34

Claims

1. A medical image processing device, comprising a processor, wherein, The processor performs: Medical image acquisition and processing involves sequentially acquiring time-series medical images. Based on the sequentially acquired medical image recognition examination status, the examination status recognition process is performed. Time measurement processing for performing time measurements related to the inspection status; Time measurement control processing that controls the action of time measurement based on the identification result of the inspection status; as well as The notification process utilizes a notification unit to inform the user of information related to the status of the time measurement. The notification process only uses the notification unit to notify the user of information related to the state measured by the time during a certain period of time, based on the timing of a change in the state measured by the time. Information related to the state of the time measurement includes one or more of the following: start, end, temporary stop, and restart of the time measurement.

2. The medical image processing apparatus according to claim 1, wherein, The notification process causes the first display area of ​​the display unit, which functions as the notification unit, to display information related to the measurement time generated by the time measurement process in real time, and causes the second display area of ​​the display unit, which is different from the first display area, to display information related to the state of the time measurement, or to notify the information related to the state of the time measurement as sound information from the speaker, which functions as the notification unit.

3. The medical image processing apparatus according to claim 1 or 2, wherein, The time measurement control process performs one or more of the following controls: start, end, temporary stop, or restart of the time measurement.

4. The medical image processing apparatus according to claim 1 or 2, wherein, The processor performs reception processing to receive information related to the state of the time measurement from the user operation unit. In the time measurement control process, when information related to the state of the time measurement is received from the user operation unit, the received information related to the state of the time measurement is used preferentially instead of the identification result in the check state identification process to control the action of the time measurement.

5. The medical image processing apparatus according to claim 1 or 2, wherein, The processor performs a process to save the time measurement results of the time measurement process to the storage unit.

6. The medical image processing apparatus according to claim 4, wherein, The processor performs a process to save the time measurement result of the time measurement process to the storage unit. When the user operation unit controls the action of the time measurement control process through the user operation unit, the method of saving the time measurement result to the storage unit is changed according to the state of the controlled action.

7. The medical image processing apparatus according to claim 1 or 2, wherein, The examination status recognition process identifies whether the examination status is "under treatment" based on the sequentially acquired medical images.

8. The medical image processing apparatus according to claim 7, wherein, The inspection status recognition process is based on the sequentially acquired medical images to detect the treatment instruments, and the detection result of the treatment instruments is used to identify whether the treatment is in progress.

9. The medical image processing apparatus according to claim 1 or 2, wherein, The medical images are endoscopic images.

10. The medical image processing apparatus according to claim 9, wherein, The examination status recognition process is based on the sequentially acquired medical image recognition endoscope observations indicating whether the endoscope is inserted, withdrawn, or external.

11. The medical image processing apparatus according to claim 9, wherein, The examination status recognition process performs landmark detection processing on the medical images acquired sequentially to detect landmarks within the lumen, and identifies the examination status based on the landmark detection results.

12. The medical image processing apparatus according to claim 10, wherein, The examination status recognition process performs activity detection processing on the sequentially acquired medical images to detect the activity of the endoscope observer, and identifies the examination status based on the detection results of the endoscope observer's activity.

13. A medical image processing method, comprising: The steps for sequentially acquiring time-series medical images; Based on the sequentially acquired medical image recognition examination status, the examination status recognition step is performed. The time measurement step for performing time measurements related to the inspection status; The time measurement control steps control the actions of the time measurement based on the identification results of the inspection status. as well as The notification step involves using a notification unit to inform the user of information related to the status of the time measurement. The processor executes each step of the processing. In the notification step, information related to the state of the time measurement is notified to the user only during a certain period based on a timing when the state measured by the time changes. Information related to the state of the time measurement includes one or more of the following: start, end, temporary stop, and restart of the time measurement.

14. The medical image processing method according to claim 13, wherein, In the notification step, the first display area of ​​the display unit, which functions as the notification unit, displays information related to the measurement time in the time measurement step in real time, and the second display area of ​​the display unit, which is different from the first display area, displays information related to the state of the time measurement, or the information related to the state of the time measurement is announced as sound information from the speaker, which functions as the notification unit.

15. The medical image processing method according to claim 13 or 14, wherein, This includes the step of receiving information related to the status of the time measurement from the user operation unit. In the time measurement control step, when information related to the state of the time measurement is received from the user operation unit, the received information related to the state of the time measurement is used preferentially instead of the identification result in the state identification step to control the action of the time measurement.

16. A recording medium, which is non-transitory and computer-readable, recording a program that causes a computer to perform the medical image processing method according to any one of claims 13 to 15.

Images