Image processing device, image processing system, image processing method, image processing program

Abstract

To provide a technique of dealing with video data properly while protecting privacy according to a content agreed by an imaged person.SOLUTION: An image processor 100 includes: an input unit for receiving input of video data obtained by imaging a patient diagnosed by a dental diagnosis device 1; a storage unit for storing consent content data 130 including a consent content of a patient on usage of video data; and an operation unit for executing image processing according to the consent content data 130 stored in the storage unit on the video data.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present disclosure relates to an image processing apparatus, an image processing system, an image processing method, and an image processing program for processing images.

Background Art

[0002] Conventionally, in order to protect the privacy of photographed people, a technique for performing image processing on a face image so that the face of the person becomes unclear is known. For example, Patent Document 1 discloses an image processing apparatus that performs a mask process of a substantially rectangular shape on the eye portion of a person represented in a photographed image.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] According to the image processing apparatus disclosed in Patent Document 1, it is possible to protect the privacy of photographed people. However, since the concept of privacy protection varies among individuals, it is important to appropriately handle video data while protecting privacy according to the content of consent regarding the privacy protection of photographed people.

[0005] The present disclosure has been made to solve the above problems, and an object thereof is to provide a technique for appropriately handling video data while protecting privacy according to the content of consent of photographed people.

Means for Solving the Problems

[0006] The image processing apparatus according to this disclosure comprises an input unit that receives video data obtained by photographing a patient being treated with a dental medical device, a storage unit that stores consent data including the patient's consent regarding the use of the video data, and a calculation unit that performs image processing on the video data according to the consent data stored in the storage unit.

[0007] The image processing system relating to this disclosure comprises a dental treatment device, a camera, and an image processing device that performs image processing on video data obtained by the camera capturing a patient being treated by the treatment device. The image processing device comprises an input unit into which video data is input, a storage unit that stores consent data including the patient's consent regarding the use of the video data, and a calculation unit that performs image processing on the video data according to the consent data stored in the storage unit.

[0008] The image processing method relating to this disclosure includes, as a process performed by a computer, the steps of: receiving video data obtained by photographing a patient being treated with a dental device; storing consent data including the patient's consent regarding the use of the video data; and performing image processing on the video data in accordance with the consent data stored in the storage step.

[0009] The image processing program relating to this disclosure causes a computer to perform the following steps: receive video data obtained by photographing a patient being treated with a dental device; store consent data including the patient's consent regarding the use of the video data; and perform image processing on the video data according to the consent data stored in the storage step. [Effects of the Invention]

[0010] According to this disclosure, image processing is performed on the video data in accordance with the consent of patients receiving treatment with dental devices. Therefore, the video data can be handled appropriately while protecting the privacy of the patients who were filmed, in accordance with their consent. [Brief explanation of the drawing]

[0011] [Figure 1] This is a schematic diagram showing the overall configuration of the image processing system according to Embodiment 1. [Figure 2] This is a diagram illustrating an example of an image captured by a full-body camera. [Figure 3] This is a diagram illustrating an example of an image captured by a patient camera. [Figure 4] This is a diagram illustrating an example of an image captured by the rear camera. [Figure 5] This is a block diagram showing the internal configuration of the image processing system according to Embodiment 1. [Figure 6] This is a diagram illustrating an example of consent data. [Figure 7] This is a diagram illustrating an example of a video corresponding to consent A. [Figure 8] This is a diagram illustrating an example of a video corresponding to consent B. [Figure 9] This is a diagram illustrating an example of a video corresponding to consent C. [Figure 10] This is a flowchart of the image processing performed by the image processing device according to Embodiment 1. [Figure 11] This is a flowchart of the image processing performed by the image processing device according to Embodiment 2. [Figure 12] This is a flowchart of the image processing performed by the image processing device according to Embodiment 3. [Figure 13] This is a flowchart of the image processing performed by the image processing device according to Embodiment 4. [Modes for carrying out the invention]

[0012] <Embodiment 1> Embodiment 1 of the present disclosure will be described with reference to the drawings. For the same or corresponding parts in the drawings, the same reference numerals are given and their descriptions are not repeated.

[0013] [Overall Configuration of Image Processing System] The overall configuration of the image processing system 1000 according to Embodiment 1 will be described with reference to FIGS. 1 to 4. FIG. 1 is a schematic diagram showing the overall configuration of the image processing system 1000 according to Embodiment 1. As shown in FIG. 1, the image processing system 1000 includes a medical device 1 and an image processing device 100.

[0014] The medical device 1 is, for example, a chair unit for an operator to perform dental treatment on a patient. "Treatment" includes at least any one of examination and treatment of a patient performed by medical personnel in dentistry (hereinafter also referred to as "operator"), such as dentists, dental assistants, teachers at dental universities, students at dental universities, and dental technicians. In Embodiment 1, the content of treatment is also referred to as "treatment content". "Examination" includes an operator exploring the dental condition and cause of a patient. In this embodiment, the content of examination is also referred to as "examination content". "Treatment" includes an operator treating a dental disease of a patient and an operator maintaining the beauty and health of a patient's teeth (esthetic treatment). In Embodiment 1, the content of treatment is also referred to as "treatment content". "Treatment content" includes at least any one of "examination content" and "treatment content". An action (procedure) performed by an operator to treat a dental disease of a patient and an action (procedure) performed by an operator to maintain the beauty and health of a patient's teeth are also referred to as a treatment, and treatment is composed of a combination of one or more treatments. In this embodiment, the content of the treatment is also referred to as "treatment content". "Treatment content" includes one or more "treatment contents".

[0015] Examples of treatment content include dental caries treatment, root canal treatment, plaque removal, implant, orthodontics, and whitening. However, any treatment content that can cure the patient's dental disease or maintain the aesthetics and health of the patient's teeth by the operator is included in the treatment content. Examples of treatment procedures include examination, cutting, filing, root canal cleaning, drying, temporary sealing, impression, scaling, prosthesis modification, measurement of periodontal pockets, suction, and tooth extraction. However, any treatment procedure that can cure the patient's dental disease or maintain the aesthetics and health of the patient's teeth by the operator is included in the treatment procedure content.

[0016] For example, root canal treatment is a treatment required when the pulp (such as nerves and blood vessels) in the root of a tooth becomes inflamed or infected due to the progression of dental caries. It involves removing the painful pulp, cleaning and disinfecting the root canal, and filling the inside of the tooth root to prevent reinfection. Root canal treatment consists of examination, pulp extirpation, root canal length measurement, cleaning and disinfection, root canal filling, and filling and covering as treatment procedures. The operator can perform root canal treatment on the patient by performing these procedures in sequence.

[0017] Examination is a procedure that includes the operator conducting a hearing on the patient and examining the patient's oral cavity to identify the patient's dental condition and cause and formulate a treatment plan. Pulp extirpation is a procedure that includes removing the painful pulp in root canal treatment. Root canal length measurement is a procedure that includes measuring the length of the hollow root canal after removing the pulp. Cleaning and disinfection is a procedure that includes cleaning and disinfecting to the depth of the hollow root canal. Root canal filling is a procedure that includes filling a special drug into the root canal to prevent bacteria from invading after cleaning and disinfection. Filling and covering is a procedure that includes filling a rubber-like filler into the root canal, making a base made of metal or fiber on it, and covering the base with a covering (crown).

[0018] The medical device 1 comprises a chair 11, a basin unit 12, a tray table 13, an instrument holder 14 for holding medical instruments 15, a foot controller 16, a display 17, an operation panel 18, a lighting device (operator light) 19, an instrument control device 21, a display control device 22, and a speaker 35.

[0019] Chair 11 is a medical chair for patients to sit or lie down on during medical examinations. Chair 11 includes a headrest 11a to support the patient's head, a backrest 11b to support the patient's back, a seat 11c to support the patient's tailbone, and a footrest 11d to support the patient's feet. The medical device 1 may also include a medical table on which the patient lies down instead of chair 11.

[0020] The basin unit 12 is a water supply and drainage device that includes a bowl 12a with a drain outlet, a cup stand 12b on which a cup is placed, and a water tap 12c for supplying water to the cup.

[0021] The tray table 13 is used as a stand for items during medical treatment. The tray table 13 is connected to the chair 11 or an arm (not shown) extending from the floor on which the chair 11 is placed. The tray table 13 may also be suspended by the chair 11 or an arm (not shown) extending from the floor on which the chair 11 is placed. For example, the tray table 13 may be suspended by an arm 6 branching off from the top of a pole 5 extending from the chair 11 or the floor on which the chair 11 is placed. The operator can manually rotate, move horizontally, and move vertically relative to the chair 11. During medical treatment, the operator may place a tray 30 on the top surface of the tray table 13. One or more medical instruments for treating the patient are placed on the tray 30.

[0022] "Medical instruments" refer to instruments used by the practitioner during treatment, such as a rubber dam isolation system, rubber dam sheet, root canal length measuring device, bur set, files (reamers), corner of the mouth counter, foul clip, broach, irrigation needle, irrigation syringe, temporary filling device, cleanser, turbine, forceps, vacuum, mirror, excavator, probe, and root canal material injector. Medical instruments 15 held by the instrument holder 14 of the medical device 1 are also included in "medical instruments," but the tray 30 mainly contains medical instruments that are not held by the instrument holder 14.

[0023] The dental instruments 15 are, for example, dental instruments such as air turbine handpieces, micromotor handpieces, ultrasonic scalers, and vacuum syringes, and are driven by the control of the instrument control device 21. However, the dental instruments 15 are not limited to these, and may also include intraoral cameras, light curing lamps, root canal length measuring devices, three-dimensional scanners, and root canal enlargers, or non-driven instruments such as mirrors, syringes, and filling instruments.

[0024] The foot controller 16 has multiple switches (pedals) that receive foot input from the operator. The operator can assign a predetermined function to each of these switches. For example, the operator can assign a function to change the configuration of the chair 11 to a switch on the foot controller 16, and when the foot controller 16 receives foot input from the operator, the headrest 11a and other components are driven based on that foot input. Furthermore, the operator can also assign a function to drive a medical instrument 15 to a switch on the foot controller 16, and when the foot controller 16 receives foot input from the operator, the medical instrument 15 is driven by the control of the instrument control device 21 based on that foot input. The operator can also assign other functions to the switches on the foot controller 16, such as a function to control the lighting and extinguishing of the lighting device 19.

[0025] The display 17 is mounted on the tray table 13 and displays various images under the control of the display control device 22.

[0026] The control panel 18 includes switches for operating the chair 11 and medical instruments 15, or for setting the operation of such operations. For example, when the control panel 18 receives an input operation to operate the chair 11, the chair 11 operates based on that input operation. For example, when the control panel 18 receives an input operation to set the rotation speed of the medical instruments 15, the rotation speed of the medical instruments 15 can be set based on that input operation.

[0027] The lighting device 19 is mounted at the tip of an arm 6 that branches off from the top of the chair 11 or a pole 5 extending from the floor on which the chair 11 is installed. The lighting device 19 can be switched between an illuminated state and an off state, and primarily supports the practitioner's treatment by illuminating the inside of the patient's mouth with light. The practitioner can change the position of the lighting device 19 by moving the arm. In addition to the lighting device 19, a display 17 may also be attached to the tip of the pole 5 or arm 6.

[0028] Speaker 35 outputs various sounds, including alert sounds for the operator and patient, and assist sounds to support medical treatment.

[0029] Furthermore, the image processing system 1000 includes a microphone 60, a display 110, and an input device 120.

[0030] Microphone 60 collects voices from patients and the practitioner during medical treatment. The voice data collected by microphone 60 is acquired by the image processing device 100.

[0031] The display 110 is connected to the image processing unit 100 and displays various images under the control of the image processing unit 100.

[0032] The input device 120 is connected to the image processing device 100 and serves as an interface for users, such as surgeons, to input various types of data into the image processing device 100. For example, the input device 120 may include a keyboard or mouse. Alternatively, the input device 120 may be an information terminal capable of communicating with the image processing device 100, such as a smartphone, smartwatch, wearable device, or tablet PC.

[0033] As described above, the medical device 1 allows the practitioner to perform dental treatment on the patient. The image processing system 1000 is configured to capture images of the practitioner and patient during dental treatment and to store and save video data including the images obtained from the capture (hereinafter also referred to as "captured images"). By accumulating and saving video data of the treatment, accidents that occur during treatment can be recorded, and the captured images can also be used as teaching materials for dental treatment.

[0034] Specifically, the image processing system 1000 includes a tray camera 51 attached to the display 17, a general camera 52 attached to the top of the pole 5, a patient camera 53 attached to the lighting device 19, and a rear camera 54 attached to the back of the chair 11.

[0035] The tray camera 51 is positioned to capture a video or still image of an area that includes at least the tray 30. The video data (hereinafter also referred to as "tray video data") obtained by the tray camera 51 is acquired by the image processing device 100. Various settings of the tray camera 51, such as the shutter speed, are pre-adjusted to the extent that the image processing device 100 can recognize the presence, shape, and type of medical instruments placed on the tray 30, as well as the position of the medical instruments in the shooting area of ​​the tray camera 51. The tray camera 51 may be installed in any location that can capture an area that includes at least the tray 30.

[0036] The overall camera 52 is positioned to capture video or still images of an area that includes at least the treatment space containing the dental device 1. Specifically, the overall camera 52 is positioned to capture video or still images of a wide area in the treatment space during treatment using the dental device 1, including at least the dentist, dental assistant, and patient. Because the overall camera 52 is mounted on top of the pole 5, it captures an overhead view of the actions of the dentist, dental assistant, and patient during treatment, as well as the operation of the dental device 1.

[0037] Figure 2 is a diagram illustrating an example of an image captured by the overall camera 52. As shown in Figure 2, the overall camera 52 captures an area that includes at least the treatment space. During treatment, the practitioner performs dental treatment using dental instruments, and the images captured by the overall camera 52 show the actions of the practitioner (dentist 3 or dental assistant 4) during treatment, the actions of the patient 2, the types of dental instruments being used, the positions of the dental instruments, and the operation of the dental device 1.

[0038] The term "treatment space" is not limited to the space containing all the objects included in the treatment space, but may also include only some of the objects included in the treatment space, such as the practitioner, the patient, and the treatment device 1 during treatment. Video data including images captured by the overall camera 52 (hereinafter also referred to as "overall video data") is acquired by the image processing device 100. Various settings such as the shutter speed of the overall camera 52 are pre-adjusted to the extent that the image processing device 100 can recognize the actions of the practitioner, such as the dentist or dental assistant, the patient, and the operation of the treatment device 1 during treatment.

[0039] The overall camera 52 may be installed in any location that can capture an area that includes at least the treatment space where the patient is being treated. In addition to recognizing the treatment space in a planar manner, the overall camera 52 may also be a depth camera capable of recognizing the treatment space in the depth direction. The overall camera 52 may also be a 360-degree camera capable of capturing the treatment space in all directions. Furthermore, the overall camera 52 may be capable of tracking subjects present in the treatment space (for example, people such as the practitioner and the patient).

[0040] The patient camera 53 is positioned to capture video or still images of an area that includes at least the inside of the patient's mouth. Normally, during treatment, the lighting device 19 is positioned in front of the patient's face, so the patient camera 53 attached to the lighting device 19 can naturally capture the area including the patient's face from the front. Video data including the images captured by the patient camera 53 (hereinafter also referred to as "patient video data") is acquired by the image processing device 100. Various settings of the patient camera 53, such as the shutter speed, are pre-adjusted to the extent that the image processing device 100 can recognize the inside of the patient's mouth during treatment.

[0041] Figure 3 is a diagram illustrating an example of an image captured by the patient camera 53. As shown in Figure 3, the patient camera 53 captures an area that includes at least the face of patient 2. During treatment, dental treatment is performed using dental instruments in the patient's oral cavity, and the image captured by the patient camera 53 shows the patient's face and the surrounding area during treatment.

[0042] The patient camera 53 may be installed in any location that allows for imaging of an area including at least the inside of the patient's mouth. The patient camera 53 may be capable of detecting three-dimensional positional coordinates, similar to the overall camera 52, so that the image processing device 100 can detect the positional relationship in the depth direction (for example, the direction in which the patient is viewed from the patient camera 53) between the area around the oral cavity and the medical instruments.

[0043] The rear camera 54 is positioned to capture video or still images from the back of the chair 11 of the medical device 1, including at least the area where the patient is located in the chair 11. The video data (hereinafter also referred to as "rear video data") obtained by the rear camera 54, including the captured information, is acquired by the image processing device 100. Various settings of the rear camera 54, such as the shutter speed, are pre-adjusted to the extent that the image processing device 100 can recognize the patient located in the chair 11 and the area around the chair 11.

[0044] Figure 4 is a diagram illustrating an example of an image captured by the rear camera 54. As shown in Figure 4, the rear camera 54 captures an area from the back of the chair 11 that includes at least the patient 2 located on the chair 11. During treatment, the practitioner performs dental treatment using dental instruments, and the images captured by the rear camera 54 show the actions of the practitioner (dentist 3 or dental assistant 4) during treatment, the actions of the patient 2, the types of dental instruments being used, the positions of the dental instruments, and the operation of the dental device 1.

[0045] The tray video data, overall video data, patient video data, and posterior video data mentioned above are collectively referred to as "video data." The term "video data" may refer to all of these, but it may also refer to at least one of them.

[0046] Furthermore, the image processing device 100 is configured to estimate the content of a procedure or treatment using data related to images taken during treatment and an estimation model that includes a machine learning-based neural network. Specifically, practitioners have a need to accurately and easily understand the content of treatment given to patients. For example, dentists are obligated to record the progress of a patient without delay after treatment. However, dentists are also required to treat patients efficiently in a short amount of time, so it may not always be possible to record the content of the procedures or treatments performed during treatment. In recent years, technologies have emerged that use AI (Artificial Intelligence) to analyze and learn from data and utilize it. If the content of dental treatment can be estimated using AI technology, it will be possible to estimate the content of treatment accurately and easily, thereby realizing more convenient dental care.

[0047] Here, a unique characteristic of dental treatment is that the practitioner performs various actions during treatment, such as combining multiple procedures according to the treatment objective, and selecting the appropriate dental instrument from multiple instruments and taking it out of tray 30 for use for each procedure. Furthermore, the order and types of dental instruments used can be said to represent the content of the treatment during treatment. Moreover, since the content of treatment is represented by a combination of multiple procedures, understanding this hierarchy makes it possible to infer the content of the treatment.

[0048] For example, the image processing device 100 can identify the presence, shape, type, and position of medical instruments placed on the tray 30 by analyzing the tray video data corresponding to the images captured by the tray camera 51 using image recognition or other methods.

[0049] The image processing device 100 can identify the actions of the practitioner during treatment, the actions of the patient, the types of medical instruments being used, the positions of the medical instruments, and the operation of the medical device 1 by analyzing the overall video data corresponding to the images captured by the overall camera 52 using image recognition or other methods.

[0050] The image processing device 100 can identify the actions of the operator during treatment, the actions of the patient, the types of medical instruments being used, and the positions of the medical instruments by analyzing patient video data corresponding to images captured by the patient camera 53 using image recognition and other methods.

[0051] The image processing device 100 can identify the actions of the operator during treatment, the actions of the patient, the types of medical instruments being used, the positions of the medical instruments, and the operation of the medical device 1 by analyzing the rear video data corresponding to the images captured by the rear camera 54 using image recognition or other methods.

[0052] The image processing system 1000 (image processing device 100) is configured to estimate the treatment or medical procedure by analyzing and learning the operation of the tray 30 and the medical device 1, the operator's actions, and the patient's actions using AI, based on tray video data, overall video data, patient video data, and back video data.

[0053] In order to estimate the content of treatment, it is necessary to train an estimation model, including a neural network, equipped with the image processing device 100, using images taken during treatment. For this purpose, it is sometimes necessary to photograph not only test patients but also actual patients who visit the dental clinic. Protecting the privacy of the photographed patients is important, but people have different views on privacy protection. For example, some patients consent to being photographed during treatment, while others consent if the images are processed in a way that makes them unidentifiable, such as by applying mosaic processing to the images. Therefore, as explained below, the image processing device 100 is configured to protect the privacy of the photographed patients according to their wishes.

[0054] [Internal Configuration of the Image Processing System] The internal configuration of the image processing system 1000 according to Embodiment 1 will be described with reference to Figure 5. Figure 5 is a block diagram showing the internal configuration of the image processing system 1000 according to Embodiment 1. As shown in Figure 5, the image processing system 1000 comprises a medical device 1, an image processing device 100, a plurality of cameras (tray camera 51, overall camera 52, patient camera 53, rear camera 54), a microphone 60, a display 110, an input device 120, and a server device 200.

[0055] The medical device 1 comprises a chair 11, an instrument control device 21, a display control device 22, a sound control device 32, and a basin unit 12. Within the medical device 1, each of the chair 11, instrument control device 21, display control device 22, sound control device 32, and basin unit 12 can transmit and receive data via CAN (Controller Area Network) communication. The medical device 1 can also transmit and receive data with the image processing device 100 via wired or wireless LAN (Local Area Network) communication, such as WiFi communication. Alternatively, the medical device 1 may transmit and receive data with the image processing device 100 via USB (Universal Serial Bus) connection.

[0056] The chair 11 includes a headrest 11a, a backrest 11b, a seat cushion 11c, and a footrest 11d, each of which is driven based on the control of the chair control unit 111. Specifically, when the chair control unit 111 receives control data based on the operator's operation received by the foot controller 16 or the operation panel 18, or control data from the image processing device 100, it raises or lowers the seat cushion 11c, or moves the headrest 11a, backrest 11b, and footrest 11d vertically or horizontally relative to the seat cushion 11c, based on the control data. When the headrest 11a, backrest 11b, and footrest 11d are positioned vertically relative to the seat cushion 11c, the chair 11 takes an upright position. This puts the patient in a seated position in the chair 11. When the headrest 11a, backrest 11b, and footrest 11d are positioned horizontally to the seat 11c, the chair 11 is tilted. This allows the patient to lie down on the chair 11 in a supine position. In this way, the chair control unit 111 changes the posture of the chair 11 by driving the headrest 11a, backrest 11b, seat 11c, and footrest 11d.

[0057] The instrument control device 21 includes an instrument control unit 211. When the instrument control unit 211 receives control data based on the operator's operation received by the foot controller 16 or the operation panel 18, or control data from the image processing device 100, it controls the driving or settings of the medical instrument 15 based on the control data. For example, when the operator steps on the switch for driving the air turbine handpiece on the foot controller 16, the instrument control device 21 rotates the cutting tool held in the head of the air turbine handpiece. For example, when the image processing device 100 outputs control data for controlling the rotation direction or rotation speed of the air handpiece, the instrument control device 21 sets the rotation direction or rotation speed of the air turbine handpiece based on the control data. In this way, the instrument control device 21 controls the driving of the medical instrument 15 based on the operator's operation to the foot controller 16 or the operation panel 18.

[0058] The display control device 22 includes a display control unit 221 and a panel control unit 222. When the display control unit 221 receives control data from the image processing device 100, it controls the display 17 based on the control data. When the panel control unit 222 receives control data based on the control of the image processing device 100, it controls the operation panel 18 based on the control data.

[0059] The sound control device 32 includes a sound control unit 321. When the sound control unit 321 receives control data from the image processing device 100, it controls the speaker 35 based on the control data.

[0060] The basin unit 12 includes a basin control unit 121 and a lighting control unit 122. When the basin control unit 121 receives control data from the image processing device 100, it controls the water supply and drainage in the basin unit 12 based on the control data. When the lighting control unit 122 receives control data from the image processing device 100, it controls the lighting and extinguishing of the lighting device 19 based on the control data.

[0061] Each of the chair control unit 111, instrument control unit 211, display control unit 221, panel control unit 222, sound control unit 321, basin control unit 121, and lighting control unit 122 described above is composed of a CPU (Central Processing Unit), ROM (Read Only Memory), and RAM (Random access memory), etc., mounted on a circuit board (not shown). Each of the chair control unit 111, instrument control unit 211, display control unit 221, panel control unit 222, sound control unit 321, basin control unit 121, and lighting control unit 122 may be pre-installed in the medical device 1, or they may be modularized and optionally attached to the medical device 1.

[0062] In the CAN communication between the chair control unit 111, instrument control unit 211, display control unit 221, panel control unit 222, sound control unit 321, basin control unit 121, and lighting control unit 122, medical treatment-related data, including log data from each control unit, is converted into communication packets for communication between the control units. However, in communication between control units, it is not necessary to convert the medical treatment-related data into communication packets as long as the data is transmitted and received.

[0063] "Medical treatment-related data" includes at least one past and present control data for at least one of the following components of the medical device 1: chair 11, basin unit 12, lighting device 19, instrument control device 21, display control device 22, medical instrument 15, foot controller 16, display 17, and operation panel 18. For example, as described above, "medical treatment-related data" includes log data showing the operation and control history of each of the components of the medical device 1: chair 11, basin unit 12, lighting device 19, instrument control device 21, display control device 22, medical instrument 15, foot controller 16, display 17, and operation panel 18. Note that "medical treatment-related data" is not limited to operation and control history data for various devices, such as log data, but may also include real-time operation and control data for various devices. Furthermore, "medical treatment-related data" may include data regarding the current status of various devices. For example, "medical treatment-related data" may include data to identify the current state (such as posture and position) of chair 11.

[0064] The basin unit 12 includes a storage unit 123 for storing medical-related data, including log data from the medical device 1, and a communication unit 124 for outputting the medical-related data stored by the storage unit 123 to the image processing device 100 via communication.

[0065] The storage unit 123 collects and stores medical treatment-related data from each of the chair control unit 111, instrument control unit 211, display control unit 221, panel control unit 222, sound control unit 321, basin control unit 121, and lighting control unit 122 by communicating with each of these control units via CAN. The storage unit 123 may be composed of memory such as ROM or RAM mounted on a circuit board (not shown), or it may be composed of a non-volatile storage medium such as a memory card.

[0066] The communication unit 124 communicates with the image processing device 100 via wired or wireless LAN communication, or USB connection. If the storage unit 123 is composed of a non-volatile storage medium such as a memory card, the image processing device 100 may acquire the medical treatment-related data stored on the storage medium via the communication unit 124 of the medical treatment device 1 using the communication device 103. Alternatively, the image processing device 100 may directly acquire the medical treatment-related data flowing through CAN communication via a volatile storage medium for temporary storage.

[0067] The image processing device 100 comprises a processing unit 101, a memory 102, a communication device 103, an input / output interface 104, and a storage device 105. The image processing device 100 may consist of a general-purpose computer connected to each medical device 1, such as a desktop PC (Personal Computer) or a laptop PC, or it may consist of an in-hospital server installed within the medical space or an external server installed outside the medical space.

[0068] The arithmetic unit 101 is an arithmetic entity (computer) that performs various processes by executing various programs, and is an example of an "arithmetic unit". The arithmetic unit 101 is composed of a processor such as a CPU (central processing unit) or an MPU (micro-processing unit). While the processor, as an example of the arithmetic unit 101, has the function of performing various processes by executing programs, some or all of these functions may be implemented using dedicated hardware circuits such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array). The term "processor" is not limited to processors in the narrow sense that execute processing using a stored-program method, such as a CPU or MPU, but may also include hardwired circuits such as ASICs or FPGAs. Therefore, the "processor," as an example of the arithmetic unit 101, can also be interpreted as a processing circuitry in which processing is predefined by computer-readable code and / or hardwired circuits. The arithmetic unit 101 may consist of one chip or multiple chips. Furthermore, the processor and associated processing circuits may comprise multiple computers interconnected by wire or wireless means, such as via a local area network or wireless network. The processor and associated processing circuits may also comprise a cloud computer that remotely performs calculations based on input data and outputs the calculation results to other devices located at a distance.

[0069] The memory 102 has a volatile storage area (e.g., a working area) for temporarily storing program code or work memory when the arithmetic unit 101 executes various programs. Examples of volatile memory 102 include DRAM (dynamic random access memory) and SRAM (static random access memory). The arithmetic unit 101 may be a microcomputer that includes memory.

[0070] The communication device 103 acquires medical treatment-related data from the medical treatment device 1 by communicating with it via wired or wireless LAN communication or USB connection. The communication device 103 may acquire medical treatment-related data from the medical treatment device 1 in other ways. For example, as described above, if the storage unit 123 is composed of a non-volatile storage medium such as a memory card, the image processing device 100 may acquire the medical treatment-related data stored on the storage medium via the communication unit 124 of the medical treatment device 1 using the communication device 103. Furthermore, the communication device 103 exchanges various types of data with the server device 200 via wired or wireless LAN communication. For example, the communication device 103 outputs video data acquired from each of the multiple cameras (tray camera 51, overall camera 52, patient camera 53, rear camera 54), or image-processed video data obtained by applying image processing to the video data, to the server device 200.

[0071] The input / output interface 104 is an interface for inputting and outputting various types of data to and from each of the multiple cameras (tray camera 51, overall camera 52, patient camera 53, rear camera 54), microphone 60, and display 110, and is an example of an "input unit" and an "output unit". For example, the input / output interface 104 acquires video data from each camera by communicating with each camera via wired or wireless LAN communication or USB connection. The input / output interface 104 may also acquire video data from each camera in other formats. For example, the input / output interface 104 may acquire video data that is temporarily recorded on a non-volatile recording medium such as a memory card removed from each camera. Furthermore, the input / output interface 104 acquires audio data indicating the voices of the patient and the operator during medical treatment by communicating with the microphone 60. In addition, the input / output interface 104 outputs control data from the arithmetic unit 101 to the display 110 by communicating with the display 110.

[0072] The storage device 105 has a non-volatile storage area for storing various programs or data executed by the arithmetic unit 101 and is an example of a "storage unit". The storage device may be one or more non-transitory computer-readable media or one or more computer-readable storage media. Examples of storage devices include HDDs (Hard Disk Drives) and SSDs (Solid State Drives).

[0073] The storage device 105 stores consent content data 130 and image processing program 150. Although not shown in the diagram, the storage device 105 may also store the estimation model described above. The arithmetic unit 101 can estimate the patient's medical treatment using the estimation model by inputting at least one of the video data acquired from each camera and the medical treatment-related data acquired from the medical treatment device 1 into the estimation model. The estimation model can improve the accuracy of estimating the patient's medical treatment by optimizing the parameters used in processing in the neural network based on training data (for example, a set of video data and medical treatment-related data and the correct medical treatment content data).

[0074] The consent data 130 includes the patient's consent regarding the use of video data obtained by filming the patient being treated by the dental device 1. Specifically, before treating the patient using the dental device 1, the practitioner asks the patient for consent regarding filming the practitioner and patient during dental treatment using the overall camera 52, patient camera 53, and rear camera 54, and regarding the protection of privacy regarding the handling of the video data obtained from such filming. The practitioner uses the input device 120 to input the content of the consent obtained with the patient (consent content) into the image processing device 100. The patient's consent regarding the use of the video data input in this way is stored in the storage device 105 as consent data 130.

[0075] The image processing program 150 is a program that allows the processing unit 101 to perform image processing on video data acquired from each camera in accordance with the consent content data 130 stored in the storage device 105.

[0076] Specifically, the computing unit 101 executes the image processing program 150 to store the video data acquired from each camera directly into the storage device 105 without processing, or to output it to the server device 200. Alternatively, the computing unit 101 executes the image processing program 150 to process the video data acquired from each camera so that at least the shape of the patient's eyes included in the video data image becomes unclear, before storing it in the storage device 105 or outputting it to the server device 200. Note that "processing" includes any process that processes the image so that at least the shape of the patient's eyes becomes unclear, such as a process that mosaics the part containing at least the patient's eyes, a process that blurs the image, a process that masks the part, a process that reduces the brightness or illumination of the part, and a process that changes the brightness (tone) of the part. Note that "parts" only need to include at least the patient's eyes (one or both), and may also include other parts of the patient such as ears, nose, and eyebrows. Furthermore, the computing unit 101 executes the image processing program 150 to select only the video data consisting of images that do not include the patient's face (for example, tray video data) from the video data acquired from each camera, and stores only that video data in the storage device 105 or outputs it to the server device 200.

[0077] Thus, "image processing" includes storing or outputting video data acquired from each camera without processing it, storing or outputting video data acquired from each camera after processing it, and selecting some video data from the video data acquired from each camera to store or output. Furthermore, the processing of video data acquired from each camera within "image processing" may include multiple processes with varying degrees of processing depending on the consent data 130 stored in the storage device 105. For example, the computing device 101 may vary the fineness of the mosaic applied to at least the patient's eyes, or vary the brightness, illuminance, or brightness (tone) of those parts, depending on the consent data 130 stored in the storage device 105.

[0078] The server device 200 is a cloud computer configured to enable communication between an in-hospital server installed within the clinical space, an external server installed outside the clinical space, or multiple image processing devices 100. The server device 200 stores and stores video data from multiple cameras (tray camera 51, overall camera 52, patient camera 53, rear camera 54) acquired from the image processing devices 100, or image-processed video data obtained by image processing performed by the processing unit 101 on said video data.

[0079] [Example of a video based on the agreed-upon content] Referring to Figures 6 to 9, an example of a video corresponding to the patient's consent generated by the image processing device 100 according to Embodiment 1 will be described. Figure 6 is a diagram illustrating an example of consent content data 130. As shown in Figure 6, the consent content data 130 includes a patient ID, facial recognition data, voice recognition data, and consent content.

[0080] The patient ID is identification information assigned to patients registered in the image processing system 1000, and is used by users such as surgeons to identify patients.

[0081] The facial recognition data is data of captured images of the patient's face registered in the image processing system 1000, and is used by the image processing device 100 to authenticate the patient. The facial recognition data is associated with each patient ID. From the standpoint of protecting patient privacy, it is preferable for the operator to obtain the patient's prior consent before storing the facial recognition data for facial recognition in the storage device 105.

[0082] The voice authentication data is voice data representing the voice of a patient registered in the image processing system 1000, and is used by the image processing device 100 to authenticate the patient. The voice authentication data is associated with each patient ID. From the standpoint of protecting patient privacy, it is preferable for the operator to obtain the patient's prior consent before storing the voice authentication data for voice authentication in the storage device 105.

[0083] The consent includes details regarding the level of protection of the patient's privacy in relation to the imaging. Specifically, three types of identification information are provided to identify the consent: consent A, consent B, and consent C. In the consent data 130, one of consent A, consent B, or consent C is associated with each patient ID.

[0084] The consent content corresponding to consent A includes not processing (image processing) the patient's eyes or other features included in the video data obtained by filming the patient. If the patient selects consent A, the image processing device 100 stores the patient's video data acquired from each camera in the storage device 105 without processing it. Alternatively, the image processing device 100 outputs the patient's video data acquired from each camera to the server device 200 without processing it. The server device 200 stores the unprocessed video data acquired from the image processing device 100.

[0085] Furthermore, if the patient selects consent A, the image processing device 100 outputs the patient's video data acquired from each camera during the medical examination to the display 110 without any processing. The display 110 displays the video of the medical examination based on the unprocessed video data acquired from the image processing device 100.

[0086] For example, Figure 7 is a diagram illustrating an example of a video corresponding to consent A. As shown in Figure 7, when the patient selects consent A, the video of the medical procedure in raw image mode is displayed on the display 110 along with an image showing the consent content selected by the patient (consent A in this example). In raw image mode, the patient and the operator are shown on the display 110 without any processing such as blurring applied to the patient's image, as captured by the tray camera 51, the overall camera 52, the patient camera 53, and the rear camera 54.

[0087] The consent content corresponding to consent B includes processing the eyes so that at least the shape of the patient's eyes in the video data images obtained by filming the patient are obscured. If the patient selects consent B, the image processing device 100 processes the patient's video data acquired from each camera so that at least the shape of the patient's eyes in the video data images are obscured, and then stores it in the storage device 105. Alternatively, the image processing device 100 processes the patient's video data acquired from each camera so that at least the shape of the patient's eyes in the video data images are obscured, and then outputs it to the server device 200. The server device 200 stores the processed video data acquired from the image processing device 100. Alternatively, the image processing device 100 outputs the patient's video data acquired from each camera and consent data indicating the consent content corresponding to consent B to the server device 200. Based on the consent content of consent B, the server device 200 processes the patient's video data acquired from the image processing device 100 so that at least the shape of the patient's eyes in the video data images are obscured, and stores the processed video data.

[0088] Furthermore, if the patient selects consent B, the image processing device 100 processes the patient's video data acquired from each camera during the medical examination and outputs it to the display 110. The display 110 displays the video of the medical examination based on the processed video data acquired from the image processing device 100.

[0089] For example, Figure 8 is a diagram illustrating an example of a video corresponding to consent B. As shown in Figure 8, when the patient selects consent B, the video of the medical procedure in processed image mode is displayed on the display 110 along with an image showing the consent content selected by the patient (consent B in this example). In processed image mode, the display 110 shows the patient and the operator being filmed by the tray camera 51, the overall camera 52, the patient camera 53, and the rear camera 54, respectively, but the image of the patient's eyes is processed so that the shape of the eyes is obscured. In this example, a mosaic image 10 is superimposed on the image of the patient's eyes so that the shape of the patient's eyes is obscured.

[0090] The consent content corresponding to consent C includes not including the patient's face in the video data obtained by filming the patient. If the patient selects consent C, the image processing device 100 stores in the storage device 105 only the video data consisting of images that do not include the patient's face from the video data of the patient acquired from each camera, and deletes the other video data consisting of images that include the patient's face. Alternatively, the image processing device 100 outputs to the server device 200 only the video data consisting of images that do not include the patient's face from the video data of the patient acquired from each camera, and deletes the other video data consisting of images that include the patient's face. The server device 200 stores the video data consisting of images that do not include the patient's face acquired from the image processing device 100. Alternatively, the image processing device 100 outputs to the server device 200 the video data of the patient acquired from each camera and consent data indicating the consent content corresponding to consent C. Based on the consent content of consent C, the server device 200 stores only the video data consisting of images that do not include the patient's face from the video data of the patient acquired from the image processing device 100, and deletes the other video data consisting of images that include the patient's face.

[0091] Furthermore, if the patient selects consent C, the image processing device 100 outputs to the display 110 only the video data consisting of images that do not include the patient's face, from the video data of the patient acquired from each camera during the medical examination. The display 110 displays the video of the medical examination based on the video data consisting of images that do not include the patient's face, acquired from the image processing device 100.

[0092] For example, Figure 9 is a diagram illustrating an example of a video corresponding to consent C. As shown in Figure 9, when the patient selects consent C, the video of the treatment in tray image mode is displayed on the display 110 along with an image showing the consent content selected by the patient (consent C in this example). In tray image mode, the display 110 shows the state of the tray being captured by the tray camera 51, but videos obtained from the overall camera 52, patient camera 53, and rear camera 54, which may show the patient's face, are not displayed on the display 110.

[0093] In this manner, the image processing device 100 performs image processing on the patient's video data obtained from each camera, according to the consent given by the patient, stores the processed video data, and displays a video based on the processed video data on the display 110.

[0094] [Image processing in image processing equipment] The image processing performed by the image processing device 100 will be described with reference to Figure 10. Figure 10 is a flowchart of the image processing performed by the image processing device 100 according to Embodiment 1. Each step shown in Figure 10 (hereinafter referred to as "S") is realized by the arithmetic unit 101 of the image processing device 100 executing the image processing program 150.

[0095] As shown in Figure 10, the image processing device 100 determines whether or not it has estimated the start of medical treatment (S1). For example, the image processing device 100 estimates that medical treatment has started if it determines, based on medical treatment-related data acquired from the medical device 1, that the backrest 11b and seat cushion 11c of the chair 11 have moved to the medical treatment position. The image processing device 100 may also estimate that medical treatment has started if it determines, based on medical treatment-related data acquired from the medical device 1, that the lighting device 19 has turned on. The image processing device 100 may also estimate that medical treatment has started if it determines, based on medical treatment-related data acquired from the medical device 1, that a medical instrument such as an air turbine handpiece has been activated. The image processing device 100 may also perform voice recognition of the conversation between the patient and the operator based on the voice data acquired from the microphone 60, and estimate that medical treatment has started based on the recognition result. The image processing device 100 may also estimate the start of medical treatment by other means, such as when it determines that the load applied to the seat cushion 11c exceeds a predetermined value, or when it determines that a medical instrument has been placed on the tray 30 placed on the top surface of the tray table 13.

[0096] If the image processing device 100 does not estimate the start of medical treatment (NO in S1), it repeats the process in S1. On the other hand, if the image processing device 100 estimates the start of medical treatment (YES in S1), it performs an authentication process to authenticate the patient who is the subject of medical treatment (S2).

[0097] For example, the image processing device 100 compares the captured image of the patient included in the video data obtained by at least one of the overall camera 52, patient camera 53, and rear camera 54 with the facial recognition data included in the consent content data 130 to determine whether the captured image of the patient's face matches the facial recognition data included in the consent content data 130. Alternatively, the image processing device 100 compares the voice data of the patient acquired by the microphone 60 with the voice recognition data included in the consent content data 130 to determine whether the voice data of the patient matches the voice recognition data included in the consent content data 130. Alternatively, the image processing device 100 may obtain the patient ID by reading a barcode printed on an ID card or the like owned by the patient with a camera or the like.

[0098] The image processing device 100 determines, based on the results of the authentication process, whether or not the patient ID of the patient being treated is registered in the consent content data 130 (S3). The image processing device 100 may authenticate patients introduced into the medical device 1 by methods other than facial recognition or voice recognition. For example, another PC installed in the hospital may track the movements of a patient whose patient ID has been identified through self-check-in or other means using security cameras within the hospital, recognize the medical device 1 into which the patient has been introduced, and notify the image processing device 100 associated with the medical device 1 of the patient's ID. By using such a method, the image processing device 100 can identify the patient ID without creating a database for facial recognition or voice recognition, thus allowing consideration for patients who do not want their facial or voice information to be acquired.

[0099] If the patient ID of the patient being treated is not registered in the consent content data 130 (NO in S3), the image processing device 100 registers the patient's ID in the consent content data 130 based on the data entered by the user, such as the surgeon, using the input device 120 (S4). Furthermore, the image processing device 100 registers the patient's consent details in the consent content data 130 based on the data entered by the user using the input device 120 (S5).

[0100] After registering the patient's patient ID and consent details in the consent details data 130, the image processing device 100 uses the tray camera 51, the overall camera 52, the patient camera 53, and the rear camera 54 to capture images of the patient and the surgeon during treatment (S6).

[0101] On the other hand, if the patient ID of the patient being treated is already registered in the consent content data 130 (YES in S3), the image processing device 100 uses the tray camera 51, the overall camera 52, the patient camera 53, and the rear camera 54 to capture images of the patient and the operator during treatment (S7). After starting to capture images in S6 or S7, the image processing device 100 stores the video data input from the tray camera 51, the overall camera 52, the patient camera 53, and the rear camera 54 in the volatile memory 102, performs image processing according to the consent content, and then stores the processed video data in the storage device 105. Subsequently, the image processing device 100 extracts the consent content of the patient identified based on the results of the authentication process from the consent content data 130 stored in the storage device 105 (S8).

[0102] The image processing device 100 determines whether the consent content registered in S5 or the consent content extracted in S8 is associated with consent A (S9). If the extracted consent content is associated with consent A (YES in S9), the image processing device 100 stores the video data stored in the volatile memory 102 directly into the storage device 105 without processing it (S10).

[0103] On the other hand, if the extracted consent content is not associated with consent A (NO in S9), the image processing device 100 determines whether the consent content registered in S5 or the consent content extracted in S8 is associated with consent B (S11). If the extracted consent content is associated with consent B (YES in S11), the image processing device 100 stores the video data stored in the volatile memory 102 in the storage device 105 after processing the eyes so that at least the shape of the patient's eyes included in the image of the video data is unclear (S12). In other words, the image processing device 100 stores the video data after performing image processing on the video data input via the input / output interface 104 in the storage device 105.

[0104] Furthermore, the image processing device 100 may perform a process to encrypt the video data stored in the storage device 105 to make it unreadable, or a process to decrypt the encrypted video data to make it readable, depending on the consent content data. For example, if the extracted consent content is associated with consent B, the image processing device 100 may encrypt the raw video data stored in memory 102, lock it to make it unreadable, and store it in the storage device 105, and at the same time, process the raw video data and store it in the storage device 105. In other words, the storage device 105 may store both locked raw video data and unlocked processed video data. In this way, even if the consent content included in the consent content data 130 is changed back from consent B to consent A, the image processing device 100 only needs to delete the processed video data stored in the storage device 105 and decrypt the raw video data stored in the storage device 105 to unlock it to make it readable.

[0105] On the other hand, if the extracted consent content is not associated with consent B (NO in S11), the image processing device 100 determines that the consent content registered in S5 or the consent content extracted in S8 is associated with consent C, and stores only the video data consisting of images that do not include the patient's face (for example, tray video data) from the video data stored in the volatile memory 102 in the storage device 105 (S13). The other video data consisting of images that include the patient's face (for example, overall video data, patient video data, and back video data) is deleted when the power supply to the memory 102 is cut off. The image processing device 100 may also delete the video data stored in the memory 102 by outputting a delete command to the memory 102 while the power supply to the memory 102 is continuing.

[0106] The image processing device 100 may, depending on the consent content data, perform a process to encrypt the video data images stored in the storage device 105 to make them unreadable, or a process to decrypt the encrypted video data images to make them readable. For example, if the extracted consent content is associated with consent C, the image processing device 100 may encrypt and lock the video data consisting of images including the patient's face (e.g., overall video data, patient video data, and back view video data) from the video data stored in memory 102 to make them unreadable, and store them in the storage device 105, while storing only the video data consisting of images that do not include the patient's face (e.g., tray video data) in the storage device 105 without encryption. In other words, the storage device 105 may store both locked video data consisting of images including the patient's face (e.g., overall video data, patient video data, and back view video data) and unlocked video data consisting of images that do not include the patient's face (e.g., tray video data). In this way, even if the consent content contained in the consent content data 130 is changed back from consent C to consent B, the image processing device 100 only needs to decode and unlock the video data (for example, overall video data, patient video data, and back view video data) consisting of images including the patient's face stored in the storage device 105 so that it can be read.

[0107] After S10, S12, or S13, the image processing device 100 displays a video of the medical treatment in a mode corresponding to the consent content on the display 110, along with an image showing the patient's consent content, as shown in Figures 7 to 9 (S14).

[0108] The image processing device 100 determines whether or not there has been a change in the consent content during the medical examination (S15). For example, the image processing device 100 determines whether or not the user has entered data to change the consent content using the input device 120 at the patient's request. Alternatively, the image processing device 100 may recognize the consent content requested by the patient by speech recognition based on the audio data of the conversation between the practitioner and the patient collected by the microphone 60, and determine whether or not there has been a change in the consent content according to the recognized consent content. Furthermore, the image processing device 100 may compare the recognized consent content with the consent content already stored in the storage device 105 as consent content data 130, and if the two do not match, it may determine that there has been a change in the consent content, and if the two match, it may determine that there has been no change in the consent content.

[0109] If there is a change in the consent content (YES in S15), the image processing device 100 modifies the consent content contained in the consent content data 130 (S16). Subsequently, the image processing device 100 performs image processing on the video data stored in the storage device 105 according to the modified consent content data to edit (modify or delete) the video data (S17).

[0110] For example, if the consent content in the consent content data 130 is changed from consent A to consent B, the image processing device 100 processes the unprocessed video data stored in the storage device 105 so that at least the shape of the patient's eyes in the video data images is unclear, and then stores it back in the storage device 105. Also, if the consent content in the consent content data 130 is changed from consent B to consent C, the image processing device 100 stores only the video data consisting of images that do not include the patient's face (for example, tray video data) from the video data stored in the storage device 105, and deletes the other video data consisting of images that include the patient's face (for example, overall video data, patient video data, and back video data). After that, the image processing device 100 returns to the process in S8.

[0111] Furthermore, if the consent content included in the consent content data is changed, the image processing device 100 may perform a process to encrypt the video data image stored in the storage device 105 to make it unreadable, or a process to decrypt the encrypted video data image to make it readable, according to the changed consent content data.

[0112] For example, if the consent content in the consent content data 130 is changed from consent A to consent B, the image processing device 100 may encrypt and lock the unprocessed video data stored in the storage device 105 to make it unreadable, while simultaneously processing the unprocessed video data and storing it in the storage device 105. In other words, the storage device 105 may store both the locked unprocessed video data and the unlocked processed video data. In this way, even if the consent content in the consent content data 130 is changed back from consent B to consent A, the image processing device 100 only needs to delete the processed video data stored in the storage device 105 and decrypt and unlock the unprocessed video data stored in the storage device 105 to make it readable.

[0113] Furthermore, if the consent content included in the consent content data 130 is changed from consent B to consent C, the image processing device 100 may encrypt and lock the video data stored in the storage device 105 that consists of images including the patient's face (for example, overall video data, patient video data, and back video data) to make it unreadable, while storing in the storage device 105 only the video data that does not consist of images that do not include the patient's face (for example, tray video data) without encryption. In other words, the storage device 105 may store both locked video data consisting of images including the patient's face (for example, overall video data, patient video data, and back video data) and unlocked video data consisting of images that do not include the patient's face (for example, tray video data). In this way, even if the consent content included in the consent content data 130 is changed back from consent C to consent B, the image processing device 100 only needs to decrypt and unlock the video data consisting of images including the patient's face (for example, overall video data, patient video data, and back video data) stored in the storage device 105 to make it readable.

[0114] On the other hand, if there is no change in the consent (NO in S15), the image processing device 100 determines whether or not it has estimated the end of the medical treatment (S18). For example, the image processing device 100 estimates that the medical treatment has ended if it determines, based on medical treatment-related data acquired from the medical device 1, that the backrest 11b and seat cushion 11c of the chair 11 have moved to a non-medical position. The image processing device 100 may also estimate that the medical treatment has ended if it determines, based on medical treatment-related data acquired from the medical device 1, that the lighting device 19 has turned off. The image processing device 100 may also estimate that the medical treatment has ended if it determines, based on medical treatment-related data acquired from the medical device 1, that a medical instrument such as an air turbine handpiece has stopped. The image processing device 100 may also perform voice recognition of the conversation between the patient and the operator based on the voice data acquired from the microphone 60, and estimate that the medical treatment has ended based on the recognition result. The image processing device 100 may also estimate the end of treatment by other means, such as when it determines that the load applied to the seat cushion 11c has fallen below a predetermined value, or when it determines that the medical instruments on the tray 30 placed on the top surface of the tray table 13 have been put away.

[0115] If the image processing device 100 does not estimate the end of the medical treatment (NO in S18), it returns to the process in S8. On the other hand, if the image processing device 100 estimates the end of the medical treatment (YES in S18), it terminates the capture by the tray camera 51, the overall camera 52, the patient camera 53, and the rear camera 54, and terminates the storage of the video data in the storage device 105. After that, the image processing device 100 terminates this processing flow.

[0116] As described above, the image processing device 100 according to Embodiment 1 can store video data in the storage device 105 without processing it, store the video data in the storage device 105 after processing the eyes so that at least the shape of the patient's eyes included in the video data image is unclear, or store only video data consisting of images that do not include the patient's face (for example, tray video data) in the storage device 105, depending on the consent content regarding the use of the video data of the patient being treated by the medical device 1. In this way, the image processing device 100 can appropriately store video data while protecting privacy according to the consent content of the captured patient. Furthermore, since the image processing device 100 automatically stores video data appropriately while protecting privacy according to the patient's consent content, user convenience is improved.

[0117] The image processing device 100 authenticates the patient when it estimates the start of dental treatment, and based on the authentication result, extracts the patient's consent from the consent data 130 stored in the storage device 105. This eliminates the need for the user to search for the patient's consent themselves, improving user convenience. Furthermore, even if, for example, multiple practitioners treat the same patient at different times, the image processing device 100 automatically authenticates the patient and extracts the patient's consent, allowing the patient's consent to be shared among multiple practitioners, eliminating the need for multiple practitioners to repeatedly ask the same patient for their consent.

[0118] The image processing device 100 authenticates the patient based on a captured image of the patient's face included in the video data, the patient's voice data, or a patient ID indicated by a barcode printed on an ID card owned by the patient. This eliminates the need for the user to authenticate the patient themselves, thus improving user convenience.

[0119] The image processing device 100 displays the patient's consent and a video based on the video data processed according to the consent data on the display 110. This allows the user to perform medical treatment while confirming that the video data has been processed according to the patient's consent. This improves user convenience because the user does not need to check the stored video data later. In addition, by checking the video displayed on the display 110, the patient can receive medical treatment with peace of mind knowing that their privacy is protected.

[0120] The image processing device 100 can appropriately process images of video data stored in the storage device 105 according to changes in consent data, by modifying or deleting the video data images according to the changed consent data when the consent data is changed. Furthermore, the image processing device 100 can accept changes in the patient's consent even during medical treatment and can appropriately process images of video data according to the changed consent, thereby improving user convenience.

[0121] <Embodiment 2> Embodiment 2 of this disclosure will be described with reference to the drawings. In the following description, only the parts of the image processing apparatus 100 according to Embodiment 2 that differ from the image processing apparatus 100 according to Embodiment 1 will be described, and the same components and processes as those in Embodiment 1 will be denoted by the same reference numerals, and their descriptions will be omitted.

[0122] Figure 11 is a flowchart of the image processing performed by the image processing device 100 according to Embodiment 2. As shown in Figure 11, the image processing device 100 performs the processes S1 to S7 in the same way as the image processing device 100 according to Embodiment 1. After starting shooting in S6 or S7, the image processing device 100 determines whether or not it has received an external output of video data (S21). If the image processing device 100 has not received an external output of video data (NO in S21), it proceeds to the process in S28.

[0123] When the image processing device 100 receives an external output of video data (YES in S21), it extracts the patient's consent content from the consent content data 130 stored in the storage device 105 (S22).

[0124] The image processing device 100 determines whether the extracted consent content is associated with consent A (S23). If the extracted consent content is associated with consent A (YES in S23), the image processing device 100 outputs the video data stored in the volatile memory 102 to the external device without processing it (S24). For example, the image processing device 100 outputs the video data stored in the volatile memory 102 to the server device 200 without processing it. The server device 200 stores the video data received from the image processing device 100. Note that "external output" is not limited to the server device 200; it may also refer to outputting the video data to the display 110 or the storage device 105.

[0125] On the other hand, if the extracted consent content is not associated with consent A (NO in S23), the image processing device 100 determines whether the extracted consent content is associated with consent B (S25). If the extracted consent content is associated with consent B (YES in S25), the image processing device 100 processes the video data stored in the volatile memory 102 so that at least the shape of the patient's eyes included in the video data image becomes unclear, and then outputs it externally (S26). For example, the image processing device 100 processes the video data stored in the volatile memory 102 so that at least the shape of the patient's eyes included in the video data image becomes unclear, and then outputs it to the server device 200. The server device 200 stores the processed video data received from the image processing device 100. Note that "external output" is not limited to the server device 200, but may also refer to outputting the video data to the display 110 or the storage device 105.

[0126] Furthermore, the image processing device 100 may encrypt the raw video data stored in memory 102, lock it to make it unreadable, and store it in storage device 105, and may also process the raw video data and output it externally to server device 200. In this way, even if the consent content included in the consent content data 130 is changed back from consent B to consent A, the image processing device 100 only needs to delete the processed video data stored in server device 200 and decrypt the raw video data stored in storage device 105 to unlock it for reading.

[0127] On the other hand, if the extracted consent content is not associated with consent B (NO in S25), the image processing device 100 determines that the extracted consent content is associated with consent C and outputs only the video data consisting of images that do not include the patient's face (for example, tray video data) from the video data stored in the volatile memory 102 to the external device (S27). For example, the image processing device 100 outputs only the video data consisting of images that do not include the patient's face (for example, tray video data) from the video data stored in the volatile memory 102 to the server device 200. The server device 200 stores the video data consisting of images that do not include the patient's face that it received from the image processing device 100. Note that "external output" is not limited to the server device 200, but may also refer to outputting the video data to the display 110 or the storage device 105. Note that other video data consisting of images that include the patient's face (for example, overall video data, patient video data, and back view video data) are deleted when the power supply to the memory 102 is cut off. The image processing device 100 may delete video data (for example, overall video data, patient video data, and back view video data) consisting of images including the patient's face stored in memory 102 by outputting a delete command to memory 102 while power supply to memory 102 is continued.

[0128] Furthermore, the image processing device 100 may encrypt and lock the video data consisting of images including the patient's face (for example, overall video data, patient video data, and back view video data) from the video data stored in memory 102, making it unreadable, and store it in storage device 105, while outputting to the server device 200 only the video data consisting of images that do not include the patient's face (for example, tray video data) in an unencrypted state. In this way, even if the consent content contained in the consent content data 130 is changed back from consent C to consent B, the image processing device 100 only needs to decrypt and unlock the video data consisting of images including the patient's face (for example, overall video data, patient video data, and back view video data) stored in storage device 105 to make it readable.

[0129] After S24, S26, or S27, the image processing device 100 determines whether or not there has been a change in the consent content (S28). If there has been a change in the consent content (YES in S28), the image processing device 100 changes the consent content included in the consent content data 130 (S29). Subsequently, the image processing device 100 performs image processing on the video data that has been output externally according to the changed consent content data to edit (modify or delete) the video data (S30). For example, the image processing device 100 may output a signal to the server device 200 to perform image processing on video data stored in the server device 200, or perform image processing on video data stored in the storage device 105 to edit (modify or delete) the video data, or perform image processing on video data displayed on the display 110 to edit (modify or delete) the video data. Subsequently, the image processing device 100 returns to the process in S21.

[0130] In addition, similar to Embodiment 1, if the consent content included in the consent content data is changed, the image processing device 100 may perform a process to encrypt the video data image stored in the storage device 105 to make it unreadable, or a process to decrypt the encrypted video data image to make it readable, according to the changed consent content data.

[0131] For example, if the consent content in the consent content data 130 is changed from consent A to consent B, the image processing device 100 may encrypt and lock the unprocessed video data stored in the storage device 105 to make it unreadable, while simultaneously processing the unprocessed video data and outputting it externally to the server device 200. In this way, even if the consent content in the consent content data 130 is changed back from consent B to consent A, the image processing device 100 only needs to delete the processed video data stored in the server device 200 and decrypt and unlock the unprocessed video data stored in the storage device 105 to make it readable.

[0132] Furthermore, if the consent content included in the consent content data 130 is changed from consent B to consent C, the image processing device 100 may encrypt the video data consisting of images including the patient's face (for example, overall video data, patient video data, and back view video data) among the video data stored in the storage device 105 and store it in the storage device 105, while outputting to the server device 200 only the video data consisting of images that do not include the patient's face (for example, tray video data) in an unencrypted state. In this way, even if the consent content included in the consent content data 130 is changed back from consent C to consent B, the image processing device 100 only needs to decrypt the video data consisting of images including the patient's face (for example, overall video data, patient video data, and back view video data) stored in the storage device 105 and unlock it so that it can be read.

[0133] On the other hand, if there is no change in the consent content (NO in S28), the image processing device 100 determines whether or not it has estimated the end of the medical treatment (S31). If the image processing device 100 has not estimated the end of the medical treatment (NO in S31), it returns to the process in S21. On the other hand, if the image processing device 100 has estimated the end of the medical treatment (YES in S31), it terminates the imaging by the tray camera 51, the overall camera 52, the patient camera 53, and the rear camera 54 (S32). After that, the image processing device 100 terminates this processing flow.

[0134] As described above, the image processing device 100 according to Embodiment 2 outputs video data to the outside without processing it, outputs the video data to the outside after processing the eyes so that at least the shape of the patient's eyes included in the video data image is unclear, or outputs video data consisting of images that do not include the patient's face (for example, tray video data) to the outside, depending on the consent content regarding the use of the video data of the patient being treated by the medical device 1. In this way, the image processing device 100 can output video data to the outside while protecting privacy according to the consent content of the captured patient. Furthermore, since the image processing device 100 automatically outputs video data to the outside while protecting privacy according to the patient's consent content, user convenience is improved.

[0135] <Embodiment 3> Embodiment 3 of this disclosure will be described with reference to the drawings. In the following description, only the parts of the image processing apparatus 100 according to Embodiment 3 that differ from the image processing apparatus 100 according to Embodiment 1 will be described, and the same components and processes as those in Embodiment 1 will be denoted by the same reference numerals, and their descriptions will be omitted.

[0136] Figure 12 is a flowchart of the image processing performed by the image processing device 100 according to Embodiment 3. As shown in Figure 12, the image processing device 100 performs the processes S1 to S7 in the same way as the image processing device 100 according to Embodiment 1. After starting image capture in S6 or S7, the image processing device 100 determines whether or not there has been a change in the consent content (S41). If there has been a change in the consent content (YES in S41), the image processing device 100 changes the consent content included in the consent content data 130 (S42). After that, the image processing device 100 returns to the process in S41.

[0137] On the other hand, if there is no change in the consent content (NO in S41), the image processing device 100 determines whether or not it has estimated the end of the medical treatment (S43). If the image processing device 100 has not estimated the end of the medical treatment (NO in S43), it returns to the process in S41. On the other hand, if the image processing device 100 has estimated the end of the medical treatment (YES in S43), it terminates imaging with the tray camera 51, overall camera 52, patient camera 53, and rear camera 54 (S44).

[0138] Subsequently, the image processing device 100 extracts the patient's consent content from the consent content data 130 stored in the storage device 105 (S45). The image processing device 100 determines whether the extracted consent content is associated with consent A (S46). If the extracted consent content is associated with consent A (YES in S46), the image processing device 100 stores the video data stored in the volatile memory 102 directly into the storage device 105 without any processing (S47).

[0139] On the other hand, if the extracted consent content is not associated with consent A (NO in S46), the image processing device 100 determines whether the extracted consent content is associated with consent B (S48). If the extracted consent content is associated with consent B (YES in S48), the image processing device 100 stores the video data stored in the volatile memory 102 in the storage device 105 after processing the eyes so that at least the shape of the patient's eyes included in the image of the video data is unclear (S49).

[0140] On the other hand, if the extracted consent content is not associated with consent B (NO in S48), the image processing device 100 determines that the extracted consent content is associated with consent C and stores only the video data consisting of images that do not include the patient's face (for example, tray video data) from the video data stored in the volatile memory 102 in the storage device 105 (S50). The other video data consisting of images that include the patient's face (for example, overall video data, patient video data, and back video data) is deleted when the power supply to memory 102 is cut off. After that, the image processing device 100 terminates this processing flow. The image processing device 100 may also delete the video data stored in memory 102 by outputting a delete command to memory 102 while power supply to memory 102 is continuing.

[0141] As described above, after each camera has finished capturing images, the image processing device 100 according to Embodiment 3 can, in accordance with the consent content regarding the use of the patient's video data to be treated by the medical device 1, either store the video data in the storage device 105 without processing it, or process the eyes in the video data so that at least the shape of the patient's eyes included in the image are unclear before storing it in the storage device 105, or store only the video data consisting of images that do not include the patient's face (for example, tray video data) in the storage device 105. This allows the image processing device 100 to appropriately store the video data while protecting privacy in accordance with the consent content of the captured patient after each camera has finished capturing images. Furthermore, since the image processing device 100 automatically stores the video data appropriately while protecting privacy in accordance with the patient's consent content, user convenience is improved.

[0142] <Embodiment 4> Embodiment 4 of this disclosure will be described with reference to the drawings. In the following description, only the parts of the image processing apparatus 100 according to Embodiment 4 that differ from the image processing apparatus 100 according to Embodiment 1 will be described, and the same components and processes as those in Embodiment 1 will be denoted by the same reference numerals, and their descriptions will be omitted.

[0143] Figure 13 is a flowchart of the image processing performed by the image processing device 100 according to Embodiment 4. As a premise, in the image processing device 100 according to Embodiment 4, the video data obtained by each camera is stored in the storage device 105 without any processing. The storage device 105 is secured so that the unprocessed video data does not leak to the outside. In this state, as shown in Figure 13, the image processing device 100 determines whether or not it has received an external output of the video data stored in the storage device 105 (S61). If the image processing device 100 has not received an external output of the video data (NO in S61), it repeats the process in S61.

[0144] When the image processing device 100 receives external output of video data (YES in S61), it extracts the patient's consent content shown in the video that was the subject of the external output from the consent content data 130 stored in the storage device 105 (S62).

[0145] The image processing device 100 determines whether the extracted consent content is associated with consent A (S63). If the extracted consent content is associated with consent A (YES in S63), the image processing device 100 outputs the video data stored in the storage device 105 to the external device without processing it (S64). For example, the image processing device 100 outputs the video data stored in the storage device 105 to the server device 200 without processing it. The server device 200 stores the video data received from the image processing device 100. Note that "external output" is not limited to the server device 200; it may also mean outputting the video data to the display 110.

[0146] On the other hand, if the extracted consent content is not associated with consent A (NO in S63), the image processing device 100 determines whether the extracted consent content is associated with consent B (S65). If the extracted consent content is associated with consent B (YES in S65), the image processing device 100 processes the video data stored in the storage device 105 so that at least the shape of the patient's eyes included in the video data images becomes unclear, and then outputs it externally (S66). For example, the image processing device 100 processes the video data stored in the storage device 105 so that at least the shape of the patient's eyes included in the video data images becomes unclear, and then outputs it to the server device 200. The server device 200 stores the processed video data received from the image processing device 100. Alternatively, the image processing device 100 outputs the video data stored in the storage device 105 and consent data indicating the consent content corresponding to consent B to the server device 200. Based on the consent content of consent B, the server device 200 processes the video data acquired from the image processing device 100 so that at least the shape of the patient's eyes included in the video data images becomes unclear, and stores the processed video data. Note that "external output" is not limited to the server device 200, but may also be output to the display 110.

[0147] On the other hand, if the extracted consent content is not associated with consent B (NO in S65), the image processing device 100 determines that the extracted consent content is associated with consent C and outputs only the video data consisting of images that do not include the patient's face (for example, tray video data) from the video data stored in the storage device 105 to the external device (S67). For example, the image processing device 100 outputs only the video data consisting of images that do not include the patient's face (for example, tray video data) from the video data stored in the storage device 105 to the server device 200. The server device 200 stores the video data consisting of images that do not include the patient's face that it received from the image processing device 100. Alternatively, the image processing device 100 outputs the video data stored in the storage device 105 and consent data indicating the consent content corresponding to consent C to the server device 200. Based on the consent content of consent C, the server device 200 stores only the video data consisting of images that do not include the patient's face from the patient video data acquired from the image processing device 100 and deletes the other video data consisting of images that include the patient's face. Note that "external output" is not limited to the server device 200; it may also involve outputting video data to the display 110. After that, the image processing device 100 terminates this processing flow.

[0148] As described above, after each camera has finished capturing images, the image processing device 100 according to Embodiment 4 can output the video data to the outside without processing it, output the video data to the outside after processing the eyes so that at least the shape of the patient's eyes included in the video data images is unclear, or output only the video data consisting of images that do not include the patient's face (for example, tray video data), depending on the consent content regarding the use of the video data of the patient being treated by the medical device 1. In this way, after each camera has finished capturing images, the image processing device 100 can appropriately output the video data to the outside while protecting the privacy of the captured patient according to the consent content. Furthermore, since the image processing device 100 automatically outputs the video data to the outside while protecting the privacy of the patient according to the consent content, user convenience is improved.

[0149] <Variation> This disclosure is not limited to the embodiments described above, and various further modifications and applications are possible. Modifications applicable to this disclosure are described below.

[0150] In the image processing apparatus 100 according to each of the embodiments described above, "image processing" is exemplified by storing or outputting video data acquired from each camera without processing, storing or outputting video data acquired from each camera after processing, and selecting some video data from the video data acquired from each camera to store or output. However, other processing may also be performed as "image processing".

[0151] For example, the image processing device 100 may change the degree to which at least the shape of the patient's eyes in the video data images obtained by each camera are obscured, depending on the level of patient privacy protection defined in the consent content included in the consent data. For example, if the level of patient privacy protection defined in the consent content is Level 1, the image processing device 100 may make the mosaic applied to at least the parts including the patient's eyes finer, or reduce the brightness, illuminance, or brightness (tone) of those parts, compared to when the level of patient privacy protection is Level 2, which is lower than Level 1.

[0152] Furthermore, the image processing device 100 may change the range in which the shape of the patient's eyes is obscured, centered on at least the position of the patient's eyes, in the video data images obtained by each camera, according to the level of patient privacy protection defined in the consent content included in the consent content data. For example, if the level of patient privacy protection defined in the consent content is Level 1, the image processing device 100 may increase the range in which the shape of the patient's eyes is obscured compared to when the level of patient privacy protection is Level 2, which is lower than Level 1.

[0153] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of this disclosure is indicated by the claims rather than the foregoing description, and all modifications within the meaning and scope of the claims are intended to be included. The configurations illustrated in these embodiments and those illustrated in the variations may be combined as appropriate. [Explanation of symbols]

[0154] 1 Dental equipment, 2 Patient, 3 Dentist, 4 Dental assistant, 5 Pole, 6 Arm, 10 Mosaic image, 11 Chair, 11a Headrest, 11b Backrest, 11c Seat, 11d Footrest, 12 Basin unit, 12a Bowl, 12b Cup stand, 12c Water tap, 13 Tray table, 14 Instrument holder, 15 Dental instruments, 16 Foot controller, 17, 110 Display, 18 Control panel, 19 Lighting device, 21 Instrument control device, 22 Display control device, 30 Tray, 32 Sound control device, 35 Speaker, 51 Tray camera, 52 Overall camera, 53 Patient camera, 54 Rear camera, 60 Microphone, 100 Image processing device, 101 Processing unit, 102 Memory, 103 Communication device, 104 Input / Output interface, 105 Memory device, 111 chair control unit, 120 input device, 121 basin control unit, 122 lighting control unit, 123 storage unit, 124 communication unit, 130 consent content data, 150 image processing program, 200 server device, 211 fixture control unit, 221 display control unit, 222 panel control unit, 321 sound control unit, 1000 image processing system.

Claims

1. An image processing device for processing images, An input unit into which video data obtained by filming patients being treated with dental equipment is input, A storage unit that stores consent data, including the patient's consent regarding the use of the aforementioned video data, An image processing apparatus comprising: a calculation unit that performs image processing on video data according to the consent content data stored in the storage unit.

2. The image processing apparatus according to claim 1, wherein the calculation unit stores in the storage unit the data obtained by performing the image processing on the video data input to the input unit.

3. The aforementioned arithmetic unit, The video data input to the input unit is stored in the storage unit. The image processing apparatus according to claim 1, which outputs to the outside data obtained by performing the image processing on the video data stored in the storage unit.

4. The aforementioned arithmetic unit, When the commencement of dental treatment is estimated, the patient's authentication is performed. An image processing apparatus according to any one of claims 1 to 3, wherein the content of the patient's consent is extracted from the storage unit based on the result of the authentication.

5. The image processing apparatus according to claim 4, wherein the calculation unit performs the authentication based on the captured image of the patient's face included in the video data.

6. The image processing apparatus according to claim 4, wherein the calculation unit performs the authentication based on the patient's voice data input from the input unit.

7. The image processing apparatus according to any one of claims 1 to 3, wherein the consent data includes, as consent content, content relating to the level of protection of the patient's privacy.

8. The image processing apparatus according to claim 7, wherein the consent content data includes, as consent content, at least one of the following: processing the eyes so that the shape of at least the patient's eyes included in the video data image becomes unclear; not processing the eyes included in the video data image; and not including the patient's face in the video data image.

9. The image processing apparatus according to claim 7, wherein the calculation unit changes the degree to which at least the shape of the patient's eyes included in the video data image is obscured, or changes the range in which the shape of the eyes is obscured, centered on the position of at least the patient's eyes included in the video data image.

10. The image processing apparatus according to any one of claims 1 to 3, wherein the calculation unit displays the consent content included in the consent content data and a video based on the video data after image processing according to the consent content data on a display.

11. The image processing apparatus according to any one of claims 1 to 3, wherein the calculation unit modifies or deletes the image of the video data stored in the storage unit according to the modified consent content data when the consent content data is changed.

12. The image processing apparatus according to any one of claims 1 to 3, wherein the calculation unit performs a process to encrypt the image of the video data stored in the storage unit to make it unreadable, or a process to decrypt the encrypted image of the video data to make it readable, according to the consent content data.

13. The aforementioned arithmetic unit, Based on the patient's voice data input from the input unit, the consent content is recognized, The image processing apparatus according to any one of claims 1 to 3, wherein the consent content data, including the recognized consent content, is stored in the storage unit.

14. The image processing apparatus according to claim 13, wherein the calculation unit compares the recognized consent content with the consent content included in the consent content data stored in the storage unit.

15. An image processing system that processes images, Dental treatment equipment, Camera and, The medical device comprises an image processing device that performs image processing on video data obtained by capturing a patient being treated by the medical device using the camera, The aforementioned image processing device is The input unit into which the aforementioned video data is input, A storage unit that stores consent data, including the patient's consent regarding the use of the aforementioned video data, An image processing system comprising: a calculation unit that performs the image processing on the video data according to the consent content data stored in the storage unit.

16. An image processing method that processes images using a computer, The process to be performed by the aforementioned computer is: The steps include receiving video data obtained by filming a patient being treated with a dental device, and A step of storing consent data, which includes the patient's consent regarding the use of the video data; An image processing method comprising the step of performing image processing on video data according to the consent content data stored in the storage step.

17. An image processing program that processes images, On the computer, The steps include receiving video data obtained by filming a patient being treated with a dental device, and A step of storing consent data, which includes the patient's consent regarding the use of the video data; An image processing program that performs the following steps: a step of performing image processing on video data according to the consent content data stored in the aforementioned storage step.

Images