Data processing device, data processing method, and data processing program

Abstract

To provide a technology that appropriately records the positions of persons included in a medical care space.SOLUTION: A data processing device 100 comprises an input unit 1101 to which at least one image data obtained by imaging a medical care space is inputted in time series, and a data processing unit 1102 that processes the at least one image data. The data processing unit 1102 executes: a generation process of generating position data that indicates the positions of persons included in the medical care space, on the basis of the at least one image data that is inputted to the input unit 1101 in time series; an addition process of adding identification information that indicates a discretionary person to the position data; a determination process of determining whether or not the identification information added to the position data is incorrect, on the basis of a change of the position data in time series; and a correction process of correcting the identification information added to the position data when the identification information added to the position data is incorrect.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] The present disclosure relates to a data processing device, a data processing method, and a data processing program for processing at least one image data obtained by capturing an image of a medical space during medical treatment. [Background technology]

[0002] 2. Description of the Related Art Conventionally, medical treatment devices for treating patients have been known. For example, Patent Document 1 discloses a medical treatment device including components for treating patients, such as a chair, medical instruments, and a lighting device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-278325 Summary of the Invention [Problem to be solved by the invention]

[0004] Practitioners such as dentists and dental assistants can treat patients using the treatment device disclosed in Patent Document 1, and if it is possible to track and analyze the positions of multiple people, such as dentists, dental assistants, and patients, in a treatment space, it is possible to predict to some extent the treatment content that the practitioner will provide. However, if a person is mistakenly recognized when tracking the position of a person during treatment, there is a risk that the accuracy of predicting the treatment content will decrease.

[0005] The present disclosure has been made to solve such problems, and its purpose is to provide a technology for appropriately recording the positions of people included in a medical treatment space. [Means for solving the problem]

[0006] According to one example of the present disclosure, there is provided a data processing device that processes at least one image data obtained by capturing an image of a medical space. The data processing device includes an input unit to which the at least one image data is input in chronological order, and a data processing unit that processes the at least one image data. The data processing unit executes a generation process that generates position data indicating the position of a person included in the medical space based on the at least one image data input to the input unit in chronological order, an assignment process that assigns identification information indicating a person to the position data, a determination process that determines whether the identification information assigned to the position data is incorrect based on changes in the position data over time, and a correction process that corrects the identification information assigned to the position data if the identification information assigned to the position data is incorrect.

[0007] According to one example of the present disclosure, at least one image data obtained by photographing a medical space is The computer A data processing method is provided, which comprises: The process performed by the computer is The method includes the steps of: inputting at least one image data in chronological order; generating position data indicating the position of a person included in the medical space based on the at least one image data input in chronological order; assigning identification information indicating an arbitrary person to the position data; determining whether the identification information assigned to the position data is incorrect based on changes in the position data over time; and correcting the identification information assigned to the position data if the identification information is incorrect.

[0008] According to one example of the present disclosure, there is provided a data processing program for processing at least one image data obtained by photographing a medical space. The data processing program causes a computer to execute the following steps: inputting at least one image data in time series; generating position data indicating the positions of persons included in the medical space based on the at least one image data input in time series; assigning identification information indicating any person to the position data; determining whether the identification information assigned to the position data is incorrect based on changes in the position data over time; and correcting the identification information assigned to the position data if the identification information is incorrect. [Effects of the Invention]

[0009] According to the present disclosure, the positions of people included in a medical treatment space can be appropriately recorded. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a schematic diagram showing the overall configuration of a data processing system according to a first embodiment. [Figure 2] FIG. 10 is a diagram illustrating an example of an image captured by a full view camera. [Figure 3] FIG. 10 is a diagram illustrating an example of an image captured by a rear camera. [Figure 4] 1 is a block diagram showing an internal configuration of a data processing system according to a first embodiment. [Figure 5] 4 is a diagram for explaining an example of position data acquired by the data processing device according to the first embodiment. FIG. [Figure 6] FIG. 10 is a diagram for explaining an example of detection points included in the position data. [Figure 7] FIG. 10 is a diagram illustrating an example of an area in rear image data. [Figure 8] FIG. 10 is a diagram illustrating an example of synchronization data for estimation. [Figure 9] 1 is a block diagram showing a functional configuration of a data processing device according to a first embodiment. [Figure 10] 4 is a flowchart illustrating an example of data processing executed by the data processing device according to the first embodiment. [Figure 11] FIG. 10 is a block diagram showing a functional configuration of a data processing device according to a second embodiment. [Figure 12] 10 is a flowchart illustrating an example of data processing executed by a data processing device according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] <First Embodiment> A first embodiment of the present disclosure will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals and their description will not be repeated.

[0012] [Overall configuration of data processing system] 1 is a schematic diagram showing the overall configuration of a data processing system 1000 according to Embodiment 1. As shown in FIG.

[0013] The medical treatment device 1 is, for example, a chair unit for a practitioner to perform dental treatment on a patient. "Treatment" includes at least one of examination and treatment of a patient performed by a dental medical professional (hereinafter also referred to as "practitioner"), such as a dentist, dental assistant, dental school professor, dental school student, or dental technician. In the first embodiment, the content of the treatment is also referred to as "treatment content." "Examination" includes the practitioner investigating the patient's dental condition and cause of the disease. In the first embodiment, the content of the examination is also referred to as "examination content." "Treatment" includes the practitioner curing a patient's dental disease and maintaining the beauty and health of the patient's teeth (aesthetic treatment). In the first embodiment, the content of the treatment is also referred to as "treatment content." "Treatment content" includes at least one of "examination content" and "treatment content." The actions (treatments) taken by a practitioner to treat a patient's dental disease and the actions (treatments) taken by a practitioner to maintain the beauty and health of the patient's teeth are also referred to as treatments, and a treatment consists of a combination of one or more treatments. In the first embodiment, the content of a treatment is also referred to as "treatment content." "Treatment content" includes one or more "treatment content."

[0014] Examples of treatments include caries treatment, root canal treatment, plaque removal, implants, orthodontics, and whitening. Treatments that are performed by a surgeon to treat a patient's dental disease and aesthetic treatments that maintain the beauty and health of a patient's teeth are all included in the treatments. Examples of procedures include examination, cutting, filing, root canal cleaning, drying, temporary filling, impression taking, scaling, prosthetic correction, periodontal pocket measurement, suction, and tooth extraction. Treatments that are performed by a surgeon to treat a patient's dental disease and aesthetic treatments that maintain the beauty and health of a patient's teeth are all included in the treatments.

[0015] For example, root canal treatment is necessary when the pulp (nerves, blood vessels, etc.) inside the root of a tooth becomes inflamed or infected due to the progression of dental caries. This treatment involves removing the damaged pulp, cleaning and disinfecting the root canal, and filling the root to prevent re-infection. Root canal treatment consists of examination, pulp removal, root canal length measurement, cleaning and disinfection, root canal filling, and filling and capping, and the surgeon can perform root canal treatment on the patient by performing each of these procedures in order.

[0016] An examination involves the surgeon interviewing the patient and examining the patient's oral cavity to identify the patient's dental condition and cause, and then formulating a treatment plan. Pulp removal is a procedure that involves removing the damaged pulp during root canal treatment. Root canal length measurement involves measuring the length of the hollow root canal after pulp removal. Irrigation and disinfection involves cleaning and disinfecting the hollow root canal to its depths. Root canal filling involves filling the root canal with a special agent to prevent bacteria from entering the root canal after cleaning and disinfection. Filling and capping involves filling the root canal with a rubber-like filling material, creating a metal or fiber base on top, and then placing a cap (crown) on the base.

[0017] The medical treatment device 1 includes a chair 11, a basin unit 12, a tray table 13, an instrument holder 14 for holding medical treatment instruments 15, a foot controller 16, a display 17, an operation panel 18, a lighting device (operating light) 19, an instrument control device 21, a display control device 22, and a speaker 35.

[0018] Chair 11 is a medical chair on which a patient sits or lies during medical treatment. Chair 11 includes a headrest 11a that supports the patient's head, a backrest 11b that supports the patient's back, a seat 11c that supports the patient's buttocks, and a footrest 11d that supports the patient's feet. Headrest 11a, backrest 11b, seat 11c, and footrest 11d are connected to chair control unit 111 and can be driven under the control of chair control unit 111.

[0019] The basin unit 12 is a water supply / drainage device including a bowl 12a formed with a drainage port, a cup stand 12b on which a cup is placed, and a water tap 12c for supplying water to the cup.

[0020] The tray table 13 is used as a storage table during medical treatment. The tray table 13 is connected to an arm (not shown) extending from the chair 11 or the floor on which the chair 11 is placed. The tray table 13 may be suspended from an arm (not shown) extending from the chair 11 or the floor on which the chair 11 is placed. For example, the tray table 13 may be suspended from an arm 6 branching off from the upper part of a pole 5 extending from the chair 11 or the floor on which the chair 11 is placed. The surgeon can manually rotate, move horizontally, and move vertically the tray table 13 relative to the chair 11. During medical treatment, the surgeon may place a tray 30 on the top surface of the tray table 13. One or more medical instruments for treating a patient are placed on the tray 30.

[0021] The "medical instruments" are instruments used by the practitioner during treatment, such as a rubber dam moisture-proof set, rubber dam sheet, root canal length measuring device, bur set, file (reamer), oral corner counter electrode, Farl clip, broach, irrigation needle, irrigation syringe, temporary filling material filler, cleanser, turbine, tweezers, vacuum, mirror, excavator, probe, and root canal material injector. Note that the "medical instruments" also include the medical instruments 15 held by the instrument holder 14 of the medical device 1, but the tray 30 mainly contains medical instruments that are not held by the instrument holder 14.

[0022] The medical instruments 15 are dental instruments such as an air turbine handpiece, a micromotor handpiece, an ultrasonic scaler, and a vacuum syringe, and are driven under the control of the instrument control device 21. However, the medical instruments 15 are not limited to these, and may also be an intraoral camera, a photopolymerization irradiator, a root canal length measuring device, a three-dimensional scanner, a root canal enlarger, or a non-driven instrument such as a mirror, a syringe, or a filling instrument.

[0023] The foot controller 16 has multiple switches (pedals) that are operated by the surgeon's feet. The surgeon can assign a specific function to each of these switches. For example, the surgeon can assign a function to change the state of the chair 11 to a switch on the foot controller 16. When the foot controller 16 receives a foot operation on the switch by the surgeon, the headrest 11a and other components are activated based on the foot operation. Furthermore, the surgeon can assign a function to activate the medical instrument 15 to a switch on the foot controller 16. When the foot controller 16 receives a foot operation on the switch by the surgeon, the medical instrument 15 is activated by the control of the instrument control device 21 based on the foot operation. The surgeon can also assign other functions to the switches on the foot controller 16, such as a function to turn on and off the lighting device 19.

[0024] The display 17 is attached to the tray table 13 and displays various images under the control of a display control device 22 .

[0025] The operation panel 18 includes switches for operating the chair 11 and the medical instrument 15, etc., or for setting the operations. For example, when the operation panel 18 receives an input operation for operating the chair 11, the chair 11 operates based on the input operation. For example, when the operation panel 18 receives an input operation for setting the rotation speed or the like of the medical instrument 15, the rotation speed or the like of the medical instrument 15 can be set based on the input operation.

[0026] The lighting device 19 is provided at the tip of an arm 6 that branches off at the top of the chair 11 or the pole 5 that extends from the floor on which the chair 11 is installed. The lighting device 19 can be switched between a lit state and an off state, and supports the surgeon in medical treatment by illuminating mainly the inside of the patient's oral cavity. The surgeon can change the position of the lighting device 19 by moving the arm. It should be noted that not only the lighting device 19 but also the display 17 may be attached to the tip of the pole 5 or the arm 6.

[0027] The speaker 35 outputs various sounds, such as sounds that alert the surgeon and patient, and assist sounds that assist medical treatment.

[0028] By using the medical treatment device 1 configured as described above, the practitioner can provide dental treatment to the patient. Here, the data processing system 1000 is configured to capture images of people such as the practitioner and the patient in the treatment space during dental treatment, and accumulate and store the captured image data (hereinafter also referred to as "image data") obtained by capturing the images. By accumulating and saving the image data during treatment, it is possible to record any accidents that occur during treatment, and the image data can also be used as educational materials for dental treatment.

[0029] Furthermore, in the first embodiment, the data processing device 100 is configured to estimate the treatment details or medical treatment details using image data obtained during treatment and an estimation model including a neural network that has undergone machine learning. Specifically, practitioners desire to accurately and easily grasp the treatment details for patients. For example, dentists are required to record the progress of patients without delay after treating them. However, because dentists are also required to treat patients efficiently within a short period of time, they may not be able to record the treatment details or medical treatment details each time they are performed during treatment. In recent years, technologies have been developed that use AI (artificial intelligence) to analyze, learn, and utilize data. If dental treatment details could be estimated using AI technology, the treatment details could be estimated accurately and easily, and the estimated treatment details could be used to realize more convenient dental treatment.

[0030] A unique feature of dental treatment is that the practitioner performs various actions during treatment, such as combining multiple procedures according to the purpose of the treatment, selecting the appropriate medical instrument from multiple instruments for each treatment, and removing and using it from the tray 30. The order and type of instruments used can also be said to represent the content of the treatment during treatment. Furthermore, since the system has a hierarchical structure in which the treatment content is expressed by the combination of multiple treatment content, it is possible to estimate the treatment content by understanding this hierarchy.

[0031] During treatment, the dentist, dental assistant, and patient each perform actions according to a roughly determined routine depending on the treatment content, and assume roughly determined postures depending on the treatment content. The order of actions and postures performed by the dentist, dental assistant, and patient can be said to represent the treatment procedure, and the treatment device 1 is also controlled to correspond to that procedure. Therefore, if the treatment procedure can be understood based on the order of actions and postures performed by the dentist, dental assistant, and patient, it will be possible to predict the control of the treatment device 1.

[0032] Furthermore, the surgeon selects an appropriate medical instrument from among a plurality of medical instruments according to the contents of the medical treatment, removes it from the tray 30, and performs the treatment using the removed medical instrument. The types of medical instruments selected and the order in which the medical instruments are used can be said to represent the procedure of the treatment, and the medical treatment device 1 is also controlled to correspond to that procedure. Therefore, if the procedure of the treatment can be understood based on the types of medical instruments selected by the surgeon and the order in which the medical instruments are used, it becomes possible to predict the control of the medical treatment device 1.

[0033] Therefore, as described above, the data processing system 1000 (data processing device 100) according to the first embodiment is configured to estimate the details of treatment or medical care by analyzing and learning the tray 30, the operation of the medical treatment device 1, the actions of the surgeon, and the actions of the patient using AI. For example, while examining a patient, a surgeon treats the patient by operating each component of the medical treatment device 1. If the actions of the surgeon during such examination can be predicted using AI technology and the medical treatment device 1 can be controlled, a medical treatment device 1 that is highly convenient for the surgeon can be provided.

[0034] To estimate the treatment details, it is necessary to train an estimation model including a neural network provided in the data processing device 100 using image data obtained during treatment. To achieve this, not only test patients but also actual patients visiting the dental clinic are sometimes photographed. To store and preserve image data obtained by photographing the treatment space during treatment, patient consent is required. However, obtaining patient consent every time the treatment space is photographed is not easy in terms of both work and time. Furthermore, if the image data obtained by photographing the patient is retained, the privacy of the photographed person cannot be adequately protected. Therefore, the data processing device 100 according to the first embodiment is configured to store only the data necessary to estimate the treatment details or treatment details while adequately protecting the privacy of the photographed person. The data processing of image data performed by the data processing device 100 is described in detail below.

[0035] 1, multiple cameras are attached to the medical examination device 1. Specifically, the data processing system 1000 includes a tray camera 51 attached to the display 17, an overall camera 52 attached to the top of the pole 5, a patient camera 53 attached to the lighting device 19, and a rear camera 54 attached to the back of the chair 11.

[0036] The tray camera 51 is positioned so as to capture moving or still images of an area including at least the tray 30. Image data including the captured image obtained by the tray camera 51 (hereinafter also referred to as "tray image data") is acquired by the data processing device 100. Various settings of the tray camera 51, such as the frame rate and shutter speed, are adjusted in advance to an extent that the data processing device 100 can recognize the presence, shape, and type of medical instruments placed on the tray 30, as well as the positions of the medical instruments in the capture area of ​​the tray camera 51. The tray camera 51 may be installed anywhere as long as it can capture an image of an area including at least the tray 30.

[0037] The overall camera 52 is positioned so as to capture video or still images of at least an area including the treatment space including the medical treatment device 1. Specifically, the overall camera 52 is positioned so as to capture video or still images of a bird's-eye view from above of a wide area including at least the dentist, dental assistant, and patient in the treatment space during treatment using the medical treatment device 1. The overall camera 52 is a fixed camera fixedly attached to the top of the pole 5, so it always captures the same area in the treatment space from the same position.

[0038] The overall camera 52 is configured with a 3D camera or the like so as to detect the three-dimensional position coordinates of the surgeon and the patient. For example, the overall camera 52 is configured with a camera capable of detecting the three-dimensional position coordinates of an object by image recognition using AI, a ToF (Time of Flight) camera capable of detecting the three-dimensional position coordinates by capturing an image with a camera and measuring the distance to the object using the reflection of light (e.g., infrared light), or a stereo camera capable of detecting the three-dimensional position coordinates by capturing an image with two cameras. Note that the overall camera 52 may also be a camera capable of capturing a planar image including two-dimensional position coordinates. In this case, the data processing device 100 may be configured to estimate the three-dimensional position coordinates based on the planar image (two-dimensional position coordinates) captured by the overall camera 52.

[0039] The "treatment space" is not limited to a space including all of the objects included in the treatment space, but may also be a space including some of the objects included in the treatment space, such as the surgeon, patient, and treatment device 1 during treatment. Image data including the captured image obtained by the overall camera 52 (hereinafter also referred to as "overall image data") is acquired by the data processing device 100. Various settings of the overall camera 52, such as the frame rate and shutter speed, are adjusted in advance to an extent that the data processing device 100 can recognize the actions of the surgeon, such as the dentist or dental assistant, the actions of the patient, and the operation of the treatment device 1 during treatment.

[0040] The overall camera 52 may be installed anywhere as long as it can capture an image of at least an area including the examination space where patients are examined. The overall camera 52 may be a depth camera that can recognize the examination space in the depth direction in addition to recognizing the examination space in a plane. The overall camera 52 may be a 360-degree camera that can capture the examination space in all directions.

[0041] Patient camera 53 is positioned so as to capture video or still images of an area including at least the inside of the patient's mouth. Normally, during medical treatment, lighting device 19 is positioned in front of the patient's face, so patient camera 53 attached to lighting device 19 can naturally capture images of an area including the patient's face from the front.

[0042] Image data including the captured image obtained by the patient camera 53 (hereinafter also referred to as "patient image data") is acquired by the data processing device 100. Various settings of the patient camera 53, such as the frame rate and shutter speed, are adjusted in advance to an extent that the data processing device 100 can recognize the inside of the oral cavity of the patient during treatment.

[0043] The patient camera 53 may be installed anywhere as long as it can capture an image of an area including at least the inside of the patient's oral cavity. The patient camera 53 may be capable of detecting three-dimensional position coordinates, similar to the whole view camera 52, so that the data processing device 100 can detect the positional relationship between the area around the oral cavity and the medical instruments in the depth direction (for example, the direction in which the patient is viewed from the patient camera 53).

[0044] The rear camera 54 is positioned so as to capture video or still images of an area including at least the patient sitting on the chair 11 from the back of the chair 11 of the medical treatment device 1. Specifically, the rear camera 54 is positioned so as to capture video or still images of a bird's-eye view from above of a wide area including at least the dentist, dental assistant, and patient in the treatment space during treatment using the medical treatment device 1. The rear camera 54 is a fixed camera attached to the wall behind the chair 11, so it always captures the same area in the treatment space from the same position.

[0045] The rear camera 54 is configured with a 3D camera or the like so as to detect the three-dimensional position coordinates of the surgeon and the patient. For example, the rear camera 54 may be configured with a camera capable of detecting the three-dimensional position coordinates of an object through image recognition using AI, a ToF camera capable of detecting the three-dimensional position coordinates by capturing an image with a camera and measuring the distance to the object using the reflection of light (e.g., infrared light), or a stereo camera capable of detecting the three-dimensional position coordinates by capturing an image with two cameras. The rear camera 54 may also be a camera capable of capturing a planar image including two-dimensional position coordinates. In this case, the data processing device 100 may be configured to estimate the three-dimensional position coordinates based on the planar image (two-dimensional position coordinates) captured by the rear camera 54.

[0046] Image data including the captured image obtained by the rear camera 54 (hereinafter also referred to as "rear image data") is acquired by the data processing device 100. Various settings of the rear camera 54, such as the frame rate and shutter speed, are adjusted in advance to an extent that the data processing device 100 can recognize the patient positioned in the chair 11 and the surroundings of the chair 11.

[0047] The above-mentioned tray image data, overall image data, patient image data, and rear surface image data are collectively referred to as “image data.” Note that the term “image data” may refer to all of the tray image data, overall image data, patient image data, and rear surface image data, or may refer to at least one of the tray image data, overall image data, patient image data, and rear surface image data.

[0048] The data processing device 100 detects the presence or absence and type of objects, such as medical instruments, included in the captured image mainly based on the tray image data and the patient image data. The data processing device 100 detects the positions of people, such as dentists, dental assistants, and patients, included in the captured image mainly based on the whole image data and the rear image data.

[0049] The data processing device 100 is equipped with a computer (a calculation device 102 described later) that estimates the details of medical treatment for a patient performed using the medical treatment device 1 based on the various acquired data. Specifically, the data processing device 100 acquires image data from a tray camera 51, a whole view camera 52, a patient camera 53, and a rear view camera 54 attached to the medical treatment device 1.

[0050] In the first embodiment, the display device 110 is connected to the data processing device 100, but the display device 110 is not limited to this, and operation tools (user interfaces) such as a keyboard and a mouse may also be connected to the data processing device 100.

[0051] [Image taken by the overall camera] Fig. 2 is a diagram for explaining an example of an image captured by the overall camera 52. As shown in Fig. 2, the overall camera 52 captures an image of an area that includes at least the treatment space. During treatment, a dental treatment is performed by an operator using treatment instruments, and the image captured by the overall camera 52 shows the behavior of the operator (dentist 3 or dental assistant 4) during treatment, the behavior of the patient 2, the types of treatment instruments being used, the positions of the treatment instruments, and the operation of the treatment device 1.

[0052] The data processing device 100 can detect the positions (e.g., X coordinate, Y coordinate, Z coordinate in a three-dimensional coordinate display) of the dentist 3, dental assistant 4, and patient 2 during treatment by analyzing the overall image data acquired by the overall camera 52 using image recognition or the like.

[0053] [Image taken by rear camera] Fig. 3 is a diagram illustrating an example of an image captured by the rear camera 54. As shown in Fig. 3, the rear camera 54 captures an image of an area including at least the patient 2 positioned on the chair 11 from the back of the chair 11. During treatment, the practitioner performs dental treatment using medical instruments, and the image captured by the rear camera 54 shows the actions of the practitioner (dentist 3 or dental assistant 4) during treatment, the actions of the patient 2, the types of medical instruments being used, the positions of the medical instruments, and the operation of the medical device 1.

[0054] The data processing device 100 can detect the positions (e.g., X coordinate, Y coordinate, Z coordinate in a three-dimensional coordinate display) of the dentist 3, dental assistant 4, and patient 2 during treatment by analyzing the rear image data acquired by the rear camera 54 using image recognition or the like.

[0055] [Internal structure of the data processing system] Fig. 4 is a block diagram showing the internal configuration of a data processing system 1000 according to embodiment 1. As shown in Fig. 4, the data processing system 1000 includes a plurality of cameras (a tray camera 51, an overall camera 52, a patient camera 53, and a rear camera 54), a medical examination device 1, and a data processing device 100.

[0056] The medical treatment device 1 includes a chair 11, an instrument control device 21, a display control device 22, a sound control device 32, and a basin unit 12. Inside the medical treatment device 1, the chair 11, the instrument control device 21, the display control device 22, the sound control device 32, and the basin unit 12 can each transmit and receive data via CAN (Controller Area Network) communication. The medical treatment device 1 can also transmit and receive data to and from the data processing device 100 via wired or wireless LAN (Local Area Network) communication, for example, WiFi communication. The medical treatment device 1 may also transmit and receive data to and from the data processing device 100 via communication via a USB (Universal Serial Bus) connection.

[0057] The chair 11 includes a headrest 11a, a backrest 11b, a seat 11c, and a footrest 11d, each of which is driven under the control of the chair control unit 111. Specifically, upon receiving control data based on the surgeon's operation received via the foot controller 16 or the operation panel 18, or control data from the data processing device 100, the chair control unit 111 raises or lowers the seat 11c and moves the headrest 11a, the backrest 11b, and the footrest 11d vertically or horizontally relative to the seat 11c based on the control data. When the headrest 11a, the backrest 11b, and the footrest 11d are positioned vertically relative to the seat 11c, the chair 11 is in an upright position. This allows the patient to sit in the chair 11 (non-examination position). When the headrest 11a, backrest 11b, and footrest 11d are positioned horizontally relative to the seat 11c, the chair 11 is inclined. As a result, the patient lies on his / her back in the chair 11 (treatment position). In this way, the chair control unit 111 drives the headrest 11a, backrest 11b, seat 11c, and footrest 11d to change the posture of the chair 11 (patient position).

[0058] The instrument control device 21 includes an instrument control unit 211. When the instrument control unit 211 receives control data based on the surgeon's operation accepted by the foot controller 16 or the operation panel 18, or control data from the data processing device 100, the instrument control unit 211 controls the drive or settings of the medical instrument 15 based on the control data. For example, when the surgeon operates a switch on the foot controller 16 for driving an air turbine handpiece, the instrument control device 21 rotates a cutting tool held in a head portion of the air turbine handpiece. For example, when the data processing device 100 outputs control data for controlling the rotation direction or rotation speed of the air handpiece, the instrument control device 21 sets the rotation direction or rotation speed of the air turbine handpiece based on the control data. In this way, the instrument control device 21 controls the drive of the medical instrument 15 based on the surgeon's operation of the foot controller 16 or the operation panel 18.

[0059] The display control device 22 includes a display control unit 221 and a panel control unit 222. When the display control unit 221 receives control data from the data processing device 100, it controls the display 17 based on the control data. When the panel control unit 222 receives control data based on the control of the data processing device 100, it controls the operation panel 18 based on the control data.

[0060] The sound control device 32 includes a sound control unit 321. When the sound control unit 321 receives control data from the data processing device 100, it controls the speaker 35 based on the control data.

[0061] Basin unit 12 includes basin control unit 121 and lighting control unit 122. Upon receiving control data from data processing device 100, basin control unit 121 controls the water supply and drainage in basin unit 12 based on the control data. Upon receiving control data from data processing device 100, lighting control unit 122 controls the lighting and extinguishing of lighting device 19 based on the control data.

[0062] Each of the chair control unit 111, instrument control unit 211, display control unit 221, panel control unit 222, sound control unit 321, basin control unit 121, and lighting control unit 122 described above is configured by a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), etc., which are mounted on a circuit board (not shown). Each of the chair control unit 111, instrument control unit 211, display control unit 221, panel control unit 222, sound control unit 321, basin control unit 121, and lighting control unit 122 may be provided in advance in the medical treatment device 1, or may be modularized so that they can be optionally attached to the medical treatment device 1.

[0063] In the CAN communication between the chair control unit 111, the appliance control unit 211, the display control unit 221, the panel control unit 222, the sound control unit 321, the basin control unit 121, and the lighting control unit 122, medical care-related data including log data in each control unit is converted into communication packets and communicated between the control units. Note that in the communication between the control units, the medical care-related data does not necessarily have to be converted into communication packets as long as the medical care-related data is transmitted and received.

[0064] The "clinical care-related data" includes at least one of past and present control data for at least one of the chair 11, basin unit 12, lighting device 19, instrument control device 21, display control device 22, medical instruments 15, foot controller 16, display 17, and operation panel 18 included in the medical care device 1. For example, as described above, the "clinical care-related data" includes log data indicating the operation and control history of each of the chair 11, basin unit 12, lighting device 19, instrument control device 21, display control device 22, medical instruments 15, foot controller 16, display 17, and operation panel 18 included in the medical care device 1. Note that the "clinical care-related data" is not limited to historical data of the operation and control of various devices, such as log data, but may also include real-time data of the operation and control of various devices. Furthermore, the "clinical care-related data" may include data regarding the current status of various devices. For example, the "clinical care-related data" may include data for identifying the current state (posture, position, etc.) of the chair 11.

[0065] The base unit 12 includes a storage unit 123 that stores medical-related data including log data within the medical device 1, and a communication unit 124 that outputs the medical-related data stored by the storage unit 123 to the data processing device 100 via communication.

[0066] The storage unit 123 collects and stores medical care-related data from each control unit by communicating via CAN with the chair control unit 111, the appliance control unit 211, the display control unit 221, the panel control unit 222, the sound control unit 321, the basin control unit 121, and the lighting control unit 122. The storage unit 123 may be configured with a memory such as a ROM or RAM mounted on a board (not shown), or may be configured with a non-volatile storage medium such as a memory card.

[0067] The communication unit 124 communicates with the data processing device 100 via wired or wireless LAN communication or USB connection. If the storage unit 123 is configured as a non-volatile storage medium such as a memory card, the data processing device 100 may acquire medical care-related data stored in the storage medium via the communication unit 124 of the medical care device 1 using the communication device 101. The data processing device 100 may also directly acquire medical care-related data flowing in CAN communication via a volatile storage medium that temporarily stores the data.

[0068] The data processing device 100 includes a communication device 101, a calculation device 102, a volatile storage unit 103, and a non-volatile storage unit 104.

[0069] The communication device 101 communicates with the medical treatment device 1, and can also communicate with each of the tray camera 51, overall camera 52, patient camera 53, and rear camera 54. The communication device 101 acquires image data from each camera by communicating with each camera via wired or wireless LAN communication or USB connection. Each camera captures video of the medical treatment space at a predetermined frame rate, thereby acquiring multiple still images per second. The image data of the multiple still images acquired by each camera is input into the data processing device 100 in chronological order via the communication device 101.

[0070] The communication device 101 may acquire image data from each camera in other formats. For example, the communication device 101 may acquire image data temporarily stored in a non-volatile storage medium such as a memory card removed from each camera. The communication with the medical examination device 1 and the communication with each camera may be performed by separate devices.

[0071] The communication device 101 acquires medical-related data from the medical device 1 by communicating with the medical device 1 via wired or wireless LAN communication or USB connection. The communication device 101 may acquire medical-related data from the medical device 1 in other formats. For example, as described above, if the storage unit 123 is configured as a non-volatile storage medium such as a memory card, the data processing device 100 may acquire medical-related data stored in the storage medium by the communication device 101 via the communication unit 124 of the medical device 1.

[0072] The arithmetic device 102 is a computing entity (computer) that executes various processes by executing various programs. The arithmetic device 102 is configured with, for example, a CPU, a field-programmable gate array (FPGA), and a graphics processing unit (GPU).

[0073] The arithmetic device 102 may be configured with at least one of a CPU, an FPGA, and a GPU, or may be configured with a CPU and an FPGA, an FPGA and a GPU, a CPU and a GPU, or a CPU, an FPGA, and a GPU. The arithmetic device 102 may also be configured with a processing circuitry.

[0074] The volatile storage unit 103 includes a volatile storage area (for example, a working area). For example, the volatile storage unit 103 is configured with a volatile memory device such as a dynamic random access memory (DRAM) or a static random access memory (SRAM). The volatile storage unit 103 temporarily stores program code, a work memory, and the like when the arithmetic unit 102 executes any program. Furthermore, the volatile storage unit 103 temporarily stores image data input from each camera via the communication device 101. When the power supply to the volatile storage unit 103 is cut off, the data stored in the volatile storage unit 103 is erased.

[0075] The nonvolatile storage unit 104 includes a nonvolatile storage area. For example, the nonvolatile storage unit 104 is configured with a nonvolatile memory device such as a ROM (Read Only Memory), a flash memory, a hard disk, or an SSD (Solid State Drive). Even if the power supply to the nonvolatile storage unit 104 is cut off, the data stored in the nonvolatile storage unit 104 is not erased but is retained.

[0076] The nonvolatile storage unit 104 is not limited to being included in the data processing device 100. For example, the nonvolatile storage unit 104 may be included in an in-hospital server communicatively connected to the medical device 1, or may be included in an external server installed outside the medical space where the medical device 1 is installed. Furthermore, the nonvolatile storage unit 104 may exist in the form of cloud computing, in which multiple data processing devices 100 provided in multiple medical devices 1 can communicate with each other. In this way, the image data and medical-related data acquired from multiple medical devices 1 can be uniformly accumulated and managed by the nonvolatile storage unit 104.

[0077] The non-volatile storage unit 104 stores an estimation model 141, an estimation program 142, and a data processing program 143.

[0078] The estimation model 141 includes, for example, a neural network and parameters (such as a judgment threshold and a weighting coefficient) used in processing in the neural network, and is used to estimate the details of a patient's medical treatment based on at least one of image data acquired from each camera and medical treatment-related data acquired from the medical treatment device 1. The estimation model 141 is optimized (adjusted) by performing machine learning based on at least one of the image data and the medical treatment-related data. Specifically, the estimation model 141 can improve the accuracy of estimating the details of a patient's medical treatment by optimizing the parameters used in processing in the neural network based on training data (for example, a set of image data and medical treatment-related data, and medical treatment details as correct data).

[0079] The process in which the arithmetic device 102 estimates medical details is also referred to as an "estimation process," and the process in which the arithmetic device 102 learns the estimation model 141 is also referred to as a "learning process." Furthermore, the estimation model 141 optimized by the learning process is also referred to as a "trained model." That is, in the first embodiment, the pre-learning estimation model and the trained estimation model 141 are collectively referred to as "estimation models," while the trained estimation model 141 is also referred to as a "trained model." Note that the image data and medical care-related data are also collectively referred to as "estimation data" because they are used in the medical care details estimation process.

[0080] The estimation program 142 is a program for executing estimation processing and learning processing by the arithmetic device 102. Note that the data processing device 100 may include a control program that generates control data for controlling each control object based on the estimation result obtained by the estimation processing, and outputs the generated control data to each control object.

[0081] The data processing program 143 is a program for executing data processing on the image data obtained by each camera using image data acquired from each camera. Specifically, the data processing program 143 includes source code for performing image recognition using AI. By executing the data processing program 143, the arithmetic device 102 detects the positions of the patient's eyes (one or both eyes), head, ears, and nose included in the captured image, and detects the position of the headrest 11a of the chair 11.

[0082] A display device 110 is connected to the data processing device 100. The display device 110 displays various images, such as images showing estimated medical treatment or procedure details. Note that, in addition to the display device 110, operation tools (user interfaces) such as a keyboard and a mouse may be connected to the data processing device 100, and the device may be configured so that a user such as an operator inputs data for learning processing. The display device 110 may be provided in the data processing device 100.

[0083] In the above example, the data processing device 100 is configured to be separate from the medical treatment device 1, but the data processing device 100 may be implemented as a small computer inside the medical treatment device 1 in the form of edge computing. In this case, the data processing device 100 may transmit and receive data to and from each of the chair 11, the instrument control device 21, the display control device 22, the sound control device 32, and the basin unit 12 included in the medical treatment device 1 via CAN communication.

[0084] [Location Data] The position data of the surgeon and the patient generated by the data processing device 100 will be described with reference to Fig. 5 and Fig. 6. Fig. 5 is a diagram for explaining an example of the position data acquired by the data processing device 100 according to the first embodiment. Fig. 6 is a diagram for explaining an example of the detection points included in the position data.

[0085] In the data processing device 100, the arithmetic device 102 generates position data for identifying the positions of people included in the captured image, such as the surgeon and the patient during medical treatment, based on the overall image data acquired by the overall camera 52. As shown in Fig. 5, the position data for each person generated by the arithmetic device 102 is accumulated and stored in the non-volatile storage unit 104.

[0086] Specifically, the data processing device 100 generates at least one piece of position data indicating the position of each person (dentist, dental assistant, and patient) included in the examination space in each piece of overall image data, based on each of at least one piece of overall image data input in chronological order. For example, if image data for 10 still images is input per second based on the frame rate, the data processing device 100 generates position data for each person based on one piece of image data at 0.1 second intervals. The data processing device 100 assigns identification information (e.g., ID) indicating the person and time information (e.g., timestamp) indicating the current time to the generated position data for each person.

[0087] The position data includes three-dimensional (X, Y, Z) position coordinates for each predetermined detection point on the person to be detected. As shown in FIGS. 5 and 6, for example, the detection points include the person's right wrist, right elbow, right shoulder, head, left shoulder, left elbow, left wrist, right ear, left ear, right eye, left eye, nose, mouth, hip joint, right knee, left knee, right ankle, and left ankle. In other words, at least one piece of position data generated by the data processing device 100 includes position data corresponding to each of multiple joints of the person included in the medical space. Note that the data processing device 100 may acquire position data for key points other than the detection points shown in FIG. 5 (e.g., eyes, nose, points midway between joints, etc.), or may acquire position data for only some of the detection points shown in FIG. 5.

[0088] As shown in Fig. 6, the position coordinates corresponding to the position data include, for example, the X coordinate and the Y coordinate when the upper left corner of the captured image obtained by the whole view camera 52 is set as the origin of the X axis and the Y axis. Note that, although Fig. 6 shows a planar image including only the X axis and the Y axis, the Z coordinate in the Z axis direction, which is the depth direction of the planar image, is also included in the position data.

[0089] When the data processing device 100 acquires a captured image such as that shown in Figure 6 from the panoramic camera 52, it identifies detection points such as the right wrist and right elbow of each person included in the treatment space through image recognition, and determines the position coordinates of each identified detection point.

[0090] For example, focusing on a certain person (dentist 3 in this example) shown in the captured image, the data processing device 100 identifies detection points P1 (right wrist), P2 (right elbow), P3 (right shoulder), P4 (head), P5 (left shoulder), P6 (left elbow), and P7 (left wrist), and identifies position information for each of the detection points P1 to P7. For example, for the position coordinates of P1, the data processing device 100 identifies X1 as the X coordinate, Y1 as the Y coordinate, and Z1 as the Z coordinate based on the captured image. The data processing device 100 stores the position coordinates of P1 thus identified in the non-volatile storage unit 104 as position data of P1.

[0091] Each person in the treatment space, such as the dentist, dental assistant, and patient, has a roughly fixed location (home position) where they will be when treatment begins. For example, in the direction of a line connecting the backrest 11b and headrest 11a of the chair 11 (Y-axis direction), a dentist is likely to be present in region R1, which includes a position from above the patient's head when the patient is lying supine (treatment position), allowing the dentist to peer into the patient's oral cavity. In the direction of a line connecting the backrest 11b and headrest 11a of the chair 11 (X-axis direction), a dental assistant is likely to be present in region R2, which includes a position next to the patient when the patient is lying supine (treatment position). In region R3, which includes the position of the chair 11, a patient is likely to be present.

[0092] When the data processing device 100 first generates position data for each detection point of a person included in region R1 based on the overall image data at the start of medical treatment, the data processing device 100 assigns "ID:001" to the generated position data as identification information corresponding to region R1. For example, as shown in FIG. 5, the data processing device 100 stores the position data for each detection point of a person included in region R1 in a data table assigned "ID:001." From then on, until the end of medical treatment, each time the data processing device 100 tracks a person included in region R1 and generates position data for each detection point of the person, it stores the generated position data in the data table assigned "ID:001."

[0093] When the data processing device 100 first generates position data for each detection point of a person included in region R2 based on the entire image data at the start of medical treatment, the data processing device 100 assigns "ID:002" to the generated position data as identification information corresponding to region R2. For example, as shown in FIG. 5, the data processing device 100 stores the position data for each detection point of a person included in region R2 in a data table assigned "ID:002." Thereafter, until the end of medical treatment, each time the data processing device 100 tracks a person included in region R2 and generates position data for each detection point of the person, the data processing device 100 stores the generated position data in the data table assigned "ID:002."

[0094] When the data processing device 100 first generates position data for each detection point of a person included in region R3 based on the overall image data at the start of medical treatment, the data processing device 100 assigns "ID:003" to the generated position data as identification information corresponding to region R3. For example, as shown in FIG. 5, the data processing device 100 stores the position data for each detection point of a person included in region R3 in a data table assigned "ID:003." From then on, until the end of medical treatment, each time the data processing device 100 tracks a person included in region R3 and generates position data for each detection point of the person, it stores the generated position data in the data table assigned "ID:003."

[0095] In this way, the data processing device 100 assigns identification information associated with the area in the treatment space where each person is habitually located to the position data of each detection point of the person included in that area. In a dental treatment scene, dentist 3 is habitually located in area R1, dental assistant 4 is habitually located in area R2, and patient 2 is habitually located in area R3. On the other hand, in a tartar removal treatment scene, dentist 3 is habitually not located in areas R1 and R2, but instead dental assistant 4 is located, and the patient is located in area 3. In this way, although the people located when viewing a certain area differ in different scenes in the treatment space, such as a dental treatment scene by dentist 3 and a tartar removal treatment scene by dental assistant 4, the area in which each person is habitually located is somewhat fixed. Therefore, even if the data processing device 100 assigns identification information to people present in each area as arbitrary people without clearly recognizing whether the people included in the treatment space are dentists 3, dental assistants 4, or patients, it is possible to recognize an arbitrary person as a specific person such as dentist 3 based on time series data (data showing the movement of an arbitrary person) using the machine-learned estimation model 141.

[0096] By storing the position data together with a timestamp in the nonvolatile storage unit 104, the data processing device 100 can reuse the position data used when newly performing machine learning when re-learning. Furthermore, even when multiple data processing devices 100 share the position data stored in the nonvolatile storage unit 104, the multiple data processing devices 100 can perform similar machine learning because the position data is associated with a timestamp.

[0097] In this way, the data processing device 100 acquires position data of each detection point for identifying the position of the surgeon and patient during medical treatment based on the overall image data acquired from the overall camera 52. The data processing device 100 associates the acquired position data with identification information and a timestamp, and accumulates and stores the data as overall image data in the non-volatile storage unit 104. Furthermore, the data processing device 100 can identify the movements of the person's hands, arms, head, waist, legs, etc. by virtually connecting the detection points with straight lines or the like.

[0098] The above-described method for identifying the position coordinates of the detection points is an example, and the data processing device 100 may identify the position coordinates of the detection points by other methods. Furthermore, the data processing device 100 may identify the position coordinates of key items related to a series of medical treatments, not just people related to the series of medical treatments. In this way, the data processing device 100 can also perform machine learning on the estimation model 141 using the position coordinates of the identified key items.

[0099] Note that when data processing device 100 newly generates position data of a person included in an area to which certain identification information is assigned, if the position data differs from the stored position data of the person, data processing device 100 may store the newly generated position data. That is, data processing device 100 may not store new position data unless the person's position changes, and may store new position data in non-volatile storage unit 104 on the condition that the person's position has changed. This makes it possible to minimize an increase in the amount of data stored in non-volatile storage unit 104.

[0100] 5 and 6 show an example in which the position data of the surgeon and the patient are detected based on the entire image data, but the case in which the position data of the surgeon and the patient are detected based on the rear image data acquired by the rear camera 54 is similar to the detection example shown in Fig. 5 and 6. Fig. 7 is a diagram for explaining an example of an area in the rear image data.

[0101] As shown in Fig. 7, in the rear image data acquired by the rear camera 54, as in the overall image data acquired by the overall camera 52, identification information is associated with each area included in the treatment space. For example, "ID:001" is associated with area R1, where a dentist is likely to be habitually positioned when treatment begins. "ID:002" is associated with area R2, where a dental assistant is likely to be habitually positioned when treatment begins. "ID:003" is associated with area R3, where a patient is likely to be habitually positioned when treatment begins.

[0102] When the data processing device 100 first generates position data for each detection point of a person included in region R1 based on rear image data at the start of medical treatment, it assigns "ID:001" to the generated position data as identification information corresponding to region R1. When the data processing device 100 first generates position data for each detection point of a person included in region R2 based on rear image data at the start of medical treatment, it assigns "ID:002" to the generated position data as identification information corresponding to region R2. When the data processing device 100 first generates position data for each detection point of a person included in region R3 based on rear image data at the start of medical treatment, it assigns "ID:003" to the generated position data as identification information corresponding to region R3.

[0103] Here, if a person to be photographed is in the blind spot of the overall camera 52 or if multiple people overlap within the shooting range of the overall camera 52, the data processing device 100 may not be able to identify some of the detection points. In this case, the data processing device 100 may generate position data for the some of the detection points based on image data acquired by a camera other than the overall camera 52, and may use the generated position data to complement the position data corresponding to the missing part of the detection points.

[0104] For example, if the data processing device 100 is unable to generate position data of a detection point of a part of a person included in area R1 based on the overall image data acquired by the overall camera 52 at a predetermined timing, it may generate position data of the detection point of the part of the person included in area R1 based on the rear image data acquired by the rear camera 54 at the same predetermined timing, and store the generated position data in the non-volatile memory unit 104 as position data of the detection point of the part in the overall image data.

[0105] In addition, if the data processing device 100 is unable to generate position data of a detection point of a part of a person included in area R1 based on the overall image data acquired by the overall camera 52 at a predetermined timing, the data processing device 100 may store the position data of the detection point of the part of the person included in area R1 based on the overall image data acquired by the overall camera 52 at a timing earlier than the predetermined timing in the non-volatile memory unit 104 as the position data of the detection point of the part at the predetermined timing.

[0106] [Example of estimated medical treatment content] An example of estimation of medical treatment details by the data processing device 100 will be described with reference to Fig. 8. Fig. 8 is a diagram for explaining an example of synchronization data for estimation. Fig. 8 shows an example of estimating root canal treatment using image data.

[0107] During medical treatment, one or more medical instruments placed on the tray 30 are photographed by the tray camera 51 at predetermined timings that depend on the shutter timing. When the tray image data is input from the tray camera 51, the data processing device 100 extracts feature amounts contained in the photographed image by image recognition.

[0108] For example, as shown in Fig. 8, the data processing device 100 estimates the presence, shape, type, position, etc. of medical instruments placed on the tray 30 at each predetermined synchronization timing (T1, T2, T3, ...), and stores data "0" in a memory area corresponding to a medical instrument present on the tray 30, and stores data "1" in a memory area corresponding to a medical instrument not present on the tray 30. In this way, as shown in Fig. 8, the data processing device 100 can obtain estimation data that can distinguish the presence or absence of a medical instrument at each synchronization timing.

[0109] During medical examination, the patient's oral cavity is photographed at predetermined timings that depend on the shutter timing by the patient camera 53. When the patient image data is input from the patient camera 53, the data processing device 100 extracts feature amounts contained in the photographed image by image recognition.

[0110] 8, the data processing device 100 estimates the position of the medical instrument in the patient's oral cavity at each predetermined synchronization timing (T1, T2, T3, ...), and stores data "0" in a memory area when the positional relationship between the medical instrument and the patient's lips or the medical instrument and the patient's cheek is not within a predetermined range, and stores data "1" in a memory area when both are within the predetermined range. In this way, the data processing device 100 can obtain estimation data that can identify the position of the medical instrument in the patient's oral cavity at each synchronization timing.

[0111] During medical treatment, the overall camera 52 captures images of the actions of the surgeon and patient, the operation of the medical treatment device 1, and the like, at predetermined timings that depend on the shutter timing. When the overall image data is input from the overall camera 52, the data processing device 100 extracts feature amounts contained in the image to generate position data of people such as the surgeon and patient. The data processing device 100 stores the generated position data in the non-volatile storage unit 104.

[0112] 8, the data processing device 100 stores position data to which identification information is assigned for each predetermined synchronization timing (T1, T2, T3, ...), thereby enabling the data processing device 100 to obtain estimation data that can identify the actions of the surgeon and the patient for each synchronization timing.

[0113] During medical treatment, the rear camera 54 captures images of the actions of the surgeon and patient, the operation of the medical treatment device 1, and the like, at predetermined timings that depend on the shutter timing. When rear image data is input from the rear camera 54, the data processing device 100 extracts feature amounts contained in the image to generate position data of people such as the surgeon and patient. The data processing device 100 stores the generated position data in the non-volatile storage unit 104.

[0114] 8, the data processing device 100 stores position data to which identification information is assigned for each predetermined synchronization timing (T1, T2, T3, ...), thereby enabling the data processing device 100 to obtain estimation data that can identify the actions of the surgeon and the patient for each synchronization timing.

[0115] The data processing device 100 can estimate the details of medical care or treatment by finding features for estimating the details of medical care or treatment based on the estimation data obtained as described above.

[0116] [Functional configuration of data processing device] Fig. 9 is a block diagram showing the functional configuration of the data processing device 100 according to embodiment 1. As shown in Fig. 9, the data processing device 100 includes, as main functional units, an input unit 1101, a data processing unit 1102, a volatile storage unit 103, and a non-volatile storage unit 104.

[0117] The input unit 1101 is a functional unit of the communication device 101, and the data processing unit 1102 is a functional unit of the arithmetic device 102. The volatile storage unit 103 is an example of a "first storage unit" and, as described above, is configured with a volatile memory device such as RAM. The non-volatile storage unit 104 is an example of a "second storage unit" and, as described above, is configured with a non-volatile memory device such as ROM or flash memory.

[0118] The input unit 1101 acquires image data obtained by imaging by the imaging unit 501. The imaging unit 501 is a functional unit of each camera, such as the overall camera 52, the tray camera 51, the overall camera 52, the patient camera 53, and the rear camera 54. The imaging unit 501 images the patient in chronological order, and at least one piece of image data captured in chronological order from the imaging unit 501 is input to the input unit 1101 in chronological order.

[0119] The data processing unit 1102 generates position data indicating the position of a person included in the medical space based on at least one image data input in chronological order to the input unit 1101, and accumulates and stores the generated position data in the non-volatile memory unit 104.

[0120] Specifically, when the data processing unit 1102 acquires one piece of image data obtained at each predetermined timing that depends on the shutter timing from the input unit 1101, the data processing unit 1102 stores the image data in the volatile storage unit 103. At this time, if past image data has already been stored in the volatile storage unit 103, the data processing unit 1102 overwrites the past image data and stores the newly acquired latest image data in the volatile storage unit 103. As a result, only the latest single piece of image data is always stored in the volatile storage unit 103.

[0121] The data processing unit 1102 generates position data indicating the positions of people included in the medical space based on the latest image data stored in the volatile storage unit 103. Specifically, as described with reference to Figures 5 and 6, the data processing unit 1102 identifies detection points of people present in each of the regions R1, R2, and R3 based on the image data stored in the volatile storage unit 103, and generates position data of the detection points.

[0122] When the data processing unit 1102 generates position data, it stores the position data in the non-volatile storage unit 104. At this time, when the data processing unit 1102 generates position data based on image data acquired for the first time since the start of medical treatment, it assigns identification information corresponding to each of the regions R1, R2, and R3 to the generated position data. For example, as shown in FIG. 5, the data processing unit 100 stores position data of the detection points of people included in each of the regions R1, R2, and R3 in a data table to which identification information (ID) corresponding to each of the regions R1, R2, and R3 is assigned. Thereafter, until the end of medical treatment, the data processing unit 100 accumulates and stores the generated position data in a storage area (data table) of the non-volatile storage unit 104 to which identification information corresponding to each of the regions R1, R2, and R3 is assigned.

[0123] In this way, the data processing unit 1102 always stores only the most recent image data in the volatile memory unit 103, while accumulating and storing position data in the non-volatile memory unit 104 without storing image data.

[0124] The data processing unit 1102 generates position data for image data acquired not only by the overall camera 52 but also by the rear camera 54 and the like, and accumulates and stores the data in the non-volatile memory unit 104, but the volatile memory unit 103 stores only the most recent image data acquired by each camera.

[0125] Image data during medical treatment stored in the volatile storage unit 103 is erased when power supply to the volatile storage unit 103 is cut off. As a result, while image data is acquired in chronological order during medical treatment and position data is generated based on the image data, only the most recent image data is always stored in the volatile storage unit 103. Furthermore, when the power supply to the data processing device 100 is cut off after the medical treatment is completed, the most recent image data stored in the volatile storage unit 103 is also erased. On the other hand, even if the power supply to the data processing device 100 is cut off after the medical treatment is completed, the position data generated during the medical treatment is accumulated and held in the non-volatile storage unit 104. As a result, the data processing device 100 can accumulate and leave only the position data used to infer the medical treatment content or treatment content in the estimation model 141 in the non-volatile storage unit 104 while appropriately protecting the privacy of the photographed person.

[0126] [Determining Identification Information] As described above, when the data processing device 100 generates position data of each detection point of a person based on the image data acquired for the first time after the start of medical treatment, it assigns identification information corresponding to the area including the person to the position data and stores the data in the non-volatile storage unit 104. Thereafter, the data processing device 100 tracks each detection point of the person, and each time it generates position data, it stores the newly generated position data in the same data table as the initial position data to which identification information has already been assigned.

[0127] Here, when the data processing device 100 tracks the position of a person based on image data, the position of the person is tracked using a two-dimensional image such as that shown in Fig. 6. Therefore, if multiple people intersect in the depth direction (Z-axis direction) in one area, a situation may occur in which the position data of different people are stored in a data table with the same identification information. In particular, since there are multiple detection points corresponding to multiple joints in the same person and the data processing device 100 assigns identification information to each of the multiple position data generated for each of the multiple detection points, there is a risk that incorrect identification information will be assigned to a detection point where multiple people are likely to intersect, such as a wrist.

[0128] For example, the position data of dentist 3 in region R1 is stored in a data table assigned "ID:001." However, dental assistant 4 may enter region R1 and cross paths with dentist 3 in the Z-axis direction. In this case, the data processing device 100 generates position data for dental assistant 4 in region R1 and stores the generated position data in the data table assigned "ID:001." If incorrect identification information is assigned to the position data in this way, the data processing device 100 may not be able to accurately estimate the medical treatment or procedure using the position data, which may reduce the accuracy of predicting the medical treatment. Therefore, the data processing device 100 is configured to determine whether the identification information assigned to the position data is incorrect, and, if the identification information assigned to the position data is incorrect, to correct the identification information assigned to the position data.

[0129] The data processing device 100 cannot correct the identification information based on multiple captured images that change over time because the data processing device 100 always stores only one latest image data in the volatile storage unit 103. Therefore, the data processing device 100 determines whether the identification information assigned to the position data is incorrect based on the distance between the position of the person at a first timing and the position of the person at a second timing that follows the first timing.

[0130] Specifically, the data processing device 100 determines the distance between the newly generated latest position data and the previous position data (e.g., position data generated based on the first image data acquired since the start of medical treatment) to which the same identification information as the newly generated position data is assigned. For example, for detection point P1 corresponding to the right wrist assigned "ID:001," the data processing device 100 compares the position corresponding to the newly generated position data with the position corresponding to the previously generated position data to calculate the distance between them. If the calculated distance exceeds a predetermined threshold, the data processing device 100 determines that the identification information assigned to the newly generated position data is incorrect.

[0131] Note that, taking into consideration the timing at which image data is input in chronological order based on a predetermined frame rate, the threshold value may be set to the limit of the distance that a person can move in the time between the first and second times. For example, for detection point P14 corresponding to a hip joint, data processing device 100 may determine that the threshold has been exceeded if the hip joint has moved 20 cm or more in the time between the first and second times (for example, 0.1 seconds). Such a threshold value may be set in advance for each of detection points P1 to P18.

[0132] Alternatively, the data processing device 100 may determine whether the identification information assigned to the position data is incorrect based on the degree to which the orientation of the person at a first timing differs from the orientation of the person at a second timing after the first timing.

[0133] Specifically, the data processing device 100 determines the difference in the person's orientation between the newly generated latest position data and the previous position data (e.g., position data generated based on the first image data acquired since the start of medical treatment) to which the same identification information as the newly generated latest position data is assigned. For example, for detection point P1 corresponding to the right wrist assigned "ID:001," the data processing device 100 compares the position corresponding to the newly generated position data with the position corresponding to the previously generated position data, and calculates the degree of difference in orientation between the two. If the calculated degree of difference in orientation exceeds a predetermined threshold, the data processing device 100 determines that the identification information assigned to the newly generated position data is incorrect.

[0134] Note that the threshold value may be set to the limit of the direction that the person can change to between the first and second timings, taking into consideration the timing at which image data is input in chronological order based on a predetermined frame rate. For example, for detection point P10 corresponding to the right eye, data processing device 100 may determine that the threshold value has been exceeded if the right eye moves from a position of 10 cm in the +X-axis direction to a position of 10 cm in the -X-axis direction between the first and second timings (for example, 0.1 seconds). Such a threshold value may be set in advance for each of detection points P1 to P18.

[0135] If the data processing device 100 determines that the identification information assigned to the position data is incorrect, it corrects the identification information assigned to the position data. For example, if the "ID:001" assigned to the newly generated position data of the detection point P1 corresponding to the right wrist is incorrect, the data processing device 100 moves the position data assigned with the incorrect identification information to a data table assigned with the correct "ID:002." In this case, for example, the data processing device 100 may identify the correct identification information by analyzing time-series changes in the position data assigned with the identification information ("ID:002" and "ID:003") corresponding to regions (R2 and R3) other than the region R1 corresponding to "ID:001." Note that if the "ID:001" assigned to the newly generated position data of the detection point P1 corresponding to the right wrist is incorrect, the data processing device 100 may delete the position data assigned with the incorrect identification information from the data table assigned with "ID:001."

[0136] In this way, the data processing device 100 is configured to correct the identification information assigned to the location data if the identification information assigned to the location data is incorrect, so that the positions of people included in the medical space can be properly recorded.

[0137] [Data processing by data processing device] The data processing executed by the data processing device 100 according to the first embodiment will be described with reference to Fig. 10. Fig. 10 is a flowchart for explaining an example of the data processing executed by the data processing device 100 according to the first embodiment. Each step (hereinafter, indicated by "S") shown in Fig. 10 is realized by the arithmetic device 102 of the data processing device 100 executing the data processing program 143. Note that the data processing device 100 executes data processing on each of the overall image data acquired by the overall camera 52 and the rear image data acquired by the rear camera 54.

[0138] 10, after the data processing device 100 is started by power supply, the data processing device 100 determines whether or not a start estimation condition for estimating the start of medical treatment is satisfied (S1). The start estimation condition is a condition that, when satisfied, allows estimation that medical treatment has started, and the data processing device 100 executes the processes from S2 onwards.

[0139] The start inference conditions include at least one of the following: the chair 11 has moved from the non-treatment position to the treatment position; the operation of the treatment tools has started; the lighting device 19 has turned on; and the basin unit 12 has started to operate. The operation of each of these components included in the treatment device 1 can be an event that can be used to infer that treatment has started. For example, when treatment starts, the chair 11 moves from the non-treatment position to the treatment position with a patient sitting in it, the treatment tools 15 start to operate as operated by the dentist, the lighting device 19 has turned on, and the basin unit 12 has started to dispense water into cups. The data processing device 100 can recognize the operation of each of the components of the treatment device 1 described above based on the treatment-related data acquired via the communication device 101.

[0140] If the initiation estimation condition is not satisfied (NO in S1), the data processing device 100 determines that medical treatment has not yet started and repeats the process of S1 until the initiation estimation condition is satisfied. On the other hand, if the initiation estimation condition is satisfied (YES in S1), the data processing device 100 acquires image data input in time series from each camera (S2).

[0141] The data processing device 100 stores the acquired image data in the volatile storage unit 103 (S3). The data processing device 100 generates position data indicating the positions of people included in the medical space based on the image data stored in the volatile storage unit 103 (S4). The processing of S3 is an example of a "generation process."

[0142] The data processing device 100 determines whether the image data used to generate the position data is the image data that has been input for the first time since the medical treatment started (the start estimation condition of S1 is met) (S5). For example, the data processing device 100 determines whether the position data has been generated based on the image data that has been acquired for the first time by the whole view camera 52 since the medical treatment started.

[0143] If the image data used to generate the position data is the first image data input since the start of medical treatment (the start estimation condition of S1 is met) (YES in S5), the data processing device 100 assigns identification information to the generated position data (S6). At this time, the data processing device 100 assigns identification information corresponding to the area containing the person corresponding to the generated position data to the generated position data. The processing of S6 is an example of an "assignment process."

[0144] The data processing device 100 stores the position data to which the identification information has been added in the nonvolatile storage unit 104 (S10). Such processing of S10 is an example of a "second storage processing."

[0145] The data processing device 100 determines whether or not a termination estimation condition for estimating the end of medical care is satisfied (S11). The termination estimation condition is a condition that, when satisfied, allows estimation that medical care has been terminated.

[0146] The end inference conditions include at least one of the following: the chair 11 has moved from the treatment position to the non-treatment position; the operation of the medical instruments has ended; the lighting device 19 has turned off; and the operation of the basin unit 12 has ended. The operation of each component included in these medical device 1 can be an event that can be used to infer that the medical treatment has ended. For example, when the medical treatment ends, the chair 11 moves from the treatment position to the non-treatment position, the operation of the medical instruments 15 has ended, the lighting device 19 has turned off, and the basin unit 12 has stopped dispensing water into the cups. The data processing device 100 can recognize the operation of each component of the medical device 1 described above based on the medical treatment-related data acquired via the communication device 101.

[0147] If the termination estimation condition is met (YES in S11), the data processing device 100 terminates this data processing. On the other hand, if the termination estimation condition is not met (NO in S11), the data processing device 100 proceeds to the processing of S2 and executes data processing on the image data acquired for the second time.

[0148] Specifically, when the data processing device 100 acquires the image data input for the second time since the start of medical treatment (the estimated start condition of S1 is satisfied) in S2, it overwrites the image data acquired first since the start of medical treatment (the estimated start condition of S1 is satisfied) with the second image data acquired this time and stores it in the volatile storage unit 103 (S3). As a result, only one latest image data is stored in the volatile storage unit 103 at any one time. The processing of S3 is an example of a "first storage processing."

[0149] The data processing device 100 generates position data indicating the position of a person included in the treatment space based on the latest image data stored in the volatile memory unit 103 (S4), and determines whether the image data used to generate the position data is the first image data input since the treatment started (the start estimation condition of S1 is met) (S5).

[0150] If the image data used to generate the position data is not the first image data input since the start of medical treatment (the start estimation condition of S1 is satisfied) (NO in S5), the data processing device 100 determines whether the identification information assigned to the position data is incorrect based on the time-series change in the position data (S7). Such processing of S7 is an example of a "determination process."

[0151] For example, as described above, the data processing device 100 compares the position corresponding to the newly generated latest position data with the position corresponding to the previous position data to which the same identification information as that of the position data is assigned (for example, position data generated based on image data acquired for the first time since the start of medical treatment), and calculates the distance between them. If the calculated distance exceeds a predetermined threshold, the data processing device 100 determines that the identification information assigned to the newly generated position data is incorrect.

[0152] If the identification information assigned to the newly generated latest location data is incorrect (YES in S8), the data processing device 100 corrects the identification information assigned to the newly generated latest location data (S9). The process of S9 is an example of a "correction process." Thereafter, the data processing device 100 accumulates and stores the location data with the corrected identification information in the non-volatile storage unit 104 (S10).

[0153] Furthermore, if the identification information assigned to the newly generated latest location data is not incorrect (NO in S8), the data processing device 100 accumulates and stores the newly generated latest location data in the non-volatile memory unit 104 without correcting the identification information (S10).

[0154] As described above, when generating position data based on image data acquired in time series during medical treatment, the data processing device 100 always stores only the most recent single image data in the volatile storage unit 103, so that only the single image data used for generating the position data can be left in the volatile storage unit 103. Furthermore, when the power supply to the data processing device 100 is turned off after the medical treatment is completed and power supply to the volatile storage unit 103 is stopped, the most recent single image data stored in the volatile storage unit 103 is also erased, so that the privacy of the photographed person can be appropriately protected. Meanwhile, the data processing device 100 accumulates and stores position data generated during medical treatment in the non-volatile storage unit 104, so that only the position data used to estimate the medical treatment content or treatment content in the estimation model 141 can be accumulated and left in the non-volatile storage unit 104.

[0155] Furthermore, each time the data processing device 100 generates position data, it determines whether the identification information assigned to the newly generated position data is incorrect, and if the identification information assigned to the newly generated position data is incorrect, it corrects the identification information assigned to the newly generated position data, thereby enabling the positions of people included in the medical space to be properly recorded.

[0156] <Embodiment 2> A data processing device 200 according to a second embodiment of the present disclosure will be described in detail with reference to the drawings. Note that, for the data processing device 200 according to the second embodiment, the portions different from the data processing device 100 according to the first embodiment will be mainly described, and the portions that are the same as those in the data processing device 100 according to the first embodiment will be assigned the same reference numerals and will not be described repeatedly.

[0157] [Functional configuration of data processing device] Fig. 11 is a block diagram showing the functional configuration of a data processing device 200 according to embodiment 2. As shown in Fig. 11, the data processing device 200 includes, as main functional units, an input unit 1101, a data processing unit 2102, a first volatile storage unit 203, a non-volatile storage unit 204, and a second volatile storage unit 205.

[0158] The data processing unit 2102 is a functional unit of the arithmetic device 102. The first volatile storage unit 203 is an example of a "first storage unit" and is configured with a volatile memory device such as a RAM, similar to the volatile storage unit 103 according to the first embodiment. The non-volatile storage unit 204 is an example of a "second storage unit" and is configured with a non-volatile memory device such as a ROM or flash memory, similar to the non-volatile storage unit 204 according to the first embodiment. Furthermore, the second volatile storage unit 205 is an example of a "third storage unit" and is configured with a volatile memory device such as a RAM, similar to the volatile storage unit 103 according to the first embodiment. Note that the second volatile storage unit 205 may be a volatile memory device physically different from the first volatile storage unit 203, or may be included in the same volatile memory device as the first volatile storage unit 203. For example, a storage area within one volatile memory device may be divided into a storage area corresponding to the first volatile storage unit 203 and a storage area corresponding to the second volatile storage unit 205.

[0159] When the data processing unit 1102 acquires one piece of image data obtained at each predetermined timing that depends on the shutter timing via the input unit 1101, the data processing unit 1102 stores the image data in the first volatile storage unit 203. At this time, if past image data has already been stored in the first volatile storage unit 203, the data processing unit 2102 overwrites the past image data and stores the latest image data acquired this time in the first volatile storage unit 203. In this way, only the latest single piece of image data is always stored in the first volatile storage unit 203.

[0160] The data processing unit 2102 identifies the detection points of people present in each of the regions R1, R2, and R3 based on the latest image data stored in the first volatile storage unit 203, and generates position data of the detection points.

[0161] When the data processing unit 2102 generates the position data, it accumulates and stores the position data in the second volatile storage unit 205. At this time, when the data processing unit 2102 generates the position data based on the image data acquired for the first time after the start of medical treatment, it assigns identification information corresponding to each of the regions R1, R2, and R3 to the generated position data.

[0162] When the medical examination is completed, the data processing unit 2102 copies the position data accumulated and stored in the second volatile storage unit 205 to the non-volatile storage unit 204.

[0163] Image data during medical examination stored in the first volatile storage unit 203 is erased when power supply to the first volatile storage unit 203 is cut off. In addition, position data accumulated and stored in the second volatile storage unit 205 is erased when power supply to the second volatile storage unit 205 is cut off.

[0164] As a result, while image data is acquired in chronological order during medical treatment and position data is generated based on the image data, only the most recent single image data is always stored in the first volatile storage unit 203. Furthermore, after the medical treatment is completed, the data processing device 100 is powered off and power supply to the first volatile storage unit 203 is cut off, thereby erasing the most recent single image data stored in the first volatile storage unit 203. On the other hand, when the medical treatment is completed, the position data accumulated and stored in the second volatile storage unit 205 is stored in the non-volatile storage unit 204. Therefore, even if the data processing device 100 is powered off after the medical treatment is completed, the position data generated during the medical treatment is accumulated and held in the non-volatile storage unit 204. As a result, the data processing device 100 can accumulate and leave only the position data used to infer the medical treatment content or treatment content in the estimation model 141, while appropriately protecting the privacy of the photographed person.

[0165] Furthermore, since the non-volatile storage unit 104, such as a flash memory, generally has a slower data transfer speed than the second volatile storage unit 205, such as a RAM, storing position data in the non-volatile storage unit 104 every time position data is generated in chronological order during medical treatment, as in the data processing device 100 according to embodiment 1, places a heavy processing burden on the arithmetic device 102. However, the data processing device 200 according to embodiment 2 accumulates and stores position data in the second volatile storage unit 205 during medical treatment, and copies the position data accumulated and stored in the second volatile storage unit 205 to the non-volatile storage unit 204 after the medical treatment is completed, thereby reducing the processing burden of data transfer on the arithmetic device 102.

[0166] [Data processing by data processing device] The data processing executed by the data processing device 200 according to the second embodiment will be described with reference to Fig. 12. Fig. 12 is a flowchart for explaining an example of the data processing executed by the data processing device 200 according to the second embodiment. Each step (hereinafter, indicated by "S") shown in Fig. 12 is realized by the arithmetic device 102 of the data processing device 200 executing the data processing program 143.

[0167] 12, the data processing device 200 determines that the start estimation condition is met in the process of S1, and when it acquires image data input in time series from each camera in the process of S2, it stores the acquired image data in the first volatile storage unit 203 (S101). Thereafter, in the process of S4, the data processing device 200 generates position data indicating the positions of people included in the examination space based on the image data stored in the first volatile storage unit 203, and stores the generated position data in the second volatile storage unit 205 (S102). The process of S102 is an example of a "third storage process."

[0168] The data processing device 200 accumulates and stores the position data generated in time series in the second volatile memory unit 205 by repeating the above-mentioned processing until it determines in S11 that the termination estimation condition is met.

[0169] When the termination estimation condition is met (YES in S11), the data processing device 200 copies and stores the position data accumulated and stored in the second volatile storage unit 205 in the nonvolatile storage unit 204 (S103). Thereafter, the data processing device 200 terminates this data processing.

[0170] As described above, during medical treatment, the data processing device 200 accumulates and stores the position data generated in time series in the second volatile memory unit 205 without storing it in the non-volatile memory unit 204, and after the termination estimation condition is met, copies the position data accumulated and stored in the second volatile memory unit 205 to the non-volatile memory unit 204, thereby reducing the processing burden of data transfer to the calculation device 102.

[0171] <Modification> The present disclosure is not limited to the above-described embodiments, and various modifications and applications are possible. Modifications applicable to the present disclosure will be described below.

[0172] [About location detection] As shown in Figures 5 to 7, the data processing device 100 preliminarily sets regions R1, R2, and R3 in the treatment space, and assigns identification information associated with the region containing the person corresponding to the position data to position data generated based on image data. However, the data processing device 100 is not limited to this assignment process. It may also identify people such as dentists, dental assistants, and patients from the position data generated based on image data through image recognition using AI (estimation model 141), and assign identification information corresponding to the people to the position data based on the identification results. In dental treatment, there are certain rules for the positions and postures of dentists, dental assistants, and patients during treatment, so it is possible to identify people based on position data using AI.

[0173] In addition, since each person in the treatment space, such as a dentist, dental assistant, or patient, generally wears a uniform of a fixed color or style (for example, a dentist usually wears a white coat), the data processing device 100 may identify whether each person is a dentist, dental assistant, or patient based on the clothing of each of the multiple people shown in the captured image of the image data.

[0174] [About the data processing device] The data processing device 100 may be provided in the medical device 1 by being included in the basin unit 12. Alternatively, the data processing device 100 may be a computer, such as an in-hospital server, separate from the medical device 1 and communicatively connected to the medical device 1. Furthermore, the data processing device 100 may exist in the form of a cloud computing system, such as an external server installed outside the medical space where the medical device 1 is installed. In this case, the data processing device 100 may be connected to multiple medical devices 1 installed in the medical space and also to multiple medical devices 1 installed in other dental clinics, and may generate control data for each of these multiple medical devices 1 based on image data and medical-related data. In this way, the frequency of machine learning by the data processing device 100 may be further increased, allowing the data processing device 100 to generate control data with higher accuracy.

[0175] The embodiments disclosed herein should be considered to be illustrative in all respects and not restrictive. The scope of the present disclosure is defined by the claims, not by the above description, and is intended to include all modifications within the meaning and scope of the claims. Note that the configurations exemplified in the embodiments and the configurations exemplified in the modified examples can be combined as appropriate. [Explanation of symbols]

[0176] 1 medical device, 2 patient, 3 dentist, 4 dental assistant, 5 pole, 6 arm, 11 chair, 11a headrest, 11b backrest, 11c seat, 11d footrest, 12 basin unit, 12a bowl, 12b cup stand, 12c water tap, 13 tray table, 14 instrument holder, 15 medical device, 16 foot controller, 17 display, 18 operation panel, 19 lighting device, 21 instrument control device, 22 display control device, 30 tray, 32 sound control device, 35 speaker, 51 tray camera, 52 overall camera, 53 patient camera, 54 rear camera, 100, 200 data processing device, 101 communication device, 102 calculation device, 103 volatile memory unit, 104, 204 non-volatile memory unit, 110 display device, 111 Chair control unit, 121 basin control unit, 122 lighting control unit, 123 storage unit, 124 communication unit, 141 estimation model, 142 estimation program, 143 data processing program, 203 first volatile memory unit, 205 second volatile memory unit, 211 appliance control unit, 221 display control unit, 222 panel control unit, 321 sound control unit, 501 photography unit, 1000 data processing system, 1101 input unit, 1102, 2102 data processing unit.

Claims

1. A data processing device that processes at least one image data obtained by photographing a medical space, an input unit to which the at least one image data is input in time series; a data processing unit for processing the at least one image data; The data processing unit a generation process for generating position data indicating the position of a person included in the medical space based on the at least one image data input to the input unit in time series; an assignment process of assigning identification information indicating any one of the persons to the location data; a determination process for determining whether the identification information assigned to the location data is incorrect based on a time series change in the location data; and if the identification information assigned to the position data is incorrect, executing a correction process to correct the identification information assigned to the position data.

2. the at least one image data is data of a captured image obtained by capturing an image of the examination space with a fixed camera; the person includes at least one of a doctor, an assistant, and a patient; the identification information is associated with an area of ​​the medical space in which the person is habitually located; The data processing device according to claim 1 , wherein the data processing unit, in the assignment process, assigns the identification information associated with an area of ​​the medical space in which the person is included to the position data indicating the position of the person included in the medical space.

3. the person includes at least one of a doctor, an assistant, and a patient; 2. The data processing device according to claim 1, wherein, in the assignment process, the data processing unit identifies the person based on characteristics of the person included in the medical space, and assigns the identification information associated with the person to the position data indicating the position of the person included in the medical space based on the person identification result.

4. 4. The data processing device according to claim 1, wherein the data processing unit assigns the identification information to the position data corresponding to each of a plurality of joints of the person in the assignment process.

5. The data processing device according to claim 1 , wherein the time-series change in the position data includes a distance between the position of the person at a first timing and the position of the person at a second timing after the first timing.

6. The data processing device according to claim 1 , wherein the change in the time series of the position data includes a degree of difference between an orientation of the person at a first timing and an orientation of the person at a second timing after the first timing.

7. The data processing device according to claim 1 , wherein the data processing unit executes the assigning process, the determining process, and the correcting process when a start estimation condition under which a start of medical treatment is estimated is satisfied.

8. The data processing device according to claim 7, wherein the start estimation condition includes at least one of the following: a chair equipped in a medical device included in the medical space has moved from a non-treatment position to a treatment position; operation of a medical instrument equipped in the medical device has started; a lighting device equipped in the medical device has turned on; and operation of a water supply / drainage device equipped in the medical device has started.

9. A data processing method in which a computer processes at least one image data obtained by photographing a medical space, comprising: The process executed by the computer is inputting the at least one image data in time series; generating position data indicating a position of a person included in the examination space based on the at least one image data input in time series; adding identification information indicating any one of the persons to the location data; determining whether the identification information assigned to the location data is incorrect based on a time series change in the location data; and correcting the identification information assigned to the location data if the identification information assigned to the location data is incorrect.

10. A data processing program for processing at least one image data obtained by photographing a medical space, The data processing program is installed on a computer. inputting the at least one image data in time series; generating position data indicating a position of a person included in the examination space based on the at least one image data input in time series; adding identification information indicating any one of the persons to the location data; determining whether the identification information assigned to the location data is incorrect based on a time series change in the location data; and correcting the identification information assigned to the location data if the identification information assigned to the location data is incorrect.

Images