Medical display system, control method, and control device

Abstract

The present technology relates to a medical display system, a control method, and a control device which make it possible to prevent an unintended operation. Provided is a medical display system comprising: a display unit for providing a display based on information output from a medical apparatus; an imaging unit for imaging, as an imaging area, a part of an area from which at least the display unit is observable; a speech acquiring unit for acquiring speech in the area from which at least the display unit is observable; and a control unit for controlling display information on the basis of the information output from the medical apparatus. The control unit, when a first user registered in advance satisfies a predetermined condition in the imaging area, controls the display information on the basis of the speech of the first user or an input triggered by the speech, and, when the first user does not satisfy the predetermined condition in the imaging area, controls the display information on the basis of the speech of a second user different from the first user or an input triggered by the speech.

Description

[Technical field]

[0001] The present technology relates to a medical display system, a control method, and a control device, and in particular to a medical display system, a control method, and a control device that are capable of suppressing the execution of unintended operations. [Background technology]

[0002] Operating rooms are zoned into clean areas, unclean areas, etc. For example, surgeons and other practitioners perform surgery in the clean area. In this case, in order to keep their hands clean, surgeons in the clean area can only operate medical equipment that meets a certain cleanliness standard.

[0003] In addition, it is difficult to perform precise operations when using foot pedals or the like to operate medical equipment. Therefore, as disclosed in Patent Document 1, it has been proposed to operate medical equipment using voice input as well. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2018 / 173681 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in an operating room, not only the surgeon but also multiple medical personnel such as assistants, nurses, and clinical engineers participate in the operation. In addition, the surgeon may be replaced by another doctor during the operation. Therefore, when operating equipment using voice input, it is necessary to suppress the execution of unintended operations.

[0006] The present technology has been made in consideration of such circumstances, and makes it possible to suppress the execution of unintended operations. [Means for solving the problem]

[0007] A medical display system according to one aspect of the present technology includes a display unit that displays based on information output from a medical device, an imaging unit that images at least a portion of the observable area of ​​the display unit as an imaging area, an audio acquisition unit that acquires audio from at least the observable area of ​​the display unit, and a control unit that controls display information based on the information output from the medical device, wherein the control unit controls the display information based on the audio of the first user or an input triggered by the audio when a first user registered in advance satisfies a predetermined condition in the imaging area, and controls the display information based on the audio of a second user different from the first user or an input triggered by the audio when the first user does not satisfy the predetermined condition in the imaging area.

[0008] A control method of one aspect of the present technology is a control method in which, when a medical display system controls display information displayed on a display unit based on information output from a medical device, when a pre-registered first user satisfies a predetermined condition in an imaging area in which the display unit is imaged as at least a part of the observable area, the display information is controlled based on the voice of the first user acquired in at least the observable area of ​​the display unit or an input triggered by the voice, and when the first user does not satisfy the predetermined condition in the imaging area, the display information is controlled based on the voice of a second user different from the first user acquired in at least the observable area of ​​the display unit or an input triggered by the voice.

[0009] In a medical display system and control method according to one aspect of the present technology, when controlling display information displayed on a display unit based on information output from a medical device, when a pre-registered first user satisfies a predetermined condition in an imaging area in which the display unit is imaged as at least a part of the observable area, the display information is controlled based on the voice of the first user acquired in at least the observable area of ​​the display unit or input triggered by the voice, and when the first user does not satisfy the predetermined condition in the imaging area, the display information is controlled based on the voice of a second user different from the first user acquired in at least the observable area of ​​the display unit or input triggered by the voice.

[0010] A control device of one aspect of the present technology includes a control unit that controls display information displayed on a display unit based on information output from a medical device, and the control unit is a control device that controls the display information based on the voice of the first user acquired in at least an observable area of ​​the display unit or an input triggered by the voice when a pre-registered first user satisfies a predetermined condition in an imaging area in which the display unit is imaged as at least a part of the observable area, and controls the display information based on the voice of a second user different from the first user acquired in at least an observable area of ​​the display unit or an input triggered by the voice when the first user does not satisfy the predetermined condition in the imaging area.

[0011] In a control device of one aspect of the present technology, display information displayed on a display unit is controlled based on information output from a medical device, and when a pre-registered first user satisfies a predetermined condition in an imaging area in which the display unit is imaged as at least a part of the observable area, the display information is controlled based on the voice of the first user acquired in at least the observable area of ​​the display unit or an input triggered by the voice, and when the first user does not satisfy the predetermined condition in the imaging area, the display information is controlled based on the voice of a second user different from the first user acquired in at least the observable area of ​​the display unit or an input triggered by the voice.

[0012] The control device according to one aspect of the present technology may be an independent device, or may be an internal block constituting a single device. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of an embodiment of a medical display system. [Diagram 2] FIG. 2 is a diagram illustrating an example of input / output and processing in a control device. [Diagram 3] 11A and 11B are diagrams illustrating an example of a voice operation when there are a plurality of user candidates. [Figure 4] 11 is a flowchart illustrating the flow of a user determination process when a voice is input. [Diagram 5] FIG. 13 is a diagram showing an example of operation authority for each category. [Figure 6] FIG. 13 is a diagram illustrating an example of a doctor table. [Figure 7] FIG. 13 is a diagram illustrating an example of a preoperative registration table. [Figure 8] 13A and 13B are diagrams showing specific examples of user determination during surgery. [Figure 9] 13 is a flowchart illustrating a flow of a voice response process. [Figure 10] 13 is a flowchart illustrating a flow of gaze corresponding processing. [Figure 11]13 is a flowchart illustrating the flow of a user determination process according to rank. [Figure 12] 13 is a flowchart illustrating the flow of an authority transfer process. [Figure 13] 13 is a flowchart illustrating the flow of an authority transfer release process. [Figure 14] 13 is a flowchart illustrating the flow of a user exclusion process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] <1. First embodiment>

[0015] (System Configuration) FIG. 1 shows a state of surgery using a medical display system according to an embodiment of the present technology.

[0016] In FIG. 1, the medical display system 1 comprises a control device 10, a microscope device 20, a monitoring device 30, and a display device 40.

[0017] In the example of FIG. 1, a medical display system 1 configured with such devices is used to display a patient P on a patient bed to a user U as a surgeon. A The figure shows a patient P performing surgery. Around the patient P on the patient bed, user candidates U, such as assistants and nurses, are B ,U C exists.

[0018] In the following description, a "user" refers to a person, such as a surgeon, who has operation authority based on voice input. Also, a "user candidate" refers to any surgical staff (surgery participants) who use the medical display system 1, including medical professionals such as surgeons, assistants, nurses, and clinical engineers. In other words, among the user candidates as surgical staff, there are users who have operation authority based on voice input.

[0019] The control device 10 is a device capable of controlling connected devices such as medical equipment and integrating information output from the connected devices, and is, for example, a CCU (Camera Control Unit), etc. The control device 10 may be connected to a network and configured to be capable of communicating with external devices such as a server and a PC (Personal Computer).

[0020] The microscope device 20 is an electronic imaging type microscope device (a so-called video type microscope device). The microscope device 20 captures an image of the operative site of the patient P, and outputs a signal representing an operative field image showing the operative field to the control device 10. Note that, instead of the microscope device 20, a medical imaging device such as an endoscope may be used as a device for capturing the operative field image.

[0021] The monitoring device 30 monitors the biological information of the patient P and generates monitoring information representing the monitoring results of the biological information. The biological information of the patient P includes the heart rate, electrocardiogram, blood oxygen saturation, arterial blood pressure, etc. The monitoring device 30 outputs a signal representing the monitoring information to the control device 10.

[0022] The display device 40 is a device that displays information output from the control device 10, and is, for example, a liquid crystal display or an EL (Electro Luminescence) display. The display device 40 is provided near a user or a potential user, such as on a wall of an operating room. The display device 40 displays various types of information related to the surgery, such as biological information, physical information, and information on the surgical procedure of the patient P, together with an image of the surgical field captured by the microscope device 20.

[0023] FIG. 2 shows an example of input / output and processing in the control device 10 of FIG.

[0024] As shown in FIG. 2, signals from a microphone 50, a camera 60, and a gaze detector 70 are input to the control device 10.

[0025] The microphone 50 is a device capable of detecting the voice of a candidate user such as a surgeon, and is, for example, an array microphone. The microphone 50 outputs to the control device 10 a signal (voice signal) representing the voice uttered by the candidate user.

[0026] The camera 60 is a device that captures an image of a user candidate such as a surgeon, and is, for example, a camera in an operating room. The camera 60 outputs to the control device 10 a signal (image signal) that represents a captured image including the user candidate.

[0027] The gaze detector 70 is a device for detecting the gaze of a candidate user such as a surgeon, and is, for example, an IR (infrared) camera. The gaze detector 70 outputs a signal (detection signal) representing the gaze of the candidate user to the control device 10.

[0028] The microphone 50, the camera 60, and the gaze detector 70 may be mounted on the display device 40, or may be configured as independent devices and connected to the control device 10. Note that, although the example of Fig. 2 shows a configuration in which the gaze detector 70 is provided as a device for gaze detection, a configuration in which the gaze of a potential user is detected by analyzing an image captured by the camera 60 may also be used.

[0029] The control device 10 acquires signals output from each of the microphone 50, the camera 60, and the gaze detector 70. The control device 10 controls medical devices such as the microscope device 20 and controls display information displayed on the display device 40 based on the analysis results of the acquired signals.

[0030] The control device 10 includes a control unit 100 including a recognition unit 111 , a determination unit 112 , and an execution unit 113 .

[0031] The recognition unit 111 performs a predetermined recognition process based on the signals output from the microphone 50, the camera 60, and the gaze detector 70, and supplies the recognition result to the determination unit 112.

[0032] For example, the recognition unit 111 recognizes a voice command included in the speech of the user candidate based on a voice signal from the microphone 50. The recognition unit 111 also recognizes the user candidate based on the voice signal from the microphone 50 or the image signal from the camera 60. The recognition unit 111 also recognizes the gaze position of the user candidate based on the image signal from the camera 60 or the detection signal from the gaze detector 70.

[0033] The determination unit 112 determines whether or not the user candidate satisfies a predetermined condition for being a user having operation authority based on the recognition result from the recognition unit 111, and supplies the determination result to the execution unit 113. The predetermined condition is determined using information acquired before the start of surgery, information acquired during surgery, and the like.

[0034] When the determination result from the determination unit 112 satisfies a predetermined condition, the execution unit 113 executes a predetermined process based on the user's voice or an input triggered by the voice.

[0035] For example, the user's voice includes a voice command issued by the user. Also, the input triggered by the voice includes the user's line of sight when the user issues the voice command. The execution unit 113 controls the display information displayed on the display device 40 as a predetermined process according to the voice command issued by the user based on the user's line of sight position on the screen of the display device 40.

[0036] The processing executed by the control unit 100 may be executed by an external device other than the control device 10. In addition, among the processing executed by the recognition unit 111, the determination unit 112, and the execution unit 113 in the control unit 100, a part of the processing may be executed by an external device.

[0037] Thus, in the medical display system 1, during surgery, the display information generated by the control device 10 is displayed on the display device 40. The user performs various procedures such as resection of the affected area while observing the state of the surgical site by looking at the display information displayed on the display device 14. In addition, when performing various procedures, the user can operate each device constituting the medical display system 1 by the user's own voice or line of sight.

[0038] (Example of voice control) FIG. 3 shows an example of an operation using voice input in a case where there are multiple user candidates in an operating room.

[0039] In Fig. 3, display information including information necessary for surgery is displayed on the display device 40. In the example of Fig. 3, the display information includes an operative field image, a reference image, monitoring information, an operation menu, etc., and these pieces of information are viewed by multiple user candidates. In Fig. 3, three circles arranged in front of the screen of the display device 40 represent three user candidates U. A ~U C respectively.

[0040] The display device 40 is also equipped with a microphone 50, a camera 60, and a gaze detector 70, and detects user candidates U A ~U C It is possible to detect voice input and gaze direction by using a touch panel. In Fig. 3, the direction of speech of the user candidate is indicated by a dashed line, and the direction of gaze is indicated by a broken line.

[0041] At this time, the operation for changing the display information (contents) displayed on the display device 40 is performed by the user candidate through voice input, gaze, or a combination of these. A In a situation where a user candidate U needs to perform voice operation, B If the above utterance is recognized as a voice command, it will cause a malfunction.

[0042] In addition, user candidate U AIn a situation where a user candidate U should perform an operation, the operation is performed by a combination of voice and gaze. A At the same time that user U issues a voice command, B When the gaze position of the user is recognized as an operation, the user candidate U A However, unintended behavior may occur.

[0043] In addition, user candidate U A ~U C The recognition is performed by using the microphone 50 and the gaze detector 70, which are input devices for detecting operations, as well as the user candidate U present in the area where the screen of the display device 40 can be observed. A ~U C This is performed using a camera 60 capable of capturing an image from the screen side.

[0044] That is, the microphone 50 and the gaze detector 70 can detect the voice and gaze of at least the area where the screen of the display device 40 can be observed, respectively, and the three user candidates U A ~U C The camera 60 can capture an image of at least a part of the observable area of ​​the screen of the display device 40 as an imaging area. In the example of FIG. 3, the camera 60 can capture the images of the three user candidates U A ~U C are present within the imaging area.

[0045] In this way, in an operating room, not only the surgeon but also multiple medical staff such as assistants, nurses, and clinical engineers are potential users. Therefore, when performing voice operation, it is necessary to extract the voice command utterance of an appropriate user from multiple user candidates and suppress the execution of unintended operations. In addition, since the surgeon may be replaced by another doctor during the operation, it is necessary to determine the appropriate user.

[0046] Therefore, this technology uses information obtained before the start of surgery and information obtained during surgery to identify one user from among user candidates who meets certain conditions, and suppresses the execution of operations based on the speech, gaze position, etc. of the other user candidates.

[0047] Therefore, in a situation where the speech and gaze positions of multiple user candidates involved in the surgery can be detected, it is possible to execute only operations based on the speech and gaze positions of appropriate users. In other words, in an operating room, it becomes possible for one appropriate user among multiple user candidates present in the clean area to operate devices present in the unclean area using a non-contact user interface (UI).

[0048] (User determination process) Next, the flow of a user determination process at the time of voice input in the case where there are multiple user candidates in the operating room will be described with reference to the flowchart of FIG.

[0049] In the control device 10, when the user candidate speaks and the microphone 50 receives a voice input from the user candidate (S11), the control unit 100 executes the processes from step S12 onwards.

[0050] In step S12, the recognition unit 111 analyzes the voice signal from the microphone 50 and recognizes the voice command included in the speech content of the user candidate who has made the voice input.

[0051] In step S13, the recognition unit 111 recognizes the user candidate who has input the voice by analyzing the voice signal from the microphone 50 and the image signal from the camera 60. For example, in this recognition process, it is recognized which user candidate in the preset information the user candidate who has input the voice corresponds to.

[0052] In step S14, the recognition unit 111 analyzes the detection signal from the gaze detector 70 and recognizes the gaze position on the screen of the display device 40 of the candidate user who has made the voice input.

[0053] In step S15, the determination unit 112 determines whether or not the candidate user who inputs the voice satisfies a predetermined condition for being a user with operation authority, based on the recognition result. The predetermined condition is determined using information acquired before the start of surgery, information acquired during surgery, etc., as will be described in detail later.

[0054] If it is determined in step S15 that the predetermined condition is satisfied ("Yes" in S16), the process proceeds to step S17. In step S17, the execution unit 113 executes a predetermined process based on the recognition result.

[0055] Here, an operation is performed by a combination of voice and gaze by a user candidate determined to satisfy a predetermined condition, that is, a user identified from among a plurality of user candidates.

[0056] For example, when a user gazing at a specific surgical site included in the surgical field image displayed on the display device 40 issues a predetermined voice command, the execution unit 113 can enlarge and display the area of ​​the specific surgical site to which the user's gaze is directed. Also, for example, when a user gazing at one thumbnail image among multiple thumbnail images displayed on the display device 40 issues a predetermined voice command, the execution unit 113 can display an image such as a reference image corresponding to the thumbnail image to which the user's gaze is directed.

[0057] In this example, the operation is performed by a combination of voice and gaze, but the operation may be performed only by voice without using the user's gaze. In the case where the operation is performed only by voice, the process of step S14 may be skipped.

[0058] On the other hand, if it is determined in step S15 that the predetermined condition is not satisfied ("No" in S16), the process returns to step S11, and the subsequent processes are repeated.

[0059] The flow of the user determination process during voice input has been described above. In this user determination process, one user who meets a certain condition is identified from among user candidates using information acquired before the start of surgery, and the identified user performs operations using a non-contact user interface such as voice and gaze. This suppresses the execution of operations using speech, gaze position, etc. of other user candidates other than the user candidate identified as the user, making it possible to suppress the execution of unintended operations when operating equipment using voice input.

[0060] That is, when surgery is performed using the medical display system 1, if a pre-registered first user satisfies a predetermined condition in the imaging area, the display information displayed on the display device 40 is controlled based on the voice of the first user or an input triggered by the voice. Also, if the first user does not satisfy a predetermined condition in the imaging area, the display information is controlled based on the voice of the second user or an input triggered by the voice. In this way, in a UI device in which a user performs an operation while watching a display unit using the user's voice and line of sight, etc., only one user is specified, and only the operation based on the user's voice, line of sight, etc. is performed.

[0061] Here, the imaging area includes an area (determination area) where the face of a user candidate included in an image captured by the camera 60 can be determined. In other words, the above-mentioned "when the first user does not satisfy a predetermined condition in the imaging area" includes both the meanings of "the first user is not present in the imaging area" and "the first user is present in the imaging area but not in the determination area."

[0062] <2. Second embodiment>

[0063] As described above, the predetermined conditions used in the above-mentioned judgment process (S15, S16 in FIG. 4) are information acquired before the start of surgery, information acquired during surgery, etc. More specifically, the predetermined conditions can be determined based on at least one of information registered before the start of surgery, information on a confirmation action before the start of surgery, and information on a specific situation during surgery.

[0064] By defining specified conditions in this manner, it is possible, for example, to determine whether the information registered before the start of surgery matches the recognition result of a user candidate who will perform voice operations during surgery, to determine whether the situation at the time the user candidate performs voice operations is appropriate for a user with operating authority, or to determine whether the user candidate is in a specified position when performing voice operations.

[0065] User candidates can be divided into categories according to the operation authority. Fig. 5 shows an example in which user candidates are divided into three categories of operation authority, A to C.

[0066] Category A corresponds to the surgeon (the person giving the instructions) among the surgical staff in the operating room. Only the speech of a user candidate (user with A authority) assigned to the authority of category A can be a valid voice command, so basically, the user with A authority has the operation authority. However, it is possible to temporarily transfer the operation authority, and only the user with A authority can switch places with a user candidate (user with B authority) specified in his / her own authority B ranking.

[0067] Category B corresponds to other surgical staff, such as assistant surgeons and scopists, who are directly involved in the surgical procedure and may operate the display device 40. User candidates assigned to authority of category B (users with authority B) have operation authority when a user with authority A is absent or unable to operate.

[0068] Users with B authority can be assigned a B authority ranking according to the number added to "B", such as B1, B2, ..., Bn (n: an integer of 1 or more). In this example, the smaller the number, the stronger the authority, with B1 authority being the strongest authority. For example, when temporarily transferring operation authority, a user with the highest B authority among user candidates within the range of the imaging area can have A authority.

[0069] Category C includes non-operators such as nurses and clinical engineers among other surgical staff. User candidates assigned to category C authority (users with C authority) do not have the authority to operate by voice command.

[0070] In addition, if the predetermined conditions include whether or not the user was registered before the start of surgery, information such as the voiceprints and facial features of the surgical staff is stored in a database, and the information is assigned to each category before the start of surgery. Specifically, databases such as the doctor table shown in Figure 6 and the preoperative registration table shown in Figure 7 are used.

[0071] 6, the doctor table stores information on voiceprints and facial features for each piece of information identifying a doctor. For example, SDr1 and FDr1 are registered as the voiceprint and facial features of a doctor named Dr. 1. In addition, SDrx and FDRx are registered as the voiceprint and facial features of each of the doctors named Dr. 2 to Dr. 10.

[0072] 7, the preoperative registration table stores information on voiceprints and facial features for each piece of information on surgeons (user candidates) who are registered before the start of surgery. That is, in the preoperative registration table (FIG. 7), the doctor data stored in the doctor table (FIG. 6) is assigned to the surgeon data.

[0073] For example, when a doctor Dr. 3 is assigned as a surgeon with B1 authority, SDr3 and FDr3 as the doctor data are assigned to voiceprints S(B1) and F(B1) as the surgeon data.

[0074] In addition, when Dr.1 is assigned to the surgeon with B2 authority, SDr1 and FDr1 are assigned as the voiceprints S(B2) and F(B2), and when Dr.5 is assigned to the surgeon with B3 authority, SDr5 and FDr5 are assigned as the voiceprints S(B3) and F(B3).

[0075] Furthermore, when Dr. 6 is assigned to a surgeon with B4 authority, SDr6 and FDr6 are assigned as voiceprints S(B4) and F(B4), and when Dr. 9 is assigned to a surgeon with B5 authority, SDr9 and FDr9 are assigned as voiceprints S(B5) and F(B5).

[0076] In this way, by assigning the doctor data stored in the doctor table (Figure 6) to the surgeon data (data of a user with B authority) stored in the preoperative registration table (Figure 7) before the start of surgery, it is possible to register facial information used to determine the B authority ranking and for face recognition, and voiceprint information used for speaker identification.

[0077] In order to prevent misidentification of users, face information and voiceprint information of users with authority C may be registered. This information can be used to determine the criteria for determining users with authority A.

[0078] Furthermore, if the predetermined conditions include whether the user candidate has performed a confirmation operation (audio confirmation, etc.) before the start of surgery, the information in the database is checked at the time the confirmation operation is performed before the start of surgery, and the surgical staff who are user candidates in the operating room are divided into categories A, B, or C. As a result, the user candidates are classified as users with authority A, authority B, or authority C. This confirmation operation is performed not only when the user first enters the operating room, but also at predetermined times such as when a timeout occurs or when operating equipment.

[0079] The predetermined condition may also use information about the position (standing position, etc.) of the user candidate. This information about the position includes a position corresponding to the center of the screen of the display device 40 (for example, approximately the center of the width of the screen) and a position corresponding to the center of the angle of view of the camera 60 mounted on the display device 40. That is, during an actual surgery, the surgeon (the surgeon instructing) often stands directly in front of the screen of the display device 40, with assistants standing around him, and this positional relationship can be used.

[0080] The above-mentioned predetermined conditions are merely examples, and other conditions may be defined as long as they are determined using information acquired before the start of surgery, information acquired during surgery, etc. For example, a predetermined condition may be determined to determine whether the contents of the procedure of the candidate user (such as the type of surgical instrument being handled) correspond to the contents of a predetermined procedure, and the like.

[0081] As a method for recognizing the above-mentioned user candidates, image processing such as face recognition and bone recognition can be performed using an image signal from the camera 60, or voice processing such as voiceprint recognition and voice direction recognition can be performed using a voice signal from the microphone 50. Publicly known technologies can be used for face recognition and bone recognition, and for voiceprint recognition and voice direction recognition.

[0082] (Examples of user judgment) FIG. 8 shows a specific example of user determination during surgery.

[0083] 8, six circles arranged around the screen of the display device 40 represent user candidates, and the characters written inside each circle identify the surgical staff, such as a doctor. Also, the characters written on the cylinders below each circle represent the authority of each user candidate.

[0084] In Fig. 8, Dr. 3, who is assigned the B1 authority, Dr. 5, who is assigned the B3 authority, and Dr. 9, who is assigned the B5 authority, are outside the imaging area. Meanwhile, Dr. 1, who is assigned the B2 authority, and Dr. 6, who is assigned the B4 authority, are inside the imaging area. In addition, other surgical staff ST, such as nurses, are inside the imaging area, but are assigned the C authority.

[0085] At this time, it is assumed that the doctor Dr. 1 utters a voice command. In this case, the surgical staff in the imaging area are imaged by the camera 60. By analyzing the image signal obtained by this imaging, the three people, the doctor Dr. 1, the doctor Dr. 6, and the other surgical staff ST, are identified from their facial features.

[0086] In this way, when there is no doctor (user) with A authority in the imaging area and furthermore no doctor Dr.3 assigned with B1 authority, the doctor Dr.1 assigned with B2 authority in the imaging area has the authority A. That is, in this example, the doctor with A authority is replaced by the doctor Dr.1 who is the highest-ranked doctor with B authority within the imaging area among the doctors designated in his / her own B authority ranking.

[0087] The speech of the doctor Dr.1 is collected by microphones 50-1 and 50-2 as shown by dashed lines L11 and L12 in the figure. The voice print S(IN_Dr1) of the voice command obtained by analyzing this voice signal matches the voice print S(B2) registered in the preoperative registration table.

[0088] In addition, the facial features F(IN_ST) of the surgical staff ST in the direction from which the sound is coming do not match the facial features F(B2) registered in the preoperative registration table, but the facial features F(IN_Dr1) of Dr. 1 match the facial features F(B2) registered in the preoperative registration table.

[0089] From these determination results, a predetermined process corresponding to the voice command uttered by Dr. 1 is executed based on the line of sight information of Dr. 1 detected by the line of sight detector 70. For example, among the lines of sight of the user candidates in the imaging area shown by the dashed lines L21 to L23 in the figure, the dashed line L22 is the line of sight of Dr. 1, so a predetermined process corresponding to the voice command uttered by Dr. 1 is executed for a specific surgical site included in the visual field image to which Dr. 1's line of sight is directed.

[0090] (Voice response processing) Here, the flow of the voice response process applicable to the user determination shown in FIG. 8 will be described with reference to the flowchart in FIG.

[0091] The recognition unit 111 recognizes the highest-ranked user of authority B from among user candidates present in the imaging area (S31), and considers the recognized user of authority B to have authority A (S32). In the example of Fig. 8, a doctor Dr.1 who is present in the imaging area and assigned authority B2 has authority A.

[0092] The determination unit 112 compares the voiceprint S(IN) of the input voice command with the recognized voiceprint S(Bx) of the user of authority B (S33).

[0093] If it is determined that the voiceprint comparison result is S(IN)=S(Bx) ("Yes" in S34), the process proceeds to step S35. Then, the execution unit 113 recognizes that the voice command is from a user with authority A (S35), and executes a predetermined process according to the voice command. In the example of FIG. 8, the voice command issued by the doctor Dr. 1 is recognized as a voice command from a user with authority A, and the predetermined process is executed.

[0094] Furthermore, if it is determined that the voiceprint comparison result is not S(IN)=S(Bx) ("No" in S34), the process proceeds to step S36. In this case, the execution unit 113 determines that the voice command is not from a user with authority A (S36) and leaves the voice command unexecuted. In the example of FIG. 8, the utterance (voice command) by the doctor Dr. 6 or other surgical staff ST is determined to not be a voice command from a user with authority A.

[0095] When the process of step S35 or S36 ends, the process ends.

[0096] (Gaze-responsive processing) Next, the flow of gaze corresponding processing applicable to the user determination shown in FIG. 8 will be described with reference to the flowchart in FIG.

[0097] The recognition unit 111 recognizes the facial features of the first user candidate present in the direction from which the sound comes (S51). The determination unit 112 compares the facial features F(IN1) of the first user candidate with the facial features F(Bx) of the recognized user of authority B (S52).

[0098] The user of authority B recognized in step S52 is the same as the user of authority B who is recognized as the highest-ranking user of authority B within the range of the imaging area in the process of step S31 in FIG.

[0099] If it is determined in step S52 that the comparison result of the facial features is F(IN1)=F(Bx) ("Yes" in S53), the process proceeds to step S54. Then, the execution unit 113 adopts the line of sight of the first user candidate identified as the user (S54), and executes a predetermined process using the voice and line of sight of the first user candidate.

[0100] If it is determined in step S52 that the comparison result of the facial features is not F(IN1)=F(Bx) ("No" in S53), the process proceeds to step S55. In this case, the recognition unit 111 recognizes the facial features of the second user candidate present in the sound arrival direction (S55).

[0101] The determination unit 112 compares the facial features F(IN2) of the second user candidate with the facial features F(Bx) of the recognized user of authority B (S56). Note that, here too, the recognized user of authority B means the user of authority B who is the highest ranking within the range of the imaging area.

[0102] If it is determined in step S57 that the comparison result of the facial features is F(IN2)=F(Bx) ("Yes" in S57), the process proceeds to step S58. Then, the execution unit 113 adopts the line of sight of the second user candidate identified as the user (S58), and executes a predetermined process using the voice and line of sight of the second user candidate.

[0103] If it is determined in step S57 that the comparison result of the facial features is not F(IN2)=F(Bx) ("No" in S57), the process proceeds to step S59. In this case, the execution unit 113 determines that the voice command is not from a user with authority A (S59), and the predetermined process using voice and gaze is skipped.

[0104] When the process of step S54, S58, or S59 ends, the process ends.

[0105] In this example, for the sake of convenience, a case has been illustrated in which the faces of two user candidates, a first user candidate and a second user candidate, are present in the direction from which the sound is coming. However, if there are faces of other user candidates, it is sufficient to compare the facial features of all the user candidates.

[0106] <3. Third embodiment>

[0107] When there are multiple user candidates who satisfy a certain condition, a priority order (rank) of operation authority is set for each of the user candidates, so that users who can operate the system can be identified according to the rank. Various methods can be used for ranking. Rank information regarding ranks can be set before the start of surgery, changed according to the situation during surgery, or changed by a user with specific authority (e.g., a nurse).

[0108] (User Judgment Process According to Rank) The flow of the rank-based user determination process will be described with reference to the flowchart in FIG.

[0109] In steps S71 to S74, the recognition result of the voice command, the recognition result of the user candidate, and the recognition result of the gaze position are obtained, similarly to steps S11 to S14 in Fig. 4. In addition, in step S78, the determination unit 112 determines the rank of the user candidate based on the rank information. The details of the rank determination process will be described later.

[0110] In step S75, the determination unit 112 determines whether or not a predetermined condition for a user having operation authority is met, based on the recognition result and the rank determination result.

[0111] If it is determined in step S75 that the predetermined condition is satisfied ("Yes" in S76), the process proceeds to step S77. In step S77, the execution unit 113 executes a predetermined process based on the recognition result.

[0112] Moreover, if it is determined in step S75 that the predetermined condition is not satisfied ("No" in S76), the process returns to step S71, and the subsequent processes are repeated.

[0113] The flow of the user determination process using rank has been described above. Here, in the above-mentioned rank determination process (S78 in FIG. 11), the determination unit 112 performs rank determination of user candidates using rank information, and for example, the following process is performed.

[0114] That is, in the determination unit 112, rank information set for each user candidate is acquired, and based on the rank of each user candidate, the user candidate with the highest rank among the user candidates present in the imaging area is determined to be the user.

[0115] Then, the execution unit 113 controls the display information to be displayed on the display device 40 based on the voice and gaze of the user who is the highest ranked according to the determination result. For example, when the first user candidate is present in the imaging area and the condition that the first user candidate is the highest ranked is satisfied, an operation based on the voice and gaze of the first user candidate as a user is accepted.

[0116] In addition, the judgment unit 112 acquires rank information regarding the rank previously assigned to each user candidate, and adjusts the rank assigned to each user candidate based on information such as the confirmation action before the start of surgery and the position of the user candidate, to perform user judgment.

[0117] For example, a user candidate who utters a voice regarding a confirmation operation before the start of surgery can be set to the highest rank. Also, when a user candidate with a higher rank than the previous rank (pre-set rank) of the currently highest ranked user candidate appears in the imaging area, the rank can be adjusted based on the position information of the user candidate. That is, the contents of the information registered before the start of surgery can be set as the initial state, and the rank of the operation authority can be updated according to the situation of the surgery.

[0118] When a specific user performs a predetermined operation, such as pressing a reset button, the adjusted rank setting of the user candidate may be returned to a preset value.

[0119] Furthermore, when the first user candidate is shifted from a position corresponding to the center of the screen of the display device 40 (e.g., approximately the center in the horizontal width of the screen) or a position corresponding to the center of the angle of view of the camera 60 and the second user candidate is positioned near the center, if the preliminary rank of the second user candidate is higher than that of the first user candidate, the determination unit 112 performs user determination to accept the voice operation of the second user candidate. Also, in this case, if the preliminary rank of the second user candidate is lower than that of the first user candidate, the user determination is performed to accept the voice operation of the first user candidate.

[0120] In addition, if a registered user candidate is not present in the imaging area, user determination may be performed to accept voice operations from a user candidate who is located at a position corresponding to the center of the screen of the display device 40 or the center of the angle of view of the camera 60.

[0121] Furthermore, the determination unit 112 performs user determination by assigning the highest rank to a user candidate designated by a specific user such as a nurse. Furthermore, by pre-registering who the surgeon is, the determination unit 112 can perform user determination of who the surgeon is. For example, while the surgeon is present, adjustment of the rank is restricted so that the rank cannot be changed.

[0122] In addition, the judgment unit 112 can use the image recognition results using the image signal from the camera 60 and the detection signal from the gaze detector 70 to set the highest rank to user candidates who are holding a surgical tool, user candidates who are holding an electric scalpel, or user candidates who are looking at the screen of the display device 40 for longer than a predetermined period of time.

[0123] As described above, even if there are multiple user candidates who satisfy a certain condition, by determining the priority (rank) of the operation authority, it becomes possible to execute an appropriate operation by the user's speech, gaze, etc. In addition, the priority (rank) of the operation authority is determined based on preset information, but may be changed according to the situation of the surgery, so that the user can execute a voice operation that is more suitable for the situation.

[0124] <4. Fourth embodiment>

[0125] As mentioned above, a user with authority A can temporarily transfer authority to a user with authority B. In cases where such authority transfer is performed, the authority may be transferred to a designated user candidate (for example, a user with authority B) when the user with authority A issues a voice command for switching users (hereinafter, referred to as a swap command).

[0126] In addition, when canceling the transfer of authority, the transfer of authority is canceled when the user with authority A issues a command to cancel the user swap (hereinafter referred to as the swap cancellation command), and the operating authority is returned from the designated user candidate (for example, the user with authority B) to the user with authority A.

[0127] (Transfer of authority process) First, the flow of the authority transfer process will be described with reference to the flowchart of FIG.

[0128] In the control device 10, when a voice input by a user candidate is accepted (S91), the control unit 100 executes the processes from step S92 onwards.

[0129] In step S92, the recognition unit 111 analyzes the voice signal corresponding to the speech of the user candidate who has made the speech input, and recognizes the swap command included in the speech content of the user candidate. Here, the user candidate to whom the authority is to be transferred (the user of authority B) is also designated, and therefore is also recognized.

[0130] In step S93, the recognition unit 111 analyzes an image signal obtained by capturing an image of the user candidate who has made the voice input, and recognizes the user candidate who has made the speech.

[0131] In step S94, the determination unit 112 determines, based on the recognition result, whether or not the candidate user who has inputted speech satisfies a predetermined condition for being a user having authority A.

[0132] If it is determined in step S94 that the predetermined condition is met ("Yes" in S95), the process proceeds to step S96. In step S96, the execution unit 113 replaces the designated B-authority user (user candidate to whom authority is transferred) with the A-authority user (user candidate who has performed voice input) based on its own B-authority ranking assigned to the A-authority user. As a result, the authority is temporarily transferred to the designated B-authority user.

[0133] Moreover, if it is determined in step S94 that the predetermined condition is not satisfied ("No" in S95), the process returns to step S91, and the subsequent processes are repeated.

[0134] When the process of step S96 ends, the process ends. The flow of the authority transfer process has been described above.

[0135] (Releasing transfer of authority) Next, the flow of the authority delegation cancellation process will be described with reference to the flowchart in Fig. 13. However, it is assumed that, prior to the authority delegation cancellation process shown in Fig. 13, the authority delegation process described above (Fig. 12) is executed and authority is transferred from the user of authority A to the user of authority B (a candidate user to whom authority has been transferred).

[0136] In the control device 10, when a voice input by speech of a user candidate is accepted (S111), the control unit 100 executes the processes from step S112 onwards.

[0137] In step S112, the recognition unit 111 analyzes a voice signal corresponding to the speech of the user candidate who has made the speech input, and recognizes a swap cancellation command included in the speech of the user candidate.

[0138] In step S113, the recognition unit 111 analyzes an image signal obtained by capturing an image of the user candidate who has made the voice input, and recognizes the user candidate who has made the speech.

[0139] In step S114, the determination unit 112 determines, based on the recognition result, whether or not the candidate user who has inputted speech satisfies a predetermined condition for being a user having authority A.

[0140] If it is determined in step S114 that the predetermined condition is satisfied ("Yes" in S115), the process proceeds to step S116. In step S116, the execution unit 113 returns the temporarily swapped B authority ranking to the original ranking for the designated B authority user (candidate user to whom authority has been transferred). This cancels the authority transfer, and the operating authority is returned from the designated B authority user (candidate user to whom authority has been transferred) to the A authority user.

[0141] Moreover, if it is determined in step S114 that the predetermined condition is not satisfied ("No" in S115), the process returns to step S111, and the subsequent processes are repeated.

[0142] When the process of step S116 ends, the process ends. The flow of the authority transfer cancellation process has been described above.

[0143] As described above, regardless of the predetermined conditions, the operation authority can be temporarily transferred to a user candidate such as a user with authority B, thereby enabling more flexible operation.

[0144] <5. Fifth embodiment>

[0145] As mentioned above, users with authority C are people who do not have the authority to operate using voice commands. For those people who are not likely to be users, a user determination process can be performed to remove them from the user candidates.

[0146] (User exclusion processing) FIG. 14 is a flowchart illustrating the flow of the user exclusion process.

[0147] This user exclusion process increases the accuracy of user determination by taking into account not only the voiceprint S(Bx) of a user with B authority but also the voiceprint S(C) of a user with C authority when comparing and determining the voiceprint S(IN) of a voice command in steps S33 and S34 of the above-mentioned voice response process (FIG. 9), thereby excluding users with C authority from user candidates.

[0148] In step S131, the determination unit 112 calculates the probability P(Bx) that the voiceprint S(IN) of the voice command is the voiceprint S(Bx) of the user with authority B.

[0149] In step S132, the determination unit 112 calculates the probability P(C) that the voiceprint S(IN) of the voice command is the voiceprint S(C) of a user with C authority.

[0150] In step S133, the determination unit 112 compares the calculated value of probability P(Bx) with the value of probability P(C) to determine whether the value of probability P(Bx) is equal to or greater than the value of probability P(C).

[0151] If it is determined in the determination process of step S133 that P(Bx)≧P(C), the process proceeds to step S134. Then, the execution unit 113 recognizes that the voice command is from a user with authority A (S134), and executes a predetermined process according to the voice command.

[0152] If it is determined in the determination process of step S133 that P(Bx)<P(C), the process proceeds to step S135. In this case, the execution unit 113 determines that the voice command is not from a user with authority A (S135), and leaves the voice command unexecuted.

[0153] When the process of step S134 or S135 ends, the process ends. The flow of the user exclusion process has been described above.

[0154] In the above-mentioned user exclusion process, in the above-mentioned voice response process (FIG. 9), the voiceprint S(C) of the user with authority C is also taken into consideration when comparing and determining the voiceprint S(IN) of the voice command in steps S33 and S34. However, this can also be applied to other processes.

[0155] For example, in the above-mentioned gaze correspondence processing (FIG. 10), when comparing and determining the facial features F(IN1) and F(IN2) in steps S52, S52 or steps S56, S57, not only the facial features F(Bx) of the user with authority B but also the facial features F(C) of the user with authority C may be taken into consideration so that the user with authority C is excluded from the user candidates.

[0156] As described above, when determining the voiceprint S(IN) of a voice command in the voice response process (Fig. 9) or the facial feature F(IN) in the gaze response process (Fig. 10), by comparing not only the data of users with authority B but also the data of users with authority C, it is possible to exclude those who are not likely to be users from user candidates. As a result, it is possible to further improve the accuracy of user determination and increase the certainty of identifying users.

[0157] (Computer Configuration) The program executed by the control device 10 (control unit 100) may be a program in which processing is performed chronologically in the order described in this specification, or a program in which processing is performed in parallel or at the required timing, such as when a call is made.

[0158] The above-mentioned series of processes can be executed by hardware or software. When the series of processes is executed by software, the program constituting the software is installed in a computer incorporated in dedicated hardware, or in a general-purpose personal computer.

[0159] The program to be installed is provided by being recorded on a removable recording medium such as an optical disc (CD-ROM (Compact Disc-Read Only Memory), DVD (Digital Versatile Disc), etc.) or a semiconductor memory. It may also be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting. The program may be pre-installed in a ROM or recording unit.

[0160] In addition, the program executed by the computer may be a program in which processing is performed chronologically in the order described in this specification, or it may be a program in which processing is performed in parallel or at the required timing, such as when called.

[0161] In this specification, a system refers to a collection of multiple components (devices, modules (parts), etc.), regardless of whether all the components are in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device in which multiple modules are housed in a single housing, are both systems.

[0162] It should be noted that the effects described in this specification are merely examples and are not limiting, and other effects may also be obtained.

[0163] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present technology.

[0164] For example, the present technology can be configured as cloud computing, in which a single function is shared and processed collaboratively by multiple devices via a network.

[0165] Each step described in the above flow chart can be executed by one device or can be shared among multiple devices. Furthermore, when one step includes multiple processes, the multiple processes included in the step can be executed by one device or can be shared among multiple devices.

[0166] The present technology can be configured as follows.

[0167] (1) A display unit that displays information based on the information output from the medical device; an imaging unit that captures at least a part of an observable area of ​​the display unit as an imaging area; a voice acquisition unit that acquires voice from at least an area where the display unit is observable; A control unit that controls display information based on information output from the medical device; Equipped with The control unit is When a pre-registered first user satisfies a predetermined condition in the imaging area, the display information is controlled based on a voice of the first user or an input triggered by the voice; When the first user does not satisfy the predetermined condition in the imaging area, the display information is controlled based on a voice of a second user different from the first user or an input triggered by the voice. Medical display system. (2) The control unit determines the predetermined condition based on at least one of information registered before the start of surgery, information regarding a confirmation operation before the start of surgery, and information regarding a specific situation during surgery. The medical display system according to (1) above. (3) The control unit is recognizing user candidates including the first user and the second user based on at least one of an image captured by the imaging unit and a sound acquired by the sound acquisition unit; Determining whether the recognized user candidate satisfies the predetermined condition The medical display system according to (2) above. (4) The information registered before the start of the surgery includes information about the candidate user. The medical display system according to (3) above. (5) The information on the user candidate includes correspondence information that associates information on the characteristics of the surgeon with information on the characteristics of the doctor. The medical display system according to (4) above. (6) The information regarding the confirmation operation before the start of the surgery includes information regarding operation authority consisting of a plurality of categories, The control unit assigns the user candidates who have performed the confirmation operation before the start of the surgery to the categories of operation authority. The medical display system according to (3) or (4). (7) The categories of the operation authority include a first category in which the user has the operation authority and a voice command is enabled, a second category in which a voice command is enabled when the operation authority is transferred, and a third category in which the user does not have the operation authority. The medical display system according to (6) above. (8) The information regarding the particular situation during surgery includes information regarding the location of the potential user. The medical display system according to (3), (4), or (6). (9) The information regarding the position of the user candidate includes a position corresponding to an approximate center portion in the width of the display unit. The medical display system according to (8) above. (10) When there are a plurality of user candidates who satisfy the predetermined condition, the control unit sets a priority order of operation authority for each of the user candidates. The medical display system according to any one of (3), (4), (6), and (8). (11) The control unit updates the priority of operation authority according to the status of the surgery, with the contents of the information registered before the start of the surgery as an initial state. The medical display system according to (10) above. (12) The control unit updates the priority of operation authorities in response to an operation by a user having a specific authority. The medical display system according to (10) or (11) above. (13) The control unit temporarily transfers an operation authority to a designated user candidate when the first user issues a voice command for changing users when the first user satisfies the predetermined condition. A medical display system according to any one of (3) to (12). (14) When the first user satisfies the predetermined condition, and the first user issues a voice command for canceling the user change, the control unit returns an operation authority from the designated user candidate to the first user. The medical display system according to (13) above. (15) The control unit performs a recognition process on user candidates who cannot be the first user and the second user, and excludes them from the user candidates. A medical display system according to any one of (3) to (12). (16) The voice-triggered input includes a gaze of the first user or the second user when the first user or the second user issues a voice command. A medical display system according to any one of (1) to (15). (17) The control unit executes a predetermined process in response to the voice command based on a gaze position of the first user or the second user on the display unit. The medical display system according to (16) above. (18) a gaze detection unit that detects a gaze of the first user or the second user present in the imaging area The medical display system according to (17) above. (19) Medical display systems When controlling display information displayed on the display unit based on information output from the medical device, when a pre-registered first user satisfies a predetermined condition in an imaging area in which an image of the display unit is captured as at least a part of an observable area of ​​the display unit, the display information is controlled based on a voice of the first user or an input triggered by the voice, which is acquired in at least an observable area of ​​the display unit; When the first user does not satisfy the predetermined condition in the imaging area, the display information is controlled based on a voice of a second user different from the first user, which is acquired in at least an area where the display unit is observable, or an input triggered by the voice. Control methods. (20) a control unit that controls display information to be displayed on the display unit based on information output from the medical device; The control unit is when a pre-registered first user satisfies a predetermined condition in an imaging area in which an image of the display unit is captured as at least a part of an observable area of ​​the display unit, the display information is controlled based on a voice of the first user or an input triggered by the voice, which is acquired in at least an observable area of ​​the display unit; When the first user does not satisfy the predetermined condition in the imaging area, the display information is controlled based on a voice of a second user different from the first user, which is acquired in at least an area where the display unit is observable, or an input triggered by the voice. Control device. [Explanation of symbols]

[0168] 1 medical display system, 10 control device, 20 microscope device, 30 monitoring device, 40 display device (display unit), 50, 50-1, 50-2 microphone (voice acquisition unit), 60 camera (imaging unit), 70 gaze detector (gaze detection unit), 100 control unit, 111 recognition unit, 112 judgment unit, 113 execution unit

Claims

1. A display unit that displays information based on the information output from the medical device; an imaging unit that captures at least a part of an observable area of ​​the display unit as an imaging area; a voice acquisition unit that acquires voice from at least an area where the display unit is observable; A control unit that controls display information based on information output from the medical device; Equipped with The control unit is When a pre-registered first user satisfies a predetermined condition in the imaging area, the display information is controlled based on a voice of the first user or an input triggered by the voice; When the first user does not satisfy the predetermined condition in the imaging area, the display information is controlled based on a voice of a second user different from the first user or an input triggered by the voice. Medical display system.

2. The control unit determines the predetermined condition based on at least one of information registered before the start of surgery, information regarding a confirmation operation before the start of surgery, and information regarding a specific situation during surgery. The medical display system of claim 1 .

3. The control unit is recognizing user candidates including the first user and the second user based on at least one of an image captured by the imaging unit and a sound acquired by the sound acquisition unit; Determining whether the recognized user candidate satisfies the predetermined condition The medical display system of claim 2 .

4. The information registered before the start of the surgery includes information about the candidate user. The medical display system of claim 3.

5. The information on the user candidate includes correspondence information that associates information on the characteristics of the surgeon with information on the characteristics of the doctor. The medical display system of claim 4.

6. The information regarding the confirmation operation before the start of the surgery includes information regarding operation authority consisting of a plurality of categories, The control unit assigns the user candidates who have performed the confirmation operation before the start of the surgery to the categories of operation authority. The medical display system of claim 3.

7. The categories of the operation authority include a first category in which the user has the operation authority and a voice command is enabled, a second category in which a voice command is enabled when the operation authority is transferred, and a third category in which the user does not have the operation authority. The medical display system of claim 6.

8. The information regarding the particular situation during surgery includes information regarding the location of the potential user. The medical display system of claim 3.

9. The information regarding the position of the user candidate includes a position corresponding to an approximate center portion in the width of the display unit. The medical display system of claim 8.

10. When there are a plurality of user candidates who satisfy the predetermined condition, the control unit sets a priority order of operation authority for each of the user candidates. The medical display system of claim 3.

11. The control unit updates the priority of operation authority according to the status of the surgery, with the contents of the information registered before the start of the surgery as an initial state. The medical display system of claim 10.

12. The control unit updates the priority of operation authorities in response to an operation by a user having a specific authority. The medical display system of claim 10.

13. The control unit temporarily transfers an operation authority to a designated user candidate when the first user issues a voice command for changing users if the first user satisfies the predetermined condition. The medical display system of claim 3.

14. When the first user satisfies the predetermined condition, and the first user issues a voice command for canceling the user change, the control unit returns an operation authority from the designated user candidate to the first user.

14. The medical display system of claim 13.

15. The control unit performs a recognition process on user candidates who cannot be the first user and the second user, and excludes them from the user candidates. The medical display system of claim 3.

16. The voice-triggered input includes a gaze of the first user or the second user when the first user or the second user issues a voice command. The medical display system of claim 1 .

17. The control unit executes a predetermined process in response to the voice command based on a gaze position of the first user or the second user on the display unit.

17. The medical display system of claim 16.

18. a gaze detection unit that detects a gaze of the first user or the second user present in the imaging area 20. The medical display system of claim 17.

19. Medical display systems When controlling display information displayed on the display unit based on information output from the medical device, when a pre-registered first user satisfies a predetermined condition in an imaging area in which the display unit is imaged as at least a part of an observable area, the display information is controlled based on a voice of the first user acquired in at least an observable area of ​​the display unit or an input triggered by the voice; When the first user does not satisfy the predetermined condition in the imaging area, the display information is controlled based on a voice of a second user different from the first user acquired in at least an observable area of ​​the display unit or an input triggered by the voice. Control methods.

20. a control unit that controls display information to be displayed on the display unit based on information output from the medical device; The control unit is when a pre-registered first user satisfies a predetermined condition in an imaging area in which the display unit is imaged as at least a part of an observable area, the display information is controlled based on a voice of the first user acquired in at least an observable area of ​​the display unit or an input triggered by the voice; When the first user does not satisfy the predetermined condition in the imaging area, the display information is controlled based on a voice of a second user different from the first user acquired in at least an observable area of ​​the display unit or an input triggered by the voice. Control device.

Images