Surgical microscope system and corresponding systems, methods, and computer programs

Abstract

Examples relate to a system, method and computer program for controlling a recording device of a surgical microscope system, and to a corresponding surgical microscope system. The system comprises one or more processors and an interface. The system is configured to acquire, via the interface, input data from one or more sensors or actuators of the surgical microscope system. The system is configured to detect, based on the input data, a start or an end of a surgery. The surgery involves the use of the surgical microscope system. The system is configured to provide, via the interface, a control signal to a recording device of the surgical microscope system based on the detected start or end of the surgery. The input data can be used to detect the start or end of a surgical procedure, which in turn can trigger the activation or deactivation of a recording performed by the recording device.

Description

TECHNICAL FIELD

[0001] Examples relate to a system, method and computer program for controlling a recording device of a surgical microscope system, and a corresponding surgical microscope system. BACKGROUND

[0002] One major focus of efforts to improve the quality of surgical procedures is the recording of surgical procedures by a camera, which can be integrated into a surgical microscope. In some countries, such as Singapore, all surgeries must be video recorded. With more and more surgeries being performed, for example in ophthalmology or more specifically cataract surgery, delays can occur due to the handling of the recording, as the surgeon can start and stop the recording manually by a foot switch or with the help of a nurse. SUMMARY

[0003] There can be a need for improved concepts for controlling the recording of a surgery.

[0004] This need is addressed by the subject matter of the independent claims.

[0005] Embodiments of the present disclosure are based on the finding that a state of a surgical microscope system comprising a surgical microscope can be analyzed to find out whether the surgical microscope is being used for a surgical procedure or whether the surgical microscope is currently not used for a surgical procedure. For example, a vertical height of the surgical microscope above ground or above a patient can be indicative of a surgical microscope system being used. Additionally or alternatively, a parallelogram portion of an arm of the surgical microscope system can have an active orientation (indicative of a surgical microscope being used), a first position (e.g. above a patient) of the surgical microscope being placed, and a parking orientation (indicative of the surgical microscope system not being used), a second position of the surgical microscope being placed away from the patient. Additionally or alternatively, imaging sensor data of an imaging sensor of the surgical microscope can be used to determine whether the captured imaging sensor data is in focus (indicative of a surgical microscope being used) or whether a patient is visible in the captured imaging sensor data (indicative of a surgical microscope being used). Based on the detection, the recording can be controlled (i.e. automatically started and / or stopped). Using this measure, delays can be avoided, thereby improving the efficiency and turnover of the operating room, thereby improving the ability to help more patients per day.

[0006] Embodiments of the present disclosure provide a system for controlling a recording device of a surgical microscope system. The system comprises one or more processors and an interface. The system is configured to obtain input data from one or more sensors or actuators of the surgical microscope system via the interface. The system is configured to detect a start or an end of a surgery based on the input data. The surgery involves the use of the surgical microscope system. The system is configured to provide a control signal to a recording device of the surgical microscope system via the interface based on the detected start or end of the surgery. The input data can be used to detect the start or the end of the surgical procedure, which in turn can trigger the activation or deactivation of a recording performed by the recording device.

[0007] In various embodiments, the input data comprises information about a position of an arm of the surgical microscope system. The system can be configured to detect the start or the end of the surgery based on the information about the position of the arm. The position of the arm can be indicative of whether the surgical microscope is being used.

[0008] In embodiments, a surgical microscope of the surgical microscope system is attached to an arm of the surgical microscope system. The information about the position of the arm can comprise information about a vertical position of the surgical microscope. The system can be configured to detect the start or the end of the surgery based on the information about the vertical position of the surgical microscope. For example, the start of the surgical procedure can be detected if the surgical microscope is brought to a lower vertical position (e.g. close to the surgical site) and the end of the surgical procedure can be detected if the surgical microscope is brought to a higher vertical position. In other words, the system can be configured to detect the start of the surgery after the vertical position of the surgical microscope is lowered below a threshold value. The system can be configured to detect the end of the surgery after the vertical position of the surgical microscope is increased above a threshold value.

[0009] In certain cases, the surgical microscope can be temporarily moved away before resuming the surgery, thereby exceeding the threshold value. In such cases, a timer can be used to avoid switching off the recording during this time. For example, the system can be configured to detect the end of the surgery if the surgical microscope remains above the threshold value for at least a predefined time interval.

[0010] To obtain a higher confidence, not only the vertical height can be analyzed, but also the usage of the surgical microscope system. Typically, the depth of field of the surgical microscope is shallow. If the surgical microscope is not used, the object viewed through the microscope can be out of focus, which is an indication that the surgical microscope is not used (at least temporarily). In other words, the input data can comprise imaging sensor data of an imaging sensor of the surgical microscope. The system can be configured to detect the end of the surgery if the surgical microscope remains above the threshold value and the imaging sensor data is out of focus for at least a predefined time interval. The imaging sensor data can increase the confidence that the surgical microscope is not currently used.

[0011] In various embodiments, the arm of the microscope system comprises a parallelogram portion. The information about the position of the arm can comprise information about the orientation of the parallelogram portion of the arm. The system can be configured to detect the start or end of the surgery based on the information about the orientation of the parallelogram portion of the arm. For example, the parallelogram portion of the arm can be movable between an active position / orientation and a parked position / orientation.

[0012] Generally, there are (at least) two ways to determine the position of the arm. In some embodiments, sensor data of a sensor of the arm is used to determine the position of the surgical microscope (and thus the orientation of the arm). For example, one or more sensors or actuators of the surgical microscope system can comprise a positioning sensor for determining the position of the arm (and thus the position of the surgical microscope). The system can be configured to obtain information about the position of the arm from the positioning sensor. Alternatively, the arm can be a robotic arm. In this case, the state of the robotic arm can reflect the position of the robotic arm, which in turn reflects the position of the surgical microscope. In other words, the arm can be a robotic arm. The system can be configured to obtain information about the position of the arm from a control circuit of the robotic arm.

[0013] In various embodiments, the input data comprises imaging sensor data of an imaging sensor of a surgical microscope of the surgical microscope system. The system can be configured to detect the start or end of the surgery based on the imaging sensor data. Various aspects of the imaging sensor data can be used to determine whether the surgical microscope is currently in use and thus determine the start and / or end of the surgical procedure.

[0014] For example, the system can be configured to detect the presence of a patient within the imaging sensor data. The system can be configured to detect the start or end of the surgery based on the presence of the patient within the imaging sensor data. For example, the start of the surgery can be detected if the patient becomes visible in the imaging sensor data and the end of the surgery can be detected if the patient is no longer visible in the imaging sensor data.

[0015] Alternatively or additionally, the (ambient) lighting can be analyzed to detect the start or end of the surgical procedure. For example, during surgery, the lighting can be focused on the patient, while other parts of the room receive less light. Outside of surgery, the lighting of the room can be more uniform. In other words, the input data can comprise information about the lighting used in the environment of the surgical microscope system. The system can be configured to detect the start or end of the surgery based on the information about the lighting used in the environment of the surgical microscope system.

[0016] The start and end of the surgery are used to control the recording device. Thus, if a start of the surgery is detected, the system can be configured to provide a control signal to start the recording to be performed by the recording device. The system can be configured to provide a control signal to stop the recording being performed by the recording upon detection of an end of the surgery. Thus, the recording can be started and / or stopped automatically.

[0017] Embodiments of the present invention further provide a surgical microscope system comprising the above-mentioned system, one or more sensors or actuators for providing the input data and the recording device. Thus, the surgical microscope system can provide an automatic control of the recording device. Furthermore, the surgical microscope system can comprise at least one of a surgical microscope, an arm attached to the surgical microscope and an optical imaging sensor.

[0018] Embodiments of the present invention further provide a control method for a recording device of a surgical microscope system. The method comprises obtaining input data from one or more sensors or actuators of the surgical microscope system. The method comprises detecting a start or an end of a surgery based on the input data. The surgery involves a use of the surgical microscope system. The method comprises providing a control signal to a recording device of the surgical microscope system based on the detected start or end of the surgery.

[0019] Embodiments of the present disclosure further provide a computer program having a program code for performing the method, when the computer program is executed on a processor. BRIEF DESCRIPTION OF DRAWINGS

[0020] Some examples of apparatus and / or methods will hereinafter be described in the context of examples only, and by way of illustration, and not limitation, with reference to the accompanying drawings, in which

[0021] Figure 1a A block diagram illustrating an embodiment of a system for controlling a recording device of a surgical microscope system is shown;

[0022] Figure 1b A schematic diagram illustrating an embodiment of a surgical microscope system is shown;

[0023] Figure 1c and 1d A more detailed schematic diagram illustrating an embodiment of a surgical microscope system is shown;

[0024] Figure 2 A flowchart illustrating a method for controlling a recording device of a surgical microscope system is shown;

[0025] Figure 3a and 3b A schematic diagram illustrating a surgical microscope system in an active position and in a parked position is shown; and

[0026] Figure 4 A schematic diagram illustrating a system comprising a microscope and a computer system is shown. DETAILED DESCRIPTION

[0027] Various examples will now be described, by way of example only, with reference to the accompanying drawings, in which various examples are shown. In the drawings, the thickness of lines, layers and / or regions can be exaggerated for clarity.

[0028] Figure 1a A block diagram of an embodiment of a system 110 for controlling a recording device 120 of a surgical microscope system 100 is shown. The system comprises one or more processors 114 and an interface 112 coupled to the one or more processors. Optionally, the system further comprises one or more storage devices 116, which are also coupled to the one or more processors. Generally, the functionality of the system 110 is provided by the one or more processors in conjunction with the interface and / or the one or more storage devices. The system is configured to obtain input data from one or more sensors or actuators of the surgical microscope system via the interface. The system is configured to detect a start or an end of a surgery based on the input data. The surgery involves the use of the surgical microscope system, e.g. a surgical microscope 140 of the surgical microscope system. The system is configured to provide a control signal to the recording device of the surgical microscope system via the interface based on the detected start or end of the surgery.

[0029] Embodiments of the present disclosure relate to a system, method and computer program for controlling a recording device 120 of a surgical microscope system 100, the surgical microscope system comprising a surgical microscope 140. Generally, a microscope is an optical instrument that is suitable for inspecting objects that are too small to be inspected by the human eye (alone). For example, a microscope can provide optical magnification of a sample. In modern microscopes, the optical magnification is typically provided for a camera or imaging sensor, such as the optical imaging sensor 142 of the microscope 140 shown in Figure 1b The surgical microscope 140 can further comprise one or more optical magnification components for magnifying a view on a sample, such as an objective lens (i.e. a lens).

[0030] There are various different types of microscopes. If the microscope is used in the medical or biological field, the object that is observed by the microscope can be a sample of organic tissue, e.g. arranged in a petri dish or present in a part of a patient’s body. In embodiments, the microscope is a surgical microscope, i.e. a microscope that is used during a surgical operation. For example, such a system is shown in Figures 1b to 1d In Figure 1b a neurosurgical microscope system is shown, comprising a neurosurgical microscope. In Figure 1c and 1d a surgical microscope for ophthalmology (i.e. diagnosis and treatment of eye diseases) is shown. Thus, the object that is observed by the microscope and displayed in the image data can be a sample of organic tissue of a patient.

[0031] The above-described system 110 is suitable for use with the surgical microscope system 100. Figure 1b A schematic diagram of an embodiment of a surgical microscope system 100 is shown. Figure 1b The image shows a neurosurgical microscope system. The surgical microscope system 100 includes a system 110, one or more sensors or actuators 132, 134, and 142 for providing input data, and a recording device 120. The surgical microscope system includes a surgical microscope 140 (…). Figure 1b The neurosurgical microscope, the arm 130 attached to the surgical microscope, and the optical imaging sensor 142 of the surgical microscope (which is one of one or more sensors used to provide input data). Figure 1b The surgical microscope system 100 shown includes several optional components, such as a base unit 102 (including system 110) with a (rolling) support, one or more displays 150, a (robotic or manual) arm 130 that holds a microscope 140 in place and is coupled to the base unit 102 and the microscope 140, and a steering handle 160 connected to the microscope 140. In other words, the surgical microscope 140 of the surgical microscope system can be attached to the arm 130 of the surgical microscope system. The one or more displays 150 can be part of the microscope 140, for example, as an auxiliary or visual display. In the context of this application, the term "surgical microscope system" is used to cover parts of the system that are not part of the actual microscope (including optical components) but are used in conjunction with the microscope, such as displays or illumination systems.

[0032] Figure 1c and 1d Another type of surgical microscope system is shown, a surgical microscope system for ophthalmology. Figure 1c and 1d A more detailed schematic diagram of an embodiment of a surgical microscope system for ophthalmology is shown. Figure 1c and 1d The image shows a surgical microscope 140 (also known as an "optical carrier") and a parallelogram 136 of the arm of the surgical microscope system. Figure 1c and 1d The surgical microscope system further includes a display / video monitor 150, which has an optional monitor arm 152, a base unit 102, a foot brake 104, and a handrail 106. Figure 1c The surgical microscope system 100 includes control units 118a and 118b for controlling the system 110 and / or for controlling the camera and video. Additionally, an interface 112 is shown, which can provide terminals (electrical, video, etc.). Figure 1c and 1d The surgical microscope system includes a speaker 108, which is part of the base unit 102.

[0033] Typically, the system is configured to detect the start or end of a procedure based on input data of one or more sensors or actuators. Thus, the system is configured to obtain, e.g. receive or read out, respective input data from one or more sensors and actuators. For example, the input data can be obtained by receiving the input data from the one or more sensors or actuators, e.g. via the interface 112, by reading out the input data from a memory of the one or more sensors or actuators, e.g. via the interface 112, or by reading the input data from a storage device 116 of the system 110, e.g. after the input data has been written to the storage device 116 by the one or more sensors or actuators or another system or processor.

[0034] Typically, input data of different types of sensors and actuators can be used to determine the start or end of a procedure, i.e. a surgical procedure. At one time, the positioning and / or orientation of the arm 130 of the surgical microscope system can be indicative of a surgical procedure being performed. Thus, the system can be configured to obtain (at least part of) the input data from the positioning sensor 132 of the arm 130, e.g. if the arm is an arm that is manually placed into position by a surgeon, or the arm can be a robotic arm, in which case the system can be configured to obtain (at least part of) the input data from the control circuit 134 of the robotic arm, i.e. a part of the actuator system. In other words, the one or more sensors or actuators of the surgical microscope system can comprise a positioning sensor 132 for determining the position of the arm, e.g. a Hall-effect based positioning sensor or a rotary encoder. The system can be configured to obtain information about the position of the arm from the positioning sensor 132. Alternatively, the system is configured to obtain information about the position of the arm from the control circuit 134 of the robotic arm. In some embodiments, the imaging sensor 142 of the surgical microscope can be used to determine the start or end of a surgical procedure. Thus, the system can be configured to obtain (at least part of) the input data from the imaging sensor 142 of the surgical microscope, e.g. imaging sensor data. Further, the surgical or ambient illumination can be analyzed to determine whether a surgical procedure is started or terminated. Thus, the system can be configured to obtain (at least part of) the input data from a control circuit of the illumination system or using imaging sensor data of the imaging sensor 142 of the microscope 140.

[0035] Based on the input data, the system is configured to detect a start or an end of a surgery (or surgical procedure), which involves the use of the surgical microscope system, e.g. the surgical microscope of the surgical microscope system. In other words, the surgical microscope of the surgical microscope system can be used (at least most of the time) during the surgery. For example, the surgery can be performed using the surgical microscope, i.e. with the aid of the surgical microscope. Thus, the input data can (implicitly) be indicative of the use of the surgical microscope system during the surgery. The system can now be used to detect a marker that is indicative of the surgical microscope (system) being used in a surgery, and thus to detect the start and / or end of the surgery. These markers can be implicit markers, i.e. they can implicitly be indicative of the start and / or end of the surgery. In other words, the input data can be analyzed to detect evidence of the start and / or end of the surgery. In various examples, the detection of the start and / or end of the surgery can not be based on the detection of a manual activation and / or deactivation of the recording. As mentioned in the previous section, different markers (detecting the start and / or end of the surgery) can be used.

[0036] One major marker is the positioning and / or orientation of the arm of the surgical microscope system, and thus of the surgical microscope itself. Thus, the input data can comprise information about the position of the arm 130 of the surgical microscope system. For example, the information about the arm position can comprise information about the vertical and lateral orientation of the arm, and / or information about the position of the surgical microscope attached to the arm. In other words, the information about the position of the arm can comprise information about the vertical position and / or lateral position of the surgical microscope attached to the arm. This information can be used to detect the start or end of a surgery. Thus, the system can be configured to detect the start or end of a surgery based on the information about the position of the arm. For example, if the position of the arm (and thus of the surgical microscope) changes to a first range of positions, the start of a surgery can be detected, and if the position of the arm (and of the surgical microscope) changes to a second range of positions, the end of a surgery can be detected. For example, the first range of positions can be indicative of a start of a surgery (or of a surgery in progress), e.g. due to the positions of the first range being close to the surgical table, and the second range of positions can be indicative of an end of a surgery (or of no surgery being performed).

[0037] The two ranges of positions can be defined along the vertical position of the surgical microscope, i.e. above a reference level, such as the ground or the height of the patient. In other words, the system can be configured to detect the start or end of the surgery based on information about the vertical position of the surgical microscope. For example, if the surgical microscope is lowered (and thereby brought closer to the patient / surgical site of the patient), this position can indicate that the surgery is starting or ongoing, if the surgical microscope is placed in a higher vertical position (and thus further away from the patient / surgical site), this position can indicate that the surgery is terminating or completed. Thus, the system can be configured to detect the start of the surgery after the vertical position of the surgical microscope is lowered below a threshold value, e.g. below a first lower threshold value, and to detect the end of the surgery after the vertical position of the surgical microscope is increased above a threshold value, e.g. a second higher threshold value. Thus, the vertical position of the surgical microscope below the (first lower) threshold value can correspond to a first range of positions, and the vertical position above the (second higher) threshold value can correspond to a second transition range.

[0038] In certain situations, the surgeon can move the surgical microscope away during the surgical procedure, e.g. to perform a part of the surgery where an overall view of the surgical site can be preferred, or to move additional surgical tools or to remove previously used tools. In such situations, it can be desirable to postpone the termination of the recording, as it can be desirable to have a continuous recording of the surgical procedure. If the surgical procedure is only recorded using the imaging sensor of the surgical microscope, this can be acceptable, as the recording can only show out-of-focus images. However, typically, the recording is performed using multiple cameras, including the imaging sensor of the surgical microscope, e.g. to save everything that happens in the operating room. Thus, the recording can not be terminated immediately once the surgical microscope is moved away from the patient. Therefore, a timer can be used to delay the detection of the end of the surgery until after the timer has expired. In other words, the system can be configured to detect the end of the surgery if the surgical microscope remains above the threshold value for at least a predefined time interval.

[0039] Other factors can also be considered. For example, as previously mentioned, the imaging sensor data provided by the imaging sensor of the surgical microscope being out-of-focus can also indicate that the surgery has ended. Thus, the system can be configured to detect the end of the surgery if the surgical microscope remains above the threshold value for at least a predefined time interval and the imaging sensor data is out-of-focus. The system can be configured to determine whether the imaging sensor data is out-of-focus, e.g. by determining the contrast of the imaging sensor data and comparing the contrast to a threshold value. A higher contrast typically indicates that the imaging sensor data is in focus. Alternatively, built-in functionality of the imaging sensor can be used.

[0040] As already introduced before, the orientation of the arm, in particular the parallelogram portion of the arm, can also indicate whether a surgery is currently being performed or not. Typically, the parallelogram portion of the arm is the portion of the arm that enables adjusting the position of the surgical microscope without changing the angle (above ground / patient) of the surgical microscope. In many embodiments, the parallelogram portion comprises a compression spring or a hydraulic system that supports changing the position of the surgical microscope. The information about the position of the arm can comprise information about the orientation of the parallelogram portion of the arm, and thus the (vertical and / or horizontal) position of the surgical microscope attached to the arm. The system can be configured to detect the start or end of a surgery based on the information about the orientation of the parallelogram portion of the arm. For example, the parallelogram portion can comprise an “activated” orientation (during surgery), which can be in a first angle (or range of angles) and / or vertical height (above the patient), and a “parked” orientation, which can be in a second angle (or range of angles) away from the patient and / or vertical height away from the patient / surgical table (see Figure 3a and 3b ). Accordingly, the position of the surgical microscope for which the parallelogram portion is in the activated orientation can be in a first range of positions, and the position of the surgical microscope for which the parallelogram portion is in the parked orientation can be in a second range of positions. The system can be configured to detect the start of a surgery if the parallelogram portion is moved into the activated orientation, and to detect the end of a surgery if the parallelogram portion is moved into the parked orientation (e.g. for at least a predefined time interval, and / or if the imaging sensor data is out of focus for at least a predefined time interval).

[0041] In various embodiments, the input data comprises imaging sensor data of an imaging sensor 142 of a surgical microscope of the surgical microscope system. For example, the system can be configured to obtain the imaging sensor data from an optical imaging sensor 142 of the microscope. For example, the optical imaging sensor 142 can comprise or be an imaging sensor based on APS (active pixel sensor) or CCD (charge-coupled device). For example, in an APS-based imaging sensor, light is recorded at each pixel using a photodetector and an active amplifier of the pixel. APS-based imaging sensors are typically based on CMOS (complementary metal-oxide-semiconductor) or S-CMOS (scientific CMOS) technology. In a CCD-based imaging sensor, incident photons are converted to electronic charges at a semiconductor oxide interface, which are then moved between capacitive binary bits in the imaging sensor by a control circuit of the imaging sensor to perform imaging.

[0042] The system can be configured to detect the start or end of a surgery based on imaging sensor data, which can originate from an imaging sensor of the surgical microscope and / or from one or more additional imaging sensors, e.g. cameras, placed in the environment of the surgical microscope system. For example, in some embodiments, if the surgical microscope is placed over a patient during a surgery, the imaging sensor data can show the patient on the operating table, which is a reliable indicator that the surgery is in progress or started. Thus, the system can be configured to detect the presence of a patient within the imaging sensor data. The system can be configured to detect the start or end of a surgery based on the presence of a patient within the imaging sensor data. For example, if or when a patient is detected, and (first) imaging sensor data is detected, the system can detect the start of a surgery, and if the patient is no longer detected, or if the patient is detected to be moved away from the operating table, the system can detect the end of a surgery.

[0043] Another sign or indicator is the illumination of the operating room. During a surgery, the ambient light is typically reduced or switched off and the illumination is focused on the patient. Between surgeries, the ambient light is switched on, e.g. in order to change the sterile covers of the surgical devices. Thus, the system can be configured to analyze the illumination situation to detect the start or end of a surgery. In other words, the input data can comprise information about the illumination used in the environment of the surgical microscope system, e.g. in the operating room. For example, the imaging sensor data of the imaging sensor of the surgical microscope (or another imaging sensor, e.g. a camera having a field of view covering the surgical team) can comprise information about the illumination used in the environment of the surgical microscope system. Alternatively or additionally, the system can be configured to obtain (at least part of) the information about the illumination used in the environment of the surgical microscope system from the illumination system of the surgical microscope system and / or from control circuitry of the operating room. The system can be configured to detect the start or end of a surgery based on the information about the illumination used in the environment of the surgical microscope system. For example, if the information about the illumination used in the environment of the surgical microscope system indicates that the ambient light is dimmed or switched off and the illumination is focused on the patient, the system can be configured to detect the start of a surgery. If the information about the illumination used in the environment of the surgical microscope system indicates that the ambient light is switched on and / or if the illumination system of the surgical microscope system is reduced or switched off, the system can be configured to detect the start of a surgery.

[0044] The system is configured to provide a control signal to a recording device of the surgical microscope system via the interface based on the detected start or end of the surgery. Typically, the control signal can be a digital signal, such as a packet of a digital transmission, or an analog signal, such as an analog current signal. The control signal can be adapted to trigger the recording device to start or end recording. For example, a different digital bit value can be transmitted, or a different analog voltage or current can be applied to trigger the recording device to start or stop recording. The system can be configured to provide the control signal to start recording performed by the recording device if the start of the surgery is detected, and to provide the control signal to stop recording performed by the recording if the end of the surgery is detected. Accordingly, the recording device can be configured to start or stop recording based on the control signal. The recording device can be configured to record imaging sensor data of an imaging sensor of the surgical microscope. Additionally (or alternatively), the recording device can be configured to record imaging sensor data of one or more additional imaging sensors (i.e. cameras) placed in the environment of the surgical microscope system, e.g. with a camera covering the field of view of the surgical time and / or the entire surgical site. The recording device can save a separate digital file for each recording, which digital file comprises the respective recording (e.g. within a storage module).

[0045] The interface 112 can correspond to one or more inputs and / or outputs for receiving and / or transmitting information, which information can be digital (bit) values according to a specified code within a module, between modules, or between modules of different entities. For example, the interface 112 can comprise interface circuitry configured to receive and / or transmit information. In embodiments, the one or more processors 114 can be implemented using one or more processing units, one or more processing devices, any means for processing, such as a processor, a computer or a programmable hardware component being operable with accordingly adapted software. In other words, the described functionality of the one or more processors 114 can be implemented in software, which is executed using one or more programmable hardware components. Such hardware components can include a general-purpose processor, a Digital Signal Processor (DSP), a micro-controller, etc. In at least some embodiments, the one or more storage devices 116 can comprise at least one element of the following group: a computer-readable storage medium such as a magnetic or optical storage medium for example a hard disk drive, a flash memory, a floppy disk, a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electronically erasable programmable read-only memory (EEPROM), or a network storage.

[0046] Further details and aspects of the system and the microscope system are outlined in connection with the proposed concepts or one or more of the examples described above or below (e.g. Figures 2 to 4The system and microscope system may include one or more additional optional features corresponding to one or more aspects of the proposed concept or one or more examples described above or below.

[0047] Figure 2 A flowchart of a corresponding method for controlling a recording device of a surgical microscope system is shown. The method includes obtaining 210 input data (such as...) from one or more sensors or actuators 132; 134; 142 of the surgical microscope system. Figures 1a to 1d (As shown). The method includes detecting the start or end of surgery 220 based on input data, the surgery involving the use of a surgical microscope system. The method includes providing a control signal 230 to the recording device 120 of the surgical microscope system based on the detected start or end of surgery.

[0048] As shown above, combined with Figures 1a to 1d The features described in system 110 and microscope system 100 can also be applied to Figure 2 The method.

[0049] Further details and aspects of the method, combined with the proposed concepts or one or more examples described above or below (e.g. Figures 1a to 1d As mentioned in 3a to 4), the method may include one or more additional optional features corresponding to one or more aspects of the proposed concept or one or more examples described above or below.

[0050] Various embodiments of this disclosure relate to workflows for automated case recording. Therefore, embodiments can provide concepts for automated recording (e.g., surgical record keeping).

[0051] As mentioned earlier, ophthalmology cases are often very high. Improving operating room efficiency and turnover rates increases the ability to help more patients each day. This is particularly relevant to cataract surgery. By setting automatic video recording preferences, recording can be intuitively started and stopped without any additional manual steps.

[0052] Various embodiments relate to automated recording features associated with the orientation of a parallelogram. In particular, integration of video recording functionality based on the position of the parallelogram in a "parked" or "operational" (i.e., active) mode can be provided. Thus, the start and stop of video recording can be automatically integrated as part of the surgical workflow. The surgeon can release the video recording foot switch control for use with other functions.

[0053] Figure 3a and 3b A schematic diagram of a surgical microscope system in the active and parked positions is shown. Figure 3a and 3bThe surgical microscope 140 and the parallelogram portion 136 of the arm as well as the patient 300 are shown. If the parallelogram is moved to the surgical position Figure 3a ), the parallelogram and thus the surgical microscope can be in the operational mode and video recording can be started. When the parallelogram is moved to the parking position after surgery Figure 3b , e.g. away from the patient, video recording can be stopped. A timer can be set to count down to end the case.

[0054] Further details and aspects of the concept are mentioned in connection with the proposed concept or one or more examples described above or below (e.g. Figures 1a to 2 , 4). The concept can comprise one or more additional optional features corresponding to one or more aspects of the proposed concept or one or more examples described above or below.

[0055] Some embodiments relate to a microscope comprising a system described in connection with one or more of Figs. 1 to 3b. Alternatively, the microscope can be part of or connected to a system described in connection with one or more of Figs. 1 to 3b. Figure 4 A schematic of a system 400 configured to perform the methods described herein is shown. The system 400 comprises a microscope 410 and a computer system 420. The microscope 410 is configured to take images and is connected to the computer system 420. The computer system 420 is configured to perform at least part of the methods described herein. The computer system 420 can be configured to perform a machine learning algorithm. The computer system 420 and the microscope 410 can be separate entities, but can also be integrated together in one common housing. The computer system 420 can be part of a central processing system of the microscope 410 and / or the computer system 420 can be part of a subcomponent of the microscope 410, such as a sensor, actuator, camera or illumination unit of the microscope 410, etc.

[0056] The computer system 420 can be a local computer device (e.g., a personal computer, a laptop, a tablet computer, or a mobile phone) having one or more processors and one or more storage devices, or can be a distributed computer system (e.g., a cloud computing system having one or more processors distributed in various locations, e.g., at a local client and / or one or more remote server farms and / or data centers). The computer system 420 can include any circuit or combination of circuits. In one embodiment, the computer system 420 can include one or more processors that can be any type of processors, such as, for example, microprocessors, microcontrollers, complex instruction set computing (CISC) microprocessors, reduced instruction set computing (RISC) microprocessors, very long instruction word (VLIW) microprocessors, graphics processors, digital signal processors (DSPs), multi-core processors, field programmable gate arrays (FPGAs), etc. Other types of circuits that can be included in the computer system 420 can be custom circuits, application-specific integrated circuits (ASICs), etc., such as, for example, one or more circuits (such as communication circuits) for a wireless device, such as a mobile phone, a tablet computer, a laptop computer, a two-way radio, and similar electronic systems. The computer system 420 can include one or more storage devices, which can include one or more memory elements suitable for the particular application, such as a main memory in the form of random-access memory (RAM), one or more hard drives, and / or one or more drives that handle removable media such as compact disks (CDs), flash memory cards, digital video disks (DVDs), etc. The computer system 420 can further include a display device, one or more speakers, and a keyboard and / or controller, which can include a mouse, a trackball, a touch screen, a voice recognition device, or any other device that allows a system user to input information to and receive information from the computer system 420.

[0057] Some or all of the method steps can be performed by (or using) a hardware apparatus, such as for example, a processor, a microprocessor, a programmable computer or electronic circuit. In some embodiments, some one or more of the most important method steps can be performed by such an apparatus.

[0058] Depending on certain implementation requirements, embodiments of the application can be implemented in hardware or in software. The implementation can be performed using a non-transitory storage medium such as a digital storage medium, for example a floppy disc, a DVD, a Blu-Ray, a CD, a ROM, a PROM, and an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium can be computer readable.

[0059] Some embodiments according to the application comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0060] Generally, embodiments of the present application can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code can for example be stored on a machine readable carrier.

[0061] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0062] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0063] A further embodiment of the inventive methods is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. The data carrier, the digital storage medium or the recorded medium are typically tangible and / or non-transitionary. A further embodiment of the inventive method is, therefore, a device (or apparatus) for performing one of the methods described herein. The device can be configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The device or system may, for example, include a file server for transferring the computer program to the receiver.

[0064] A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals can for example be configured to cause one of the methods described herein to be performed, when the data stream or the sequence of signals is retrieved and executed by a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like.

[0065] A further embodiment comprises a processing means, such as a computer, or a programmable logic device, configured to or adapted for performing one of the methods described herein.

[0066] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0067] A further embodiment according to the application comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The apparatus or system may, for example, include a file server for transferring the computer program to the receiver.

[0068] In some embodiments, a programmable logic device (e.g., a field programmable gate array) can be used to perform some or all of the functions of the methods described herein. In some embodiments, a field programmable gate array can cooperate with a microprocessor to perform one of the methods described herein. Generally, these methods are preferably performed by any hardware apparatus.

[0069] As used herein, the term “and / or” includes any and all combinations of one or more of the associated listed items and can be abbreviated as “ / ”.

[0070] Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.

[0071] List of reference signs

[0072] 100 surgical microscope system

[0073] 102 base unit

[0074] 104 foot brake

[0075] 106 armrest

[0076] 108 loudspeaker

[0077] 110 system

[0078] 112 interface

[0079] 114 one or more processors

[0080] 116 one or more storage devices

[0081] 118a system control unit

[0082] 118b camera and video control unit

[0083] 120 recording device

[0084] 130 arm

[0085] 132 positioning sensor

[0086] 134 control circuit of the arm

[0087] 136 parallelogram portion

[0088] 140 surgical microscope

[0089] 150 display

[0090] 152 monitor arm

[0091] 160 steering handle

[0092] 210 acquiring input data

[0093] 220 detecting a start or end of a procedure

[0094] 230 providing a control signal

[0095] 300 patient

[0096] 400 system

[0097] 410 microscope

[0098] 420 computer system

Claims

1. A system (110; 420) for controlling a recording device (120) of a surgical microscope system (100; 400), the system comprising one or more processors (114) and an interface (112), wherein the system is configured to: Input data is acquired via an interface from one or more sensors or actuators (132; 134; 142) of the surgical microscope system; The start or end of a surgery is detected based on input data, and the surgery involves the use of a surgical microscope system; and Based on the detected start or end of the surgery, control signals are provided to the recording device of the surgical microscope system via an interface; The input data includes information about the position of the arm (130) of the surgical microscope system, the arm of which includes a parallelogram portion, wherein the information about the position of the arm includes information about the orientation of the parallelogram portion of the arm, and wherein the system is configured to detect the start of the surgery when the parallelogram portion is moved to an active orientation, or the end of the surgery when the parallelogram portion is moved to a parking orientation, based on the information about the orientation of the parallelogram portion of the arm. The surgical microscope (140; 410) of the surgical microscope system is attached to the arm of the surgical microscope system, wherein the information regarding the position of the arm includes information regarding the vertical position of the surgical microscope, and the input data includes imaging sensor data of the imaging sensor (142) of the surgical microscope, wherein the system is configured to detect the end of the surgery if the vertical position of the surgical microscope remains above a threshold and the imaging sensor data is out of focus for at least a predefined time interval.

2. The system according to claim 1, wherein, The system is configured to detect the start of the surgery after the vertical position of the surgical microscope has been lowered below the threshold.

3. The system according to any one of claims 1 to 2, wherein the one or more sensors or actuators of the surgical microscope system include a positioning sensor (132) for determining the position of the arm, wherein the system is configured to obtain information about the position of the arm from the positioning sensor.

4. The system according to any one of claims 1 to 2, wherein the arm is a robotic arm, and wherein the system is configured to obtain the information about the position of the arm from the control circuit (134) of the robotic arm.

5. The system according to any one of claims 1 to 2, wherein the input data includes imaging sensor data of the imaging sensor (142) of the surgical microscope of the surgical microscope system, wherein the system is configured to detect the start or end of the surgery based on the imaging sensor data.

6. The system of claim 5, wherein the system is configured to detect the presence of a patient within the imaging sensor data, wherein the system is configured to detect the start or end of the procedure based on the presence of the patient within the imaging sensor data.

7. The system according to any one of claims 1 to 2, wherein, The input data includes information about the lighting used in the environment of the surgical microscope system, wherein the system is configured to detect the start or end of the surgery based on the information about the lighting used in the environment of the surgical microscope system.

8. The system according to any one of claims 1 to 2, wherein, The system is configured to provide the control signal to start recording to be performed by the recording device if the start of surgery is detected, and to stop recording being performed by the recording device if the end of surgery is detected.

9. A surgical microscope system (100; 400) comprising a system (110; 420) according to any one of claims 1 to 8, one or more sensors or actuators (132; 134; 142) for providing input data, and a recording device (120).

10. A method for controlling a recording device in an surgical microscope system, the method comprising: (210) Input data is acquired from one or more sensors or actuators of the surgical microscope system; Based on input data, the start or end of surgery (220) is detected, the surgery involving the use of a surgical microscope system; and Based on the detected start or end of the surgery, a (230) control signal is provided to the recording device of the surgical microscope system; The input data includes information about the position of the arm (130) of the surgical microscope system, the arm of which includes a parallelogram portion, wherein the information about the position of the arm includes information about the orientation of the parallelogram portion of the arm, and wherein the system is configured to detect the start of the surgery when the parallelogram portion is moved to an active orientation, or the end of the surgery when the parallelogram portion is moved to a parking orientation, based on the information about the orientation of the parallelogram portion of the arm. The surgical microscope (140; 410) of the surgical microscope system is attached to the arm of the surgical microscope system, wherein the information regarding the position of the arm includes information regarding the vertical position of the surgical microscope, and the input data includes imaging sensor data of the imaging sensor (142) of the surgical microscope, wherein the system is configured to detect the end of the surgery if the vertical position of the surgical microscope remains above a threshold and the imaging sensor data is out of focus for at least a predefined time interval.

11. A computer program product having program code for performing the method of claim 10 when the program code is executed on a processor.

Images