Apparatus and method for controlling a camera

Abstract

An apparatus and method for controlling a camera are provided, the apparatus having an input unit configured to acquire video data from the camera, an image processing unit configured to determine from the acquired video data whether a particular person is alone in a first area monitored by the camera, a control unit configured to generate a control signal for controlling the camera to operate in a first monitoring mode or a second monitoring mode based on a determination by the image processing unit of whether the particular person is alone in the first area monitored by the camera, and an output unit configured to output the control signal to the camera.

Description

Technical Field

[0001] The present invention relates to an apparatus and a method for controlling a camera, and more particularly, to an apparatus and a method for controlling a camera for monitoring a patient in a hospital room or a nursing facility. The present invention further relates to a corresponding system.

Background Art

[0002] In various hospital wards, camera technology is increasingly being used for remote and automatic monitoring. This camera is an unobtrusive sensor that provides a lot of information about the situation of a particular measurement. In the intensive care unit (ICU), the camera can provide added value and can be used for applications such as delirium monitoring, sleep monitoring, pressure ulcer management, fall management, and vital sign monitoring.

Summary of the Invention

Problems to be Solved by the Invention

[0003] For these applications, various aspects including subtle movements need to be robustly detected. For example, a delirious patient exhibits typical hand movements such as fidgeting with the sheets. In order to be able to detect these small movements, it is important to keep the moving object well within the field of view.

[0004] There are various camera modalities such as RGB (color), infrared or depth cameras, all of which have advantages and disadvantages for use in the ICU. The ICU environment is a dynamic environment with dynamic light conditions, the position of the bed, etc., in addition to the many activities of staff and visitors. Depth cameras have the advantage of being insensitive to light changes, such as being affected by blinking monitor lamps, casting shadows, and dark conditions at night.

[0005] Most depth cameras have a fixed focal length, but camera mounts already exist that support adjustable focal length (zoom) and mechanical movement (pan, rotation / tilt). This makes 3D depth cameras with PTZ (Pan-Tilt-Zoom) capabilities possible. PTZ cameras offer the opportunity to precisely zoom in on subtle movements, improving the detection and output of specific measurements, such as delirium scores.

[0006] U.S. Patent Application Publication No. 2020 / 0267321 discloses a method for capturing images of a scene. The current positions of multiple objects within a video frame capturing the scene, which has one or more events of interest, are determined. For at least one of these events of interest, the time and position of each of the multiple objects associated with the event of interest are predicted based on the current positions of the multiple objects. For each of the multiple frame subsets in the frame, a frame subset score is determined, and each of the multiple frame subsets includes one or more of the multiple objects based on the predicted time and predicted position of the event of interest. One of the determined frame subsets is selected based on the score of this determined frame subset. Images of the event of interest are captured using a camera, based on the camera orientation settings of the selected frame subset, and the captured images have the selected frame subset. [Means for solving the problem]

[0007] An object of the present invention is to apply the above-mentioned camera to monitor people, for example, patients in an ICU, and to further utilize options for controlling the camera to provide additional functions.

[0008] In a first embodiment of the present invention, a device for controlling a camera is shown, which device is An input unit configured to acquire video data from the aforementioned camera, An image processing unit configured to determine from acquired video data whether a specific person is alone in a first area monitored by the camera, A control unit configured to generate a control signal for controlling the camera to operate in a first monitoring mode or a second monitoring mode, based on the determination by the image processing unit as to whether the specific person is alone in a first area monitored by the camera, and Output unit configured to output the control signal to the camera The camera has a control signal configured to operate in a first monitoring mode in which the first or third area is monitored if the specific person is not alone in the first area, and to operate in a second monitoring mode in which the second area is monitored if the specific person is alone in the first area, wherein the second area is smaller than the first area and substantially includes the specific person, and the third area is larger than the second area but smaller than the first area and includes one or more persons other than the specific person who are in the first area.

[0009] In a further embodiment of the present invention, a system for monitoring a region is shown, which system A camera configured to operate in a first monitoring mode and a second monitoring mode according to a control signal, and An apparatus disclosed herein for generating a control signal and outputting the control signal to the camera. It has.

[0010] In a further embodiment of the present invention, a corresponding method, a computer program having program code means for causing the computer to perform the steps of the method disclosed herein when executed on the computer, and a non-temporary computer-readable recording medium storing the computer program for causing the method disclosed herein when executed by a processor.

[0011] Preferred embodiments of the present invention are defined in the dependent claims. The claimed methods, systems, computer programs and media are understood to have preferred embodiments similar to and / or identical to the claimed systems, as defined in the dependent claims and disclosed herein.

[0012] The present invention is based on the idea of ​​using a camera to monitor a certain area, such as an ICU room, and controlling the camera to operate in at least one of two different monitoring modes based on an evaluation of the video data acquired by the camera. Switching from one monitoring mode involves adjusting one or more of the camera's settings, such as changing the field of view, focal area, zoom ratio, etc.

[0013] In most cases, the monitoring mode is switched based on whether a particular person, for example, the patient being primarily monitored, is in a first area, for example, a room being monitored in the first monitoring mode (also called scene or room monitoring mode). If the particular person is alone in the first area (i.e., no other person is in the room), the camera switches to a second monitoring mode (also called patient monitoring mode), in which only the particular person is monitored, for example, so that the camera zooms in on the particular person (i.e., a patient lying in bed or sitting in a chair). On the other hand, if one or more other people are in the first area, the camera switches to (or remains in) the first monitoring mode, in which either the entire first area or a third area (e.g., an area where some or all of the other people are) is monitored. Thus, in embodiments of the present invention, one or more camera settings are adjusted to best suit the activity currently taking place in the room.

[0014] In realistic scenarios, the present invention is useful in preventing false or unnecessary alarms. For example, if a camera is used to detect whether a patient is at risk of falling out of bed, or to detect one or more vital signs (e.g., heart rate, respiratory rate, oxygen saturation (SpO2), etc.), and therefore the camera is focused on the patient and the bed, then an alarm will be issued (e.g., on a central monitoring screen and / or a caregiver's mobile device) if the patient moves to the edge of the bed, or if certain vital signs indicate a critical situation, as these situations would cause an alarm to be issued. However, if it is known that another person (e.g., a nurse or caregiver) is also in the room, this information can be obtained if the camera is switched to a first monitoring mode as soon as one or more other people are detected in addition to the patient, and such alarms become unnecessary and can be suppressed because the other person is already in the room, directly recognizes the patient's critical situation, and can take appropriate action.

[0015] Another scenario in which the presence of another person in the room is detected makes sense, even when the camera is used to detect delirium or epileptic seizures. Furthermore, the camera can be switched to a mode in which small movements of the patient are detected, thereby increasing its ability and reliability in detecting and recognizing movements of delirium or epileptic seizures.

[0016] In another embodiment, the focus may be on movements within the bed. Patient movements while lying in bed, such as fiddling with the bedsheets, provide important clues for detecting delirium. By detecting movement, small movements of the patient can be captured, and the camera can be zoomed in for more detail. This is done not only for better visualization but also provides better (high-resolution) input for algorithms used to further analyze these subtle movements. To detect delirium from subtle movements, methods such as those described in U.S. Patent Application Publication No. 2014 / 235969 may be used.

[0017] In one embodiment, the image processing unit is configured to determine whether a particular person is alone in a first region by detecting activity in a first region from acquired video data and determining whether that activity represents the movement of one or more people. Such an embodiment can be easily implemented and provides reliable results.

[0018] In certain embodiments, the image processing unit is further configured to determine, if one or more other persons are detected in the first region, whether one or more of those other persons are in, overlapping with, or adjacent to the second region. Furthermore, the control unit is configured to generate control signals to operate in a first monitoring mode and control the camera to monitor a third region substantially including the specific person and one or more of the other persons, if one or more of those other persons are in, overlapping with, or adjacent to the second region. Thus, if one or more persons are near a specific person, for example, if one or more caregivers or relatives are near a patient, the area around the patient is monitored so that not only the patient but also the persons adjacent to the patient are monitored in order to better understand what is happening to the patient.

[0019] The control unit is further configured to operate in a second monitoring mode or in a first monitoring mode and generate control signals to control the camera to monitor the first area if no other person is in, overlaps with, or is adjacent to the second area. Therefore, in the example of monitoring a hospital room, if no one is adjacent to the patient, the camera will either zoom in on the patient or monitor the entire room.

[0020] In another embodiment, the image processing unit is configured to detect specific furniture, particularly a bed or chair (e.g., a patient's bed in a hospital room or a lounge chair in a rest room), in the acquired video data, determine whether a specific person is on or in the furniture, and define a region substantially containing the specific furniture on or in which the specific person is as a second region. For example, specific furniture may be detected, and based on depth information in the video data, it may be determined whether a specific person is on or in the furniture. Depth information may be obtained, for example, from a 3D or depth camera.

[0021] The image processing unit is configured to correct the acquired video data by correcting the camera's field of view so that the ground plane is positioned horizontally, and to use the corrected video data to determine whether a particular person is alone in a first area. Using such correction makes it easier to determine the distance between people and objects detected in the video data, as well as to detect spatial relationships. Furthermore, this is useful for detecting specific furniture, such as a patient's bed.

[0022] In another embodiment, instead of calculating corrections to the video data, the control unit is configured to generate a control signal to control the rotation of the camera so that the ground is oriented horizontally.

[0023] In another embodiment, the image processing unit, in the second monitoring mode, performs one or more of expression analysis of the facial expression of a specific person, detection of getting out of bed, getting into bed or the risk of falling out of bed of a specific person, movement detection for detecting a specific movement of a specific person, vital sign detection for detecting one or more vital signs of a specific person, and approach detection for detecting whether one or more other persons are approaching or moving away from the specific person. Therefore, there are various options that can be used alternatively or in combination according to the user requirements, general needs or applications of the devices and methods of the present invention.

[0024] The camera of the proposed system may be configured to pan, tilt and / or zoom under the control of a control signal. For example, a 3D pan-tilt-zoom (PTZ) camera may be used. This camera may be fixedly attached to a room (e.g., ceiling or wall), or may be configured to be movably arranged in the room (e.g., using a movable stand or tripod).

[0025] Advantageously, the system may further include a fisheye camera configured to always monitor a first area or an even larger area. Therefore, in addition to the above-described cameras controlled to monitor different areas, monitoring of a first area (e.g., a hospital ward from a specific view) or an even larger area (e.g., a wider view, thus a larger area of the hospital ward) is thus always possible.

Brief Description of the Drawings

[0026] These and other aspects of the present invention will become apparent from the embodiments described below and will be described with reference to these. [Figure 1] FIG. 1 shows a schematic diagram of a system and a device according to the present invention. [Figure 2] FIG. 2 shows the situation of a realistic scenario to which the present invention is applied. [Figure 3] FIG. 3 shows the situation of a realistic scenario to which the present invention is applied. [Figure 4] Figure 4 shows a realistic scenario in which the present invention is applied. [Figure 5] Figure 5 shows a flowchart of an exemplary embodiment of the monitoring method according to the present invention. [Figure 6A] Figure 6A shows the camera view from a camera mounted on the ceiling above the patient's bed. [Figure 6B] Figure 6B shows a depth view from the same perspective as the camera view shown in Figure 6A. [Figure 7A] Figure 7A shows a camera view facing the edge of the patient bed. [Figure 7B] Figure 7B shows the corrected view after correcting the field of view. [Figure 8A] Figure 8A shows an image of the identified patient region. [Figure 8B] Figure 8B shows images of the patient area and a person adjacent to this patient area. [Figure 8C] Figure 8C shows images of only the person adjacent to the patient area. [Modes for carrying out the invention]

[0027] Figure 1 shows a schematic diagram of System 1 and Apparatus 20 according to the present invention. This system includes at least a camera 10 capable of operating in a first monitoring mode and a second monitoring mode according to a control signal, and an apparatus 20 for generating a control signal and outputting the control signal to the camera. For this purpose, the apparatus 20 acquires (e.g., receives or captures) video data acquired by the camera 10 and evaluates that data.

[0028] The device 20 has an input unit 21 for acquiring video data from the camera. The input unit 21 may also be a data interface for receiving the video data via a wired or wireless connection to the camera, for example, via a WLAN or LAN connection.

[0029] The device 20 further includes an image processing unit 22, such as a processor or computer, for determining from the acquired video data whether a specific person, such as a patient, is alone in a first area monitored by the camera 10.

[0030] The device 20 further includes a control unit 23, such as a controller, processor, or computer (for example, the same processor or computer that implements the image processing unit 22), for generating control signals to control the camera 10 to operate in a first monitoring mode or a second monitoring mode based on a decision made by the image processing unit 22.

[0031] The device 20 further includes an output unit 24 for outputting the control signal to the camera. The output unit 24 may also be a data interface for transmitting the control signal via a wired or wireless connection to the camera, for example, via a WLAN or LAN connection.

[0032] The device 20 may be implemented in hardware and / or software. For example, the device 20 may be implemented as a appropriately programmed computer or processor. Depending on the application, the device 20 may be, for example, a computer, workstation, or mobile user device, such as a smartphone, laptop, tablet, or smartwatch. For example, in an application in a hospital or nursing home, the device 20 may be implemented on a caregiver's smartphone so that the caregiver can constantly monitor the patient or obtain new monitoring information if, for example, the monitoring mode changes. In another application, the device 20 can be implemented on a computer in a central monitoring room where many patient rooms are centrally managed.

[0033] System 1 may further include a display 30 for displaying video acquired by camera 10, such as a computer monitor or a mobile user device display. Furthermore, a fisheye camera 40 may be provided to always monitor the first area or a larger area, even if camera 10 switches between different monitoring modes.

[0034] Figures 2 to 4 illustrate various situations in a realistic scenario in which the present invention is used. In this scenario, patient 2 is lying on a patient's bed 3 in a hospital room 4. A camera 10, such as a 3D PTZ camera, is mounted on the wall (or ceiling) and can pan, tilt, and rotate. An optional fisheye camera 40 is mounted in the upper corner of room 4. The device 20 and display 30 may be located in another room, such as a central monitoring room.

[0035] The control signals generated by the device 20 control the camera 10 to operate in either a first monitoring mode or a second monitoring mode. If a specific person is not alone in the first area, the first area 11 or the third area 13 is monitored in the first monitoring mode. If a specific person is alone in the first area, the second area 12 is monitored in the second monitoring mode. This is illustrated with reference to the situations illustrated in Figures 2 to 4.

[0036] In the situation shown in Figure 2, since patient 2 is alone in room 4 (e.g., ICU room), camera 10 is controlled to operate in a second monitoring mode. In this case, the second area 12 is monitored, which is substantially patient 2's own area and substantially includes patient 2 or a part of patient 2 (e.g., the patient's face or upper body).

[0037] In the situation shown in Figure 3, patient 2 is not alone in room 4, but another person 5, such as a caregiver, doctor, or visitor, is in room 4, so camera 10 is controlled to operate in first monitoring mode. In this situation, the other person 5 is in the area of ​​door 6. Therefore, the first area 11, which is the maximum field of view of camera 10, is monitored. In the scenario shown in Figure 3, the first area 11 is almost the entire room 4 as far as camera 10 can see.

[0038] In the situation shown in Figure 4, patient 2 is not alone in room 4, but another person 7, such as a caregiver, doctor, or visitor, is present in room 4, so camera 10 is controlled again to operate in the first monitoring mode. In this situation, the other person 7 is in the area of ​​bed 3. Therefore, a third area 13, which is the area around bed 3, is monitored to include patient 2 and the other person 7.

[0039] Therefore, as shown in Figures 2 to 4, the second region 12 is smaller than the first region 11 and substantially includes the specific person 2, and the third region 13 is larger than the second region 12 but smaller than the first region 11 and includes one or more people 5, 7 other than the specific person 2 who are in the first region 11.

[0040] Accordingly, according to the present invention, one or more settings of the 3D PTZ camera (e.g., focal area and / or zoom level) are automatically adapted based on current activity and scene analysis in the hospital room to achieve optimal monitoring results.

[0041] In the first monitoring mode (also called patient monitoring mode), several options exist. A flowchart of an exemplary embodiment of the monitoring method according to the present invention is shown in Figure 5. This exemplary embodiment will be described with reference to similar scenarios shown in Figures 2 to 4.

[0042] In the first step S10, the camera rotates so that the ground is flat. The bed may be segmented using a scene segmentation algorithm, and the zoom of the image is adjusted so that the bed area is in focus.

[0043] In the second step S11, it is detected whether the patient is in the bed based on the depth profile obtained from the video data. If the patient is not in the bed, the depth profile of the bed is flat. In an exemplary implementation, the distribution (histogram) of depth values ​​in the bed area is checked. If the patient is not in the bed, the camera is switched to a second (room) monitoring mode to check whether the patient is sitting in a chair.

[0044] In the third step S12, it is detected whether the patient is in a chair. This is done by checking the depth profile in the surrounding area close to the bed. The chair typically corresponds to a segmented blob by the scene segmentation algorithm. If the camera cannot find a chair with a patient, it switches to a second (room) monitoring mode.

[0045] If the patient is in bed or chair, there are various options for further monitoring. Motion estimation is performed to capture further details. The camera zooms in on areas with slight, subtle movements, particularly in the hand area (step S14). If no movement is detected in the zoomed-in area for one minute, the camera zooms out to bring the entire bed area into view (step S13).

[0046] Prioritization of zoomed-in regions may be performed. If motion is detected in two or more regions of the image with partial, subtle movements, it may be decided to zoom in based on the priority of the regions with subtle movements, and then zoom in on the region with the highest priority. Typical delirious behaviors manifest primarily through hand movements. Therefore, it is advantageous to zoom in to optimally detect hand movements. A person pose detector (e.g., based on an OpenPose detector) may be used to localize different body parts, such as the head, arms, hands, and legs.

[0047] The focus is then placed on the patient's facial region (Step S15). This allows for the analysis of facial expressions, which may provide important clues about the patient's condition, such as possible delirium, pain, and / or stress.

[0048] Alternatively, the camera may be automatically switched to a second (room) monitoring mode if it is detected that the person is getting out of or into bed (step S16).

[0049] Alternatively, another person may be detected (step S17). For example, another person may be detected entering the room in addition to the patient, particularly within the camera's field of view. If the patient is not alone in the monitoring area, the camera switches to a second monitoring mode and learns about the measurement situation. Depending on the location of the other person, the focus may be adjusted to a smaller area of ​​the scene (third area) rather than the entire room (first area).

[0050] In the second monitoring mode, if the patient is not in bed or chair, the activity (movement) level is measured (step S18). Based on that level, the camera view is zoomed and / or shifted to focus on the most intense activity or interaction.

[0051] In one embodiment, a standard 3D PTZ camera is used, mounted on the ceiling and facing downwards. A pre-calibration step is performed to estimate the camera's tilt angle (relative to the ground, such as the floor of a room), and the camera is then mechanically rotated so that the ground is flat in the camera's view. In another embodiment, the video data is computer-corrected so that the ground is positioned horizontally. Figure 6A shows the camera view from a ceiling-mounted camera above a patient's bed. Figure 6B shows a depth view from this perspective view.

[0052] In another embodiment, for the convenience of further analysis, the camera's field of view can be corrected so that the ground is always horizontal on the XY plane. Figure 7A shows a point cloud representation of the depth data shown in Figure 6B. This point cloud representation can generally be constructed from a side view and / or associated top view. It is assumed that the area from the original 3D point cloud (as shown in Figure 7A (Original view) with the field of view pointed towards the edge of the patient's bed and two people standing next to the bed) up to 0.5 meters above the farthest point (or any other appropriate value pre-set or estimated for a particular view) constitutes the approximate ground pixels. These selected pixels are used to align the 3D ground in a robust manner, such as a technique called RANSAC (RANDOM Sample Consensus). RANSAC-based estimation represents a very robust solution due to its ability to handle outliers. Based on the estimated plane parameters, the camera is rotated so that the ground is on the XY plane (as shown in Figure 7B (Corrected view after camera rotation)). The Z-values ​​of the new point cloud reflect the true height of scene objects above the ground. Note that other methods may be applied to achieve such corrections instead of the RANSAC algorithm.

[0053] To determine what to monitor in the room, it is determined whether another person is in the room or whether the patient is alone in the room. In one embodiment, this is done as follows:

[0054] In the first step, an ROI (Region of Interest) detection algorithm is applied to outline the patient region, for example, as shown in Figure 8A. This can be done, for example, using the method described in International Patent Application No. 2020 / 127014. From the depth profile of the patient region (referred to herein as the second region), it can be determined whether the patient is in bed or not. If the patient is in bed, the following steps are performed to determine whether the patient is alone in the room.

[0055] Motion maps are generated from video data, for example, based on frame differences, H3DRS, or optical flow. If a detected motion area (outside the patient area) is not adjacent to the patient area, it can be assumed that another person is in the room, but they are not physically interacting with the patient.

[0056] If the motion region intersects with the patient region (for example, as shown in Figures 8B and 8C), the relevant actual depth values ​​of these motion regions are further checked. Based on the above, it can be determined whether there is any interaction between the patient region and another person (e.g., a nurse, doctor, visitor, etc.). The camera is then controlled to zoom out so that the patient and the other person interacting with the patient region are within the field of view; that is, the camera is controlled to monitor a third region larger than the patient region.

[0057] If the aforementioned movement is outside the patient area, the camera can be controlled to switch to patient monitoring mode (second monitoring mode) and zoom in on the patient area.

[0058] If no patient is detected in the bed area, other detected connected parts surrounding this bed area (e.g., blobs) are further checked to locate the presence of a possible chair. This can be confirmed again by the shape and depth profile of the blob. If the patient is in a chair, the same logic as described above for the situation where the patient is in bed can be followed.

[0059] Many different algorithms are commonly known for detecting activity or motion from video data, such as algorithms that use background subtraction (which determines the difference between the current video frame and a reference or preceding video frame), or algorithms that use optical flow-based models. Similarly, known algorithms can be applied to determine whether activity represents human movement, such as algorithms that evaluate the texture, shape, and motion patterns of the image region indicating activity. Various algorithms are described, for example, in "Paul, M., Haque, SME & Chakraborty, S. Human detection in surveillance videos and its applications - a review. EURASIP J. Adv. Singan Process. 2013, 176 (2013)." Distinguishing between different individuals can be achieved, for example, by detecting whether the area containing activity is clearly separated.

[0060] When a patient is alone in a room, one or more of the following analytical features or modes are applied for real-time monitoring. These can be activated by manually selected or automatically detected events.

[0061] In one embodiment, full-bed monitoring may be performed. In this operation, the camera's zoom level is adjusted so that the bed area occupies the majority of the field of view (e.g., 80%). An example is shown in Figure 8A. This operation can be used as the default operation.

[0062] In another embodiment, in-bed motion focusing may be performed. Patient movements while lying in bed, such as fiddling with the bedsheets, provide important clues for detecting delirium. By detecting motion, small movements of the patient are captured, and the camera is zoomed in for more detail. This is done not only for better visualization but also provides better (high-resolution) input for algorithms used to further analyze these subtle movements. To detect delirium from subtle movements, methods such as those described in U.S. Patent Application Publication No. 2014 / 235969 may be used.

[0063] In another embodiment, facial expression analysis may be performed. Facial expressions are one of the important communication signals for patients in the ICU. For this operation, an automated face detection algorithm, such as that described in "Weon SH., Joo SI., Choi HI. (2014) Using Depth Information for Real-Time Face Detection. In: Stephanidis C. (eds) HCI International 2014 - Posters' Extended Abstracts. HCI 2014, Communications in Computer and Information Science, vol 434, Springer," can be used to locate facial regions. The PTZ camera can then zoom in on these regions. These images can be used for visual inspection or fed into the automated facial expression analysis algorithm. This operation can be manually selected or activated by facial movements in a manner similar to that described above.

[0064] In another embodiment, detection of entering / exiting the bed may be performed. The bed boundary is constantly monitored to detect any event of entering / exiting the bed. This is achieved, for example, by checking the direction of movement across the bed boundary, either from outside to inside or inside to outside. When an event of entering / exiting the bed is detected, the camera is controlled to a zoom level to monitor the entire room. To detect entering / exiting the bed, methods described, for example, in U.S. Patent Application Publication 2019 / 228866 or U.S. Patent Application Publication 2019 / 192052 may be used.

[0065] In another embodiment, the entry / exit of another person (e.g., a nurse or visitor) may be detected. It may also be detected when another person enters the image following the patient. Once this is detected, the camera can adjust its zoom operation accordingly.

[0066] To continuously monitor all activity in the room, the camera may be combined with, for example, a fisheye camera, so that all activity in the room can be recorded. Images from such additional cameras are analyzed for hot spots of motion. The additional cameras provide this information for the analysis and are used to switch back from patient monitoring mode to room monitoring mode (first monitoring mode).

[0067] For a single PTZ camera, digital zoom is used to focus on an area of ​​interest while using the original maximum resolution image to continuously monitor the entire room.

[0068] The present invention provides a camera-based function for monitoring ICU rooms, but can also be used in general wards, geriatric wards, and other medical facilities that utilize camera monitoring. Furthermore, the present invention also enables the automatic characterization of patient movements in the ICU. Further options provided by the present invention include delirium detection, vital sign monitoring, pressure ulcer management, and fall management.

[0069] Although the present invention has been illustrated and described in detail in the drawings and the above description, such illustrations and descriptions should be considered descriptive or illustrative and not limiting; that is, the present invention is not limited to the disclosed embodiments. Other variations of the disclosed embodiments can be understood and practiced by those skilled in the art in carrying out the claimed invention by examining the drawings, this disclosure and the appended claims.

[0070] In a claim, the word “having” does not preclude other elements or steps, nor does it preclude multiple elements or steps if it is not stated that there are multiple. A single element or other unit may perform the functions of several items enumerated in a claim. The mere fact that certain means are described in different dependent claims does not imply that combinations of these means cannot be used advantageously.

[0071] Computer programs may be stored / distributed on suitable non-temporary media, such as optical or solid-state storage media provided together with or as part of other hardware, or they may be distributed in other forms, such as via the Internet or other wired or wireless communication systems.

[0072] No reference numeral in a claim should be construed as limiting its scope.

Claims

1. A device for controlling a camera, wherein the device is An input unit configured to acquire video data from the aforementioned camera, An image processing unit configured to determine from the acquired video data whether a specific person is alone in a first area monitored by the camera, A control unit configured to generate a control signal for controlling the camera to operate in a first monitoring mode or a second monitoring mode, based on the determination by the image processing unit as to whether the specific person is alone in a first area monitored by the camera, and Output unit configured to output the control signal to the camera A device having a control signal configured to control the camera to operate in a first monitoring mode in which the first or third area is monitored if the specific person is not alone in the first area, and in a second monitoring mode in which the second area is monitored if the specific person is alone in the first area, wherein the second area is smaller than the first area and substantially includes the specific person, and the third area is larger than the second area but smaller than the first area and includes one or more persons other than the specific person in the first area.

2. The apparatus according to claim 1, wherein the image processing unit is configured to determine whether a particular person is alone in the first region by detecting activity in the first region from the acquired video data and determining whether the activity represents the movement of one or more people.

3. The image processing unit is configured to determine, when one or more other persons are detected in the first region, whether one or more of those other persons are in the second region, overlapping with the second region, or adjacent to the second region, and The apparatus according to claim 2, wherein the control unit is configured to operate in the first monitoring mode and generate a control signal to control the camera to monitor the third area which substantially includes the specific person and one or more of the other persons, when one or more of the other persons are in the second area, overlapping with the second area, or adjacent to the second area.

4. The apparatus according to claim 3, wherein the control unit is configured to operate in the first monitoring mode and generate control signals to control the camera to monitor the first area if none of the other persons are in the second area, do not overlap with the second area, or are not adjacent to the second area.

5. The apparatus according to any one of claims 1 to 4, wherein the image processing unit is configured to detect specific furniture in the acquired video data, determine whether a specific person is on or in the furniture, and define a region substantially including the specific furniture in which the specific person is on or in as a second region.

6. The apparatus according to claim 5, wherein the image processing unit is configured to detect the specific furniture and determine, based on the depth information of the video data, whether the specific person is on or in the furniture.

7. The apparatus according to any one of claims 1 to 6, wherein the image processing unit is configured to correct acquired video data by correcting the field of view of the camera so that the ground is positioned horizontally, and to use the corrected video data to determine whether a particular person is alone in the first area.

8. The apparatus according to any one of claims 1 to 7, wherein the control unit is configured to generate a control signal for controlling the rotation of the camera so that the ground is positioned horizontally.

9. The apparatus according to any one of claims 1 to 8, wherein the image processing unit is configured to perform one or more of the following in the second monitoring mode: facial expression analysis of the specific person's facial expression; bed exit detection to detect whether the specific person is getting into or out of bed, or whether there is a risk of falling out of bed; motion detection to detect specific movements of the specific person; vital sign detection to detect one or more vital signs of the specific person; and approach detection to detect whether one or more other people are approaching or moving away from the specific person.

10. A system for monitoring an area, A camera configured to operate in a first monitoring mode and a second monitoring mode according to a control signal, and An apparatus according to any one of claims 1 to 9 for generating a control signal and outputting the control signal to the camera. A system that has

11. The system according to claim 10, wherein the camera is configured to pan, tilt and / or zoom under the control of the control signal.

12. The system according to claim 10 or 11, wherein the camera is configured to be fixedly mounted in the room or to be movable within the room.

13. The system according to claim 10, 11, or 12, further comprising a fisheye camera configured to continuously monitor the first region or a larger region.

14. A method for controlling a camera, wherein the method is A step of acquiring video data from the aforementioned camera, A step of determining from the acquired video data whether a specific person is alone in a first area monitored by the camera, A step of generating a control signal to control the camera to operate in a first monitoring mode or a second monitoring mode, based on the determination of whether the specific person is alone in the first area monitored by the camera. step of outputting the control signal to the camera A method comprising, wherein the control signal controls the camera to operate in a first monitoring mode in which the first or third area is monitored if the specific person is not alone in the first area, or in a second monitoring mode in which the second area is monitored if the specific person is alone in the first area, the second area being smaller than the first area and substantially including the specific person, and the third area being larger than the second area but smaller than the first area and including one or more persons other than the specific person in the first area.

15. A computer program having program code means for causing a computer to perform the steps of the method according to claim 14 when executed on the computer.

Images