Systems and methods for guiding an adaptive camera system

Abstract

Systems and methods for guiding an adaptive camera system are disclosed. The disclosed computer-implemented method for guiding an adaptive camera system may include: (i) monitoring a person within a physical environment via a camera that is part of a camera system; (ii) detecting, by the camera system, a gesture performed by the person; (iii) determining, by the camera system, that the gesture indicates an element of the physical environment; and (iv) performing, by the camera system, an action directed toward an element of the physical environment in response to the camera system determining that the gesture indicates an element of the physical environment. Various other methods, systems, and computer-readable media are also disclosed.

Description

Background Art

[0001] Many devices, from desktop computing systems to mobile devices (such as tablets and smartphones) to wearable systems (such as smart watches and gaming headsets), include cameras. Users can use the cameras on their devices for a variety of activities, such as conference calls for business purposes, video calls to friends and family, recording images or videos to post to social media sites, creating video tutorials of various processes, capturing video as part of an augmented reality system, or any other of a multitude of activities. Whatever the activity, it is essential that the subject is recorded and in focus within the camera frame.

[0002] Traditionally, cameras have a fixed camera frame that can be changed by physically moving the camera or by changing settings on the camera to pan, zoom, or otherwise change the camera frame. Manually changing the camera's settings may not be ideal if the user is engaged in some activity that takes up the user's hands or attention. Accordingly, the present disclosure identifies and addresses the need for systems and methods for directing an adaptive camera system. Summary of the invention

[0003] As will be described in more detail below, the present disclosure describes systems and methods for directing an adaptive camera system via intuitive gestures that are already understood by the user.

[0004] In one instance, a computer-implemented method for directing an adaptive camera system may include: (i) monitoring a person within a physical environment via a camera that is part of the camera system; (ii) detecting, by the camera system, a gesture performed by the person; (iii) determining, by the camera system, that the gesture indicates an element of the physical environment; and (iv) performing, by the camera system, an action directed toward an element of the physical environment in response to the camera system determining that the gesture indicates an element of the physical environment.

[0005] In one embodiment, the gesture may include a human-readable gesture. In one example, the gesture may indicate an element of the physical environment to another person.

[0006] In one embodiment, the action performed by the camera system may include determining that the gesture is associated with a predetermined action from a list of actions associated with gestures. In some instances, performing the action may include physically moving the camera by the camera system.

[0007] Additionally or alternatively, performing the action may include adjusting a camera frame of the camera. In some instances, performing the action may include moving the camera frame of the camera to include within the camera frame an element of the physical environment indicated by the gesture. In some instances, performing the action may include adjusting the camera frame of the camera to increase the prominence of the element of the physical environment indicated by the gesture within the camera frame. In some instances, performing the action may include changing the focus of the camera to the element of the physical environment indicated by the gesture.

[0008] In some instances, performing the action may include retrieving, by the camera system, information about the element of the physical environment indicated by the gesture. In some instances, performing the action may include displaying information about the element of the physical environment indicated by the gesture. Additionally or alternatively, performing the action may include storing, by the camera system, information about the element of the physical environment indicated by the gesture.

[0009] In one embodiment, the gesture may include a pointing gesture at an element of the physical environment, and the element of the physical environment may include an object. Additionally or alternatively, the gesture may indicate a portion of the physical environment, and determining by the camera system that the gesture indicates an element of the physical environment may include examining, by the camera system, the portion of the physical environment indicated by the gesture.

[0010] In some examples, the gesture may include a hand gesture, and monitoring the person within the physical environment via a camera that is part of the camera system may include motion tracking the person's hands. Additionally or alternatively, detecting, by the camera system, a gesture performed by the person may include detecting an audio trigger in an audio feed of the camera system and detecting the gesture in response to detecting the audio trigger.

[0011] In one example, the camera system may include an augmented reality system, and performing the action may include the augmented reality system augmenting the physical environment with augmented reality elements corresponding to the elements of the physical environment. In another example, the camera system may include a video conferencing system, monitoring people within the physical environment may include: monitoring a participant in a video conference, the video conference including at least one additional participant that does not share the physical environment with the participant; and performing the action may include increasing visibility of the element of the physical environment within a video feed displayed by the camera system to the additional participant in the video conference.

[0012] In one embodiment, a system for implementing the above method may include several modules stored in a memory, including: (i) a monitoring module that monitors a person in a physical environment via a camera that is part of a camera system; (ii) a detection module that detects a gesture performed by a person through the camera system; (iii) a determination module that determines through the camera system that the gesture indicates an element of the physical environment; (iv) an action module that performs an action directed toward an element of the physical environment through the camera system in response to the camera system determining that the gesture indicates an element of the physical environment; and (v) at least one physical processor that executes the monitoring module, the detection module, the determination module, and the action module.

[0013] In some instances, the above methods may be encoded as computer-readable instructions on a computer-readable medium. For example, a computer-readable medium may include one or more computer-executable instructions that, when executed by at least one processor of a computing device, may cause the computing device to: (i) monitor a person within a physical environment via a camera that is part of a camera system; (ii) detect, by the camera system, a gesture performed by the person; (iii) determine, by the camera system, that the gesture indicates an element of the physical environment; and (iv) perform, by the camera system, an action directed toward an element of the physical environment in response to the camera system determining that the gesture indicates an element of the physical environment.

[0014] Features from any of the above-described embodiments may be used in combination with each other according to the general principles described herein. These and other embodiments, features and advantages will be more fully understood by reading the following detailed description in conjunction with the accompanying drawings and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings illustrate several exemplary embodiments and are a part of the specification. Together with the following description, these drawings demonstrate and explain various principles of the present disclosure.

[0016] Figure 1 is a flow chart of an exemplary method for directing an adaptive camera system.

[0017] Figure 2 is an illustration of an exemplary system for directing changes to a camera frame for an adaptive camera system.

[0018] Figure 3 is an illustration of an exemplary system for physically moving a camera for directing an adaptive camera system.

[0019] Figure 4 is an illustration of an exemplary system for directing an adaptive camera system to retrieve information about elements of a physical environment.

[0020] Figure 5is an illustration of an exemplary system for directing an adaptive camera system to record information about elements of a physical environment.

[0021] Figure 6 is an illustration of an exemplary system for guiding the interaction of an adaptive camera system with an augmented reality element.

[0022] Figure 7 is an illustration of an exemplary system for directing an adaptive camera system to perform actions related to elements of a physical environment.

[0023] Figure 8 is an illustration of an exemplary system for directing an adaptive camera system via gestures that includes an audio component.

[0024] Fig. 9 is an illustration of an exemplary system for directing an adaptive camera system to perform an action related to a person performing a gesture.

[0025] Fig.10 is an illustration of an exemplary system for directing an adaptive camera system to perform actions related to elements of a physical environment.

[0026] Fig.11 is an illustration of an exemplary system for guiding an adaptive camera system via two-hand gestures.

[0027] Fig.12 is a block diagram of an exemplary system for guiding an adaptive camera system.

[0028] Throughout the drawings, the same reference numerals and descriptions indicate similar but not necessarily identical elements. Although the exemplary embodiments described herein are susceptible to various modifications and alternative forms, specific embodiments are shown by way of example in the drawings and will be described in detail herein. However, the exemplary embodiments described herein are not intended to be limited to the specific forms disclosed. Instead, the present disclosure encompasses all modifications, equivalents, and alternatives that fall within the scope of the appended claims. DETAILED DESCRIPTION

[0029] The present disclosure generally relates to systems and methods for guiding an adaptive camera system. As will be explained in more detail below, by guiding the adaptive camera system in response to gestures made by a user, the systems and methods described herein can enable a user to guide the camera system without manually changing the camera's settings or failing due to counterproductive autofocus mechanical structures. By using intuitive human-readable gestures, the systems and methods described herein can enable a user to guide an adaptive camera system without the user having to spend time learning a set of manual gestures specifically designed for guiding the camera system. In addition, the systems and methods described herein can improve the functionality of a computing device by improving the responsiveness of the computing device to user intent. These systems and methods can also improve the field of video conferencing and / or augmented reality by allowing a user to more effectively guide an adaptive camera system during a video conference and / or augmented reality experience.

[0030] The following will refer to Figure 1 A detailed description of an exemplary method for guiding an adaptive camera system is provided. Figures 2 to 11 A detailed description of an exemplary system for guiding an adaptive camera is provided. Fig.12 A detailed description of an exemplary system for guiding an adaptive camera system is provided.

[0031] Figure 1 is a flow chart of an exemplary computer-implemented method 100 for directing an adaptive camera system. Figure 1 The steps shown in the figure may be executed by any suitable computer executable code and / or computing system (including Fig.12 In one example, Figure 1 Each step shown in may represent an algorithm whose structure includes and / or is represented by multiple sub-steps, examples of which are provided in more detail below.

[0032] like Figure 1 As shown in , at step 110 , one or more systems described herein may monitor a person within a physical environment via a camera that is part of a camera system.

[0033] In some instances, the term "camera" as used herein may refer to a hardware component of a camera system that captures visual information. In some embodiments, a camera may capture still images. Additionally or alternatively, a camera may capture video. In some instances, a camera may have a field of view that includes everything visible to the camera. In one instance, a camera (or a subsystem that processes input from a camera) may define a camera frame that includes everything that the physical camera is currently capturing. In another instance, a camera system may define a camera frame that includes a subset of the area captured by the camera hardware that is processed and / or saved by a camera application. Additionally or alternatively, a camera system may define a frame as a subset of the area captured by the camera that is displayed on a viewing screen. In some embodiments, a camera and / or camera system may have multiple camera frames.

[0034] In some instances, the term "camera system" as used herein may refer to a combination of hardware and / or software components that include and / or direct a camera. In some embodiments, the camera system may control camera settings, such as field of view, camera frame, and / or focus. In one embodiment, the camera system may include one or more visual displays that display information from the camera to one or more users. For example, the camera system may display information on a screen of a fixed computing device, a screen of a mobile computing device, a projector screen, and / or an augmented reality helmet. In some embodiments, the camera system may include and / or communicate with one or more applications (including but not limited to video conferencing applications, video recording applications, social media applications, and / or augmented reality applications).

[0035] In some instances, the term "physical environment" as used herein may refer to a physical location that includes a camera system and at least one person. Examples of physical environments may include, but are not limited to, conference rooms, classrooms, homes, and / or outdoor areas. In some instances, the physical environment may include multiple people, not all of whom may be users of the camera system. In some instances, the user of the camera system may be a person with a user account associated with the camera system, and / or a person who owns, manages, and / or controls the camera system. For example, the physical environment may include a presenter who is giving a lecture and is a user of the camera system, and one or more attendees who are not users of the camera system. In another instance, the physical environment may include multiple presenters, only one of which is registered as a user of the camera system.

[0036] The systems described herein may perform step 110 in a variety of ways. In one example, the systems described herein may monitor a predetermined user of the camera system. For example, an augmented reality helmet may monitor only the wearer of the augmented reality helmet. In other examples, the systems described herein may monitor anyone within the field of view of the camera system. For example, a camera system in a conference room may monitor anyone standing in front of a whiteboard and / or anyone within the conference room, regardless of whether the person is registered as a user of the camera system.

[0037] In some implementations, the systems described herein may monitor a video feed of a camera system. For example, the systems described herein may monitor a video feed to detect whether an action performed by a person within the field of view of the camera system matches a gesture in a set of predetermined gestures that are meaningful to the camera system. In one implementation, the systems described herein may track the movement of a hand of at least one person in the field of view of the camera system in order to detect a hand gesture made by the person.

[0038] Additionally or alternatively, the systems described herein may monitor the audio feed of the camera system. In some instances, the systems described herein may monitor the audio feed for a prompt that reminds the systems described herein to check the video feed for a gesture. For example, the systems described herein may monitor the audio feed for the sound of a finger snap, and then may prompt the systems described herein to check the video feed to determine whether a person has snapped their fingers, and if so, where the finger snap gesture occurred. In some embodiments, the systems described herein may identify the approximate location of the finger snap gesture based on the audio prompt of the finger snap sound. These systems may then search for a portion of the video feed corresponding to the approximate location of the finger snap gesture and / or may move the camera in the direction of the finger snap sound, which may not originate from a finger snap gesture performed in an area currently covered by the video feed. In some instances, the audio prompt may remind the systems described herein of the type of gesture. For example, the sound of a finger snap may remind the systems described herein to check the video feed for a finger snap gesture, and the sound of a hand clap may remind the systems described herein to check the video feed for a hand clap gesture. In some embodiments, audio prompts can change how the systems described herein interpret gestures. For example, the systems described herein can interpret a pointing gesture accompanied by a verbal exclamation such as "over here!" as being more important, more urgent, and / or indicating a different action than a pointing gesture not accompanied by an exclamation.

[0039] At step 120 , one or more of the systems described herein may detect a gesture performed by a person as part of and / or in conjunction with a camera system.

[0040] In some instances, the term "gesture" as used herein may refer to any automatic, instinctive and / or intentional action made by a person. In one instance, a gesture may be a hand gesture performed with one or two hands, including but not limited to pointing, snapping, waving, beckoning, shushing and / or framing. In some instances, a gesture may be a static gesture, such as pointing to an object. In other instances, a gesture may include further actions, such as drawing a circle around an area with a pointing finger. In some instances, a gesture may include letters, words or phrases in a sign language (such as American Sign Language). In some embodiments, a gesture may include an auditory component, such as the noise of snapping or clapping. In one embodiment, a gesture may be an arm gesture, such as waving or drawing a circle with a forearm or an entire arm. In some embodiments, a gesture may be a human-readable gesture. That is, a gesture may be a gesture that is meaningful and / or understandable to others (such as pointing to an object), rather than a gesture that is specifically learned by a person to communicate with a user interface but is not usually used by a person to communicate with others (such as manipulating a virtual mouse and / or keyboard). In some instances, gestures may indicate elements of the physical environment to another person. For example, a person pointing at a dog may be performing a gesture that indicates the dog as an element of interest to the camera system and any viewers.

[0041] The systems described herein may perform step 120 in a variety of ways. In one instance, the systems described herein may detect gestures using motion tracking technology that compares the movement of a person and / or a part of a person's body to a set of recognized gestures. In some embodiments, the systems described herein may detect gestures by detecting an audio trigger in an audio feed of a camera system, and may then detect the gesture in response to detecting the audio trigger. For example, the systems described herein may detect the sound of snapping fingers, and may then visually detect that a person is snapping their fingers with their left hand. In another instance, the systems described herein may detect the sound of a person whistling, and may then detect that a person (in some instances, the same person as the whistle, but in other instances, a different person) is pointing their finger at an object.

[0042] At step 130 , one or more of the systems described herein may determine, as part of and / or in conjunction with a camera system, that the gesture is indicative of an element of the physical environment.

[0043] In some instances, the term "element of the physical environment" or "element" as used herein may refer to any object, area, person, animal, and / or other feature of the physical environment. In one instance, an element of the physical environment may be a discrete physical object, such as a set of car keys. In other instances, an element of the physical environment may be part of another object and / or an area of ​​interest, such as part of a whiteboard containing text and / or drawings. In some instances, an element may be a person (such as a person performing a gesture, or another person indicated by a gesture).

[0044] The systems described herein can perform step 130 in various ways. In some instances, the systems described herein can determine that a gesture is a human-readable gesture that indicates an element and / or is associated with one or more predetermined actions by comparing the gesture to a list of gestures. In some instances, the list can be manually specified by a developer and / or user of the camera system, for example, by compiling a list of gestures and / or by marking relevant gestures in a catalog of possible gestures. In some embodiments, the systems described herein can compile a list of gestures based at least in part on user feedback (e.g., after a video conference) about elements leaving the camera frame, being out of focus, and / or other situations that are not adequately captured in an image and / or video. In some instances, the feedback can specify a gesture that indicates an element, while in other cases, the feedback may not include information about the gesture.

[0045] In some embodiments, the systems described herein may apply machine learning techniques to determine which gestures indicate elements of the environment to others. Additionally or alternatively, the systems described herein may apply machine learning techniques to determine in which direction, with what responsiveness, and / or in what manner a gesture indicates an element. In one embodiment, the systems described herein may determine that a gesture appears to be directed toward an element to attract the attention of another person (e.g., in the environment or on the other side of a video call) based on voice, eye contact, a person orienting a gesture toward another person, and / or other cues.

[0046] In one instance, the system described herein can determine the type of element indicated by a gesture based on the type of gesture. For example, a pointing gesture can indicate a discrete object, while a circle gesture can indicate an area and / or portion of an object. In another instance, a waving gesture can indicate that the element is a person performing the gesture. In some embodiments, the system described herein can determine the characteristics of an element based on the intensity of the gesture. For example, the system described herein can determine that a wide waving gesture indicates a larger area than a more constrained waving gesture involving less arm rotation. In another instance, the system described herein can determine that a fully extended arm with a pointing gesture and / or a raised arm with a pointing gesture indicates a farther element than a pointing gesture without a fully extended arm.

[0047] In some implementations, the systems described herein can examine the area immediately surrounding and / or indicated by the gesture to determine the elements of the environment indicated by the gesture. For example, if the gesture is a pointing gesture, the systems described herein can examine the physical environment in the direction of the pointing gesture. In another example, if the gesture is a framing gesture, the systems described herein can examine the area within the box created by the gesture.

[0048] In some instances, a gesture may indicate a portion of a physical environment, and the camera system may determine that the gesture indicates an element of the physical environment by examining the portion of the physical environment indicated by the gesture. In one instance, if the gesture is a waving gesture, the system described herein may examine an area in the direction of the waving gesture to identify the element indicated by the gesture. In some instances, the system described herein may determine that an area immediately surrounding the gesture is the element indicated by the gesture. For example, a snap gesture may indicate an area around the snap gesture.

[0049] At step 140 , in response to determining that the gesture indicates an element of the physical environment, one or more of the systems described herein may, as part of and / or in conjunction with the camera system, perform an action directed toward the element of the physical environment.

[0050] The systems described herein can perform various actions. Examples of actions can include, but are not limited to, panning a camera, tilting a camera, zooming a camera, refocusing a camera, adjusting the resolution and / or quality of a live video stream and / or recording, adjusting a gain control, adjusting a color balance, taking a picture, recording a video, saving information about an element of a physical environment, retrieving information about an element of a physical environment, displaying information about an element of a physical environment, and / or annotating a display of an element of a physical environment.

[0051] In some embodiments, the system described herein may determine that a gesture is associated with an action from a predetermined list of actions. For example, the system described herein may determine that a waving gesture is associated with a camera zooming in, while a shushing gesture is associated with a camera zooming out. In another example, the system described herein may determine that a framing gesture is associated with photographing an area within a frame made by a person's hand, while a pointing gesture is associated with focusing the camera on an object pointed at by the gesture. In some examples, the subject of the action may determine which gesture is associated with the action. For example, the system described herein may determine that a pointing action at an object is associated with zooming in on the object, while a pointing gesture at a person is associated with selecting the person being pointed at as a new subject of interest (e.g., indicating to the camera system that the gesture of that person is to be focused on instead of the original person's gesture, and / or in addition to the original person's gesture).

[0052] In one example, the systems described herein may center a camera frame around an element indicated by a gesture and / or move the camera frame of a camera to include within the camera frame an element of the physical environment indicated by the gesture. In some implementations, a camera system may have a field of view that is much wider than a camera frame that actively displays and / or records visual information, and may adjust the camera frame within the field of view in response to a gesture. For example, Figure 2 As shown in , the camera frame 202 may contain only a portion of the camera system's field of view, which may cover the entire whiteboard as well as the window. In this example, a person 204 may perform a pointing gesture 206 to indicate an element 208 (an equation on the whiteboard). In some embodiments, the system described herein may center the camera frame 202 on the element 208 in response to the gesture 206. By adjusting the camera frame 202 to center on the element 208 in response to the gesture 206, the system described herein may enable the lecturer to continue the lecture with the camera focused on key materials without having to pause the lecture to adjust the camera. By using human-readable gestures such as pointing to prompt the camera system, the system described herein may enable the lecturer to direct the camera without having to perform any actions that distract from the lecture. Additionally or alternatively, the camera system may add an additional frame centered on the element 208 in response to the gesture 206. In some examples, the camera system may display the additional frame in a view screen alongside the camera frame 202. In one example, the camera system may display the additional frame in an inset within the camera frame 202 display. In some examples, the camera system can enable a user to switch between views of camera frame 202 and additional camera frames. In some implementations, the camera system can record the contents of both camera frames.

[0053] In some embodiments, the camera system can physically move the camera. Figure 3, people 304 can make pointing gesture 306 to indicate element 308 (dog performing performance). In this example, camera 302 may not be centered on element 308 at first, but can be physically moved to make the camera frame centered on element 308. In some embodiments, the system described herein can perform multiple actions in response to determining the gesture indicating element. For example, the system described herein can focus camera 302 on element 308 and save the picture of element 308 in response to people 304 indicating element 308 with gesture 306. In some embodiments, the system described herein can create a custom action chain associated with gesture. For example, in response to gesture 306, the system described herein can focus camera 302 on element 308, save the picture of element 308, and then publish the picture to the social media account associated with people 304.

[0054] In some examples, the systems described herein can retrieve information about elements of the physical environment indicated by a gesture through a camera system. Figure 4 , person 404 may wear necklace 402 including a camera system. In some embodiments, necklace 402 may be part of an augmented reality system. Additionally or alternatively, necklace 402 may be a piece of smart jewelry that includes various hardware and / or software components (such as a recorder, a network connection, and / or an artificial intelligence routine). In one example, person 404 may make a gesture 406 to element 408. In some examples, a camera system operating within and / or in communication with necklace 402 may determine that gesture 406 is associated with retrieving information about element 408. In one example, the camera system may take a picture of element 408, compare the picture to pictures of other birds, determine the species of the bird and / or other characteristics of the bird (such as range and / or habits), then store information about the bird, relay information about the bird to person 404 (e.g., vocally, by displaying information on a screen of an additional device used by person 404, and / or via some other channel), and / or perform some other action with the retrieved information about the bird.

[0055] In some examples, the systems described herein can perform actions by storing, by the camera system, information about elements of the physical environment indicated by the gesture. Figure 5, a person 504 wearing a necklace 502 equipped with a camera system can make a gesture 506 indicating an element 508. In one example, the camera system can store information about the element 508, such as a picture of the element, a label and / or keyword describing the element, the location of the element, and / or the time when the element was indicated. In one example, the person 504 can later ask the camera system for a list of all elements indicated during a specified time span (such as during the last hour or during the day), and the camera system can provide information about all of these elements to the person 504. In another example, the person 504 can later ask the camera system with a directional query (such as, "Where did I put my keys?"), and the camera system can retrieve the stored information about the element 508 to convey to the person 504 that the keys are on the desk in the office.

[0056] In one embodiment, the camera system may include and / or may be part of an augmented reality system, and performing the action may include the augmented reality system augmenting the physical environment with augmented reality elements corresponding to elements of the physical environment. Figure 6 As shown in , person 604 can wear augmented reality helmet 602, which can include an external camera and an internal visual display. In one instance, person 604 can make a gesture 606 to element 608 corresponding to an augmented reality element within the augmented reality system. In some instances, helmet 602 can display an overlay 610 including information about element 608 on the internal visual display. For example, if the augmented reality system is a tour of a historically important monument, overlay 610 can include historical information about the monument. In another instance, if the augmented reality system is a game in which participants capture targets to earn points, overlay 610 can include information about how to capture element 608, the point value of element 608, and / or the current in-game owner of element 608. In one embodiment, overlay 610 can simply be a translucent red block indicating that the red team currently controls element 608.

[0057] In some examples, the systems described herein may adjust the camera frame of a camera to increase the prominence of elements of the physical environment indicated by the gesture within the frame. Figure 7, person 704 can perform gesture 706 by drawing a circle around element 708. In this example, element 708 can be the entire area circled by gesture 706 and / or specifically the equation circled by gesture 706. In one embodiment, the camera system can center the camera frame on element 708. In another embodiment, the camera system can zoom in the camera on element 708. Additionally or alternatively, the camera system can bring element 708 into the focus of the camera (e.g., by changing the focus of the camera from person 704 to element 708). In some embodiments, the camera system can perform multiple actions in response to a particular gesture. For example, the camera system can zoom in the camera on element 708 and also display additional information about element 708, for example, by annotating a streaming video of a lecture with a link to a wiki page explaining the equation.

[0058] In some implementations, a gesture may include an audio component and / or an audio trigger may prompt the camera system to detect the gesture. Figure 8 , person 804 can be working on an artifact that includes element 808. In this example, person 804 can snap their fingers to perform gesture 806, prompting a camera system within tablet computer 802 to focus the camera on element 808. In this example, by focusing the camera system in response to gesture 806, the systems described herein can enable person 804 to continue working on the artifact without having to pause to manually adjust the positioning of tablet computer 802 and / or the settings of the camera system in tablet computer 802.

[0059] In some instances, the person performing the gesture may also be an element of the physical environment indicated by the gesture. Fig. 9 , person 904 may perform a beckoning gesture 906, in response to which the camera system may pan the camera frame to element 908, which is person 904 itself. In one example, the camera frame may have previously been centered on person 910 and may not include person 904, who may have been within the camera system's field of view and therefore visible to the camera system despite not being within the camera frame.

[0060] In other examples, an element of the physical environment may be a person who is not performing a gesture. For example, person 904 may perform a shush gesture in the direction of person 910, triggering the camera system to pan the camera frame to include and / or center on person 910.

[0061] In one embodiment, the camera system may include and / or be part of a video conferencing system, and monitoring a person within a physical environment may include: monitoring a participant in a video conference that includes at least one additional participant that does not share a physical environment with the participant; and performing an action may include increasing visibility of an element of the physical environment within a video feed displayed by the camera system to the additional participants in the video conference. For example, person 904 and person 910 may be part of a video conference that includes participants in a remote office. In one instance, person 904 may not initially be within a camera frame of a video feed being sent to the remote participants. In this instance, person 904 may perform a beckoning gesture 906, in response to which the camera system may shift the video frame to include person 904 so that the remote participants in the video conference can see person 904 while speaking during the video conference. In another instance, the camera frame may initially include person 904 and person 910, and person 904 may perform a beckoning gesture 906 to instruct the camera to zoom in on person 904 so that he has greater visibility to the remote participants.

[0062] In some instances, a person may use a palm-up gesture to indicate elements of the physical environment to others. Fig.10 , person 1004 may perform a palm-up display gesture 1006 to indicate element 1008. In one example, the camera system may recognize the palm-up display gesture 1006 as indicating element 1008, and in response, may temporarily increase the quality of the video stream and / or adjust the gain control, color balance, and / or other elements of the video settings to ensure that element 1008 is clear to the viewer of the video stream. In some embodiments, different gestures may prompt the camera system to perform different actions. For example, a pointing gesture may prompt the camera system to zoom in on the indicated element, a circle gesture may prompt the camera system to save a picture of the indicated element, and / or a palm-up display gesture may prompt the camera system to visually highlight the indicated element in the video feed. Additionally or alternatively, the same gesture may prompt the camera system to perform different actions in different contexts. For example, a palm-up display gesture during a video conference may prompt the camera system to visually highlight the element, while the same palm-up display gesture performed outside of a video conference may prompt the camera system to retrieve information about the indicated element.

[0063] In some instances, a person may use more than one hand to perform a gesture. Fig.11, a person may perform a framing gesture 1106 by performing gesture 1106(a) with their left hand and gesture 1106(b) with their right hand. In some implementations, the framing gesture 1106 may prompt a camera system to match the camera frame to the frame 1102 formed by the gesture 1106 and / or focus on an element 1108 within the frame 1102. In one example, the framing gesture 1106 may prompt the camera system to capture everything within the frame 1102. Other examples of gestures performed with both hands may include, but are not limited to, waving with both hands, shushing with both hands, outlining a shape with both hands, and / or pointing with both hands.

[0064] In some implementations, the systems described herein may be represented as modules stored in a memory on a computing device, such as, Fig.12 1202 in. In one embodiment, the computing device 1202 may host a camera system 1212. In some instances, the monitoring module 1204 may monitor a person 1214. At a certain point in time, the detection module 1206 may detect a gesture 1216 performed by the person 1214. Next, the determination module 1208 may determine that the gesture 1216 indicates an element 1218. In response to determining that the gesture 1216 indicates the element 1218, the action module 1210 may perform an action 1220 on the element 1218. In some embodiments, the computing device 1202 may represent a personal computing device, such as a desktop computer, a laptop computer, and / or a tablet computer. In one embodiment, the computing device 1202 may represent a mobile device such as a mobile phone. In some embodiments, the computing device 1202 may represent a wearable device, such as a helmet, a smart necklace, a smart watch, and / or other types of wearable devices. Additionally or alternatively, the computing device 1202 may represent a smart home appliance, such as a television system with an integrated camera and / or a security camera. In one embodiment, computing device 1202 may represent a video conferencing system. Although shown as one element, in some embodiments, computing device 1202 may represent multiple computing devices. For example, a smart necklace with limited processing power may include monitoring module 1204, but may send a video feed via detection module 1206 and / or determination module 1208 to a server that analyzes the video feed and then sends instructions to action module 1210 on the smart necklace.

[0065] As explained above, the systems and methods described herein may enable a person to direct an adaptive camera system using intuitive human-readable gestures, such as pointing, waving, or snapping fingers. By enabling a person to direct a camera system via gestures, the systems described herein may enable a user to change settings of a camera, save information, retrieve information, display additional information, and / or perform other tasks related to the camera system without having to pause activities in which other users may be engaged, such as teaching a class, leading a meeting, or working on a craft. By using human-readable gestures rather than manual gestures, the systems described herein may enable a user to intuitively control an adaptive camera system without having to learn new gestures or perform awkward gestures that interrupt the flow of the user's other activities.

[0066] As described in detail above, the computing devices and systems described and / or illustrated herein broadly represent any type or form of computing device or system capable of executing computer-readable instructions (such as, computer-readable instructions contained within the modules described herein). In its most basic configuration, these computing devices can each include at least one memory device and at least one physical processor.

[0067] In some instances, the term "memory device" generally refers to any type or form of volatile or non-volatile storage device or medium capable of storing data and / or computer-readable instructions. In one example, a memory device can store, load, and / or maintain one or more of the modules described herein. Examples of memory devices include, but are not limited to, random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid-state drive (SSD), optical disk drive, cache, one or more variants or combinations thereof, or any other suitable memory.

[0068] In some instances, the term "physical processor" generally refers to any type or form of hardware-implemented processing unit capable of interpreting and / or executing computer-readable instructions. In one instance, the physical processor can access and / or modify one or more modules stored in the above-mentioned memory device. Examples of physical processors include, but are not limited to, microprocessors, microcontrollers, central processing units (CPUs), field programmable gate arrays (FPGAs) implementing soft-core processors, application specific integrated circuits (ASICs), one or more portions thereof, one or more variants or combinations thereof, or any other suitable physical processor.

[0069] Although shown as separate elements, the modules described and / or shown herein may represent parts of a single module or application. In addition, in some embodiments, one or more of these modules may represent one or more software applications or programs that can cause a computing device to perform one or more tasks when executed by a computing device. For example, one or more of the modules described and / or shown herein may represent modules stored and configured to run on one or more computing devices or systems in the computing device or system described and / or shown herein. One or more of these modules may also represent all or part of one or more special-purpose computers configured to perform one or more tasks.

[0070] In addition, one or more of the modules described herein may transform data, physical devices, and / or representations of physical devices from one form to another. For example, one or more of the modules described herein may receive visual data to be converted into computing instructions, convert the visual data into computing instructions, output the conversion results to instruct the camera system to perform an action, use the conversion results to perform the action, and store the conversion results to maintain an operation log. Additionally or alternatively, one or more of the modules described herein may transform a processor, volatile memory / non-volatile memory, and / or any other part of a physical computing device from one form to another by executing on a computing device, storing data on a computing device, and / or otherwise interacting with a computing device.

[0071] In some embodiments, the term "computer-readable medium" generally refers to any form of device, carrier or medium capable of storing or carrying computer-readable instructions. Examples of computer-readable media include, but are not limited to, transmission-type media such as carrier waves, and non-transitory-type media such as magnetic storage media (e.g., hard drives, tape drives, and floppy disks), optical storage media (e.g., compact disks (CDs), digital video disks (DVDs), and BLU-RAY disks), electronic storage media (e.g., solid-state drives and flash memory media), and other distribution systems.

[0072] The process parameters and order of the steps described and / or illustrated herein are given as examples only and may be changed as needed. For example, although the steps shown and / or described herein may be shown or discussed in a particular order, these steps do not necessarily need to be performed in the order shown or discussed. The various exemplary methods described and / or illustrated herein may also omit one or more steps described or illustrated herein, or include additional steps in addition to those disclosed.

[0073] The foregoing description is provided to enable other persons skilled in the art to best utilize the various aspects of the exemplary embodiments disclosed herein. This exemplary description is not intended to be exhaustive or limited to any exact form disclosed. Many modifications and variations may be made without departing from the spirit and scope of the present disclosure. The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. In determining the scope of the present disclosure, reference should be made to the attached claims and their equivalents.

[0074] Unless otherwise indicated, the terms "connected to" and "coupled to" (and their derivatives) used in the specification and claims should be interpreted as allowing both direct and indirect (i.e., via other elements or components) connections. In addition, the terms "a" or "an" used in the specification and claims should be understood to mean "at least one". Finally, for ease of use, the terms "including" and "having" (and their derivatives) used in the specification and claims are interchangeable with the word "comprising" and have the same meaning.

Claims

1. A computer-implemented method for guiding an adaptive camera system, comprising: Monitoring people within a physical environment via a camera as part of a camera system; detecting, by the camera system, a gesture performed by the person being monitored by the camera, wherein the gesture is understandable to a human and used by the person to communicate with others; identifying the element of the physical environment based on a determination by the camera system and in response to detecting the gesture that the gesture indicates that the element of the physical environment is an element of interest to the camera system; determining an action to be performed by the camera system based on the gesture; and The action directed at the element of the physical environment is performed by the camera system and in response to identifying the element of the physical environment and determining the action to be performed, treating the element of the physical environment as the element of interest.

2. The computer-implemented method of claim 1 , wherein: The gesture comprises a human-readable gesture.

3. The computer-implemented method of claim 1 , wherein: The gesture indicates the element of the physical environment to another person; and In response to determining that the gesture indicates the element of the physical environment to the other person, the camera system performs the action.

4. The computer-implemented method of claim 1 , wherein: Performing the action by the camera system includes determining that the gesture is associated with a predetermined action from a list of actions associated with gestures.

5. The computer-implemented method of claim 1 , wherein: Performing the action includes physically moving, by the camera system, the camera.

6. The computer-implemented method of claim 1 , wherein: Performing the action includes adjusting a camera frame of the camera.

7. The computer-implemented method of claim 6, wherein: Performing the action includes moving the camera frame of the camera to include the element of the physical environment indicated by the gesture within the camera frame.

8. The computer-implemented method of claim 6, wherein: Performing the action includes adjusting the camera frame of the camera to increase prominence of the element of the physical environment indicated by the gesture within the camera frame.

9. The computer-implemented method of claim 1 , wherein: Performing the action includes changing the focus of the camera to the element of the physical environment indicated by the gesture.

10. The computer-implemented method of claim 1, wherein: Performing the action includes retrieving, by the camera system, information about the element of the physical environment indicated by the gesture.

11. The computer-implemented method of claim 1 , wherein: Performing the action includes displaying information about the element of the physical environment indicated by the gesture.

12. The computer-implemented method of claim 1, wherein: Performing the action includes storing, by the camera system, information about the element of the physical environment indicated by the gesture.

13. The computer-implemented method of claim 1 , wherein: The gesture comprises a pointing gesture at the element of the physical environment; and The elements of the physical environment include objects.

14. The computer-implemented method of claim 1 , wherein: The gesture indicates a portion of the physical environment; and Determining, by the camera system, that the gesture indicates the element of the physical environment includes examining, by the camera system, the portion of the physical environment indicated by the gesture.

15. The computer-implemented method of claim 1 , wherein: The gesture comprises a hand gesture; and Monitoring the person within the physical environment via the camera as part of a camera system includes motion tracking a hand of the person.

16. The computer-implemented method of claim 1, wherein: The camera system includes an augmented reality system; and Performing the actions includes augmenting, by the augmented reality system, the physical environment with an augmented reality element corresponding to the element of the physical environment.

17. The computer-implemented method of claim 1, wherein: The camera system comprises a video conferencing system; and Monitoring the person within the physical environment includes: monitoring a participant in a video conference that includes at least one additional participant that does not share the physical environment with the participant; and Performing the action includes increasing visibility of the element of the physical environment within a video feed displayed by the camera system to the additional participants in the video conference.

18. A system for guiding an adaptive camera system, comprising: a surveillance module stored in the memory, the surveillance module monitoring a person within the physical environment via a camera as part of the camera system; a detection module stored in a memory, the detection module detecting, via the camera system, gestures performed by the person being monitored by the camera, wherein the gestures are understandable to humans and used by the person to communicate with others; A determination module stored in a memory, the determination module: identifying the element of the physical environment based on a determination by the camera system and in response to detecting the gesture that the gesture indicates that the element of the physical environment is an element of interest to the camera system; and determining an action to be performed by the camera system based on the pose; an action module stored in a memory, the action module, via the camera system and in response to identifying the element of the physical environment and determining the action to be performed, treating the element of the physical environment as the element of interest to perform the action directed at the element of the physical environment; as well as At least one physical processor, the physical processor executes the monitoring module, the detection module, the determination module, and the action module.

19. The system of claim 18, wherein: The gesture comprises a human-readable gesture.

20. A non-transitory computer-readable medium comprising: One or more computer readable instructions that, when executed by at least one processor of a computing device, cause the computing device to: Monitoring people within a physical environment via a camera as part of a camera system; detecting, by the camera system, a gesture performed by the person being monitored by the camera, wherein the gesture is understandable to a human and used by the person to communicate with others; identifying an element of the physical environment based on a determination by the camera system and in response to detecting the gesture that the gesture indicates that the element of the physical environment is an element of interest to the camera system; determining an action to be performed by the camera system based on the gesture; and The action directed at the element of the physical environment is performed by the camera system and in response to identifying the element of the physical environment and determining the action to be performed, treating the element of the physical environment as the element of interest.

Images