Real-time unintended camera start detection and camera deactivation

Abstract

The invention relates to real-time unintended camera start detection and camera deactivation. In some embodiments, a method for detecting whether an application is initiated due to an unintentional interaction on a user device is disclosed. The method includes analyzing signals from one or more sensors of the user device to identify any obstacles in the vicinity. By checking the media stream, the device may derive a measure indicating whether the camera is obstructed. The metrics in combination with the sensor data may indicate that the launch is due to unintentional interaction, resulting in initiation of the first mitigation operation. The first mitigation operation involves reducing a power state of the device and dimming a display of the device. After initiating the first mitigation operation, the user interface is monitored to identify an interaction reflecting an intent to launch the application. If a predetermined period has elapsed without such interaction, a confirmation is made that the unintentional application launch prompts execution of a second mitigation operation.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Application No. 63 / 645,999, filed May 13, 2024, entitled “REAL-TIME UNINTENDED PHOTOCAPTURE DETECTION AND CAMERA DEACTIVATION,” the entire contents of which are hereby expressly incorporated herein by reference. Technical Field

[0003] This technology involves reducing unintentional camera activation and closing the camera application when the camera is unintentionally activated to stop frame capture on the device. Background Technology

[0004] User devices have become increasingly integrated into daily life, serving as essential tools for communication, productivity, and entertainment. Cameras are often integrated into these user devices, meaning users almost always carry a camera with them. These cameras and camera apps are convenient and easy to use. Attached Figure Description

[0005] Details of one or more embodiments of the subject matter described in this disclosure are set forth in the accompanying drawings and description below. However, the drawings illustrate only some typical embodiments of this disclosure and should not be considered as limiting the scope of this disclosure. Other features, embodiments, and advantages will become apparent from the description, drawings, and claims.

[0006] Figure 1 Example devices are illustrated in some implementations of this technology where unintentional access by an application has an impact.

[0007] Figure 2 Example architectures according to some implementations of the present technology are illustrated, which are used to detect unintentional camera launches and use the results of this detection to determine whether the application should be allowed to launch, or whether the application should remain launched if it has already been launched.

[0008] Figure 3 Example flowcharts illustrating how to determine whether an application is launched due to unintentional user-device interaction according to some implementations of this technology are provided after application launch.

[0009] Figure 4 The process for detecting when an application is launched due to unintentional user-device interaction is illustrated according to some embodiments of the present technology.

[0010] Figure 5The process for starting a media capture device according to some embodiments of the present technology is illustrated.

[0011] Figure 6 This is a system diagram illustrating some embodiments of the device according to the present technology.

[0012] Figure 7 An example computing system for implementing a specific embodiment of this technology is shown. Detailed Implementation

[0013] Various examples of this disclosure will be discussed in detail below. While specific embodiments are discussed, it should be understood that this is done for illustrative purposes. Those skilled in the art will recognize that other components and configurations can be used without departing from the spirit and scope of this disclosure. Therefore, the following description and drawings are illustrative and should not be construed as limiting. Numerous specific details are described to provide a thorough understanding of this disclosure. However, in some cases, well-known or conventional details have not been described to avoid obscuring the description. An example of this disclosure or a reference to an example may be a reference to the same example or any example; and such a reference may refer to at least one of these examples.

[0014] The reference to "an embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with that embodiment can be included in at least one embodiment of the invention. The phrase "in an embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. Furthermore, various features that may be exhibited in some embodiments but not in others are described.

[0015] The terms used in this specification generally have their common meaning in the art, in the context of this disclosure, and in the specific context of each term's use. Alternative terms and synonyms may be used for any one or more of the terms discussed herein, and no special meaning should be assigned to any term regardless of its detailed description or discussion herein. In some cases, synonyms for certain terms are provided. The description of one or more synonyms does not preclude the use of other synonyms. The use of examples (including examples containing any of the terms discussed herein) anywhere in this specification is illustrative and not intended to further limit the scope and meaning of this disclosure or any of the example terms. Similarly, this disclosure is not limited to the various embodiments given in this specification.

[0016] Without intending to limit the scope of this disclosure, examples of instruments, apparatus, methods, and related results according to embodiments of this disclosure are given below. It should be noted that headings or subheadings may be used in the examples for the reader's convenience, and these headings or subheadings should not limit the scope of this disclosure. Unless otherwise defined, all technical terms used herein have the meanings commonly understood by one of ordinary skill in the art to which this disclosure pertains. In case of conflict, this document (including definitions) shall prevail.

[0017] Additional features and advantages of this disclosure will be set forth in the following description, and will be apparent in part from that description, or may be learned by practicing the principles disclosed herein. The features and advantages of this disclosure may be realized and obtained by means of the instruments and combinations particularly pointed out in the appended claims. These and other features of this disclosure will become more apparent from the following description and the appended claims, or may be learned by practicing the principles set forth herein.

[0018] User devices (including mobile devices, tablets, and similar gadgets) often incorporate cameras that can be easily activated through interaction with the user interface or the device itself (via software buttons on the interface or physical buttons on the device). However, in certain scenarios, such as when stored in a confined space like a pocket or bag, these buttons may be inadvertently interacted with, leading to unintended application launches. Regarding camera applications, the widespread availability of cameras on user devices also introduces privacy concerns stemming from unintentional camera activation caused by unintended user interface input. Such unintentional camera activation poses the risk of unintentional collection of video, audio, and images, potentially compromising user privacy or capturing events that the user does not wish to be digitally recorded or captured. Furthermore, unintentional access to applications that may consume significant battery power can deplete the device's limited battery capacity.

[0019] The disclosed technology addresses the need in the art to reduce unintentional camera activation, ensuring that when the camera is unintentionally activated, the camera application is quickly shut down to stop frame capture by the device. The disclosed technology includes methods for application launch suppression to minimize unintended application launches that could harm user privacy and device battery life. The disclosed technology can receive an indication that an application has been launched and one or more media capture devices on the user device are actively capturing frames of its surroundings. Measurements obtained from one or more media capture devices, along with other sensors such as proximity sensors, gyroscopes, and accelerometers, can determine whether the application launch is unintentional. Once this determination is made, a first mitigation action is triggered, including but not limited to reducing the device's power state and dimming the display while allowing the application to continue running. After these mitigation actions are initiated, the user interface is intensively monitored to identify interactions reflecting an intent to launch the application or capture frames. If no such interaction occurs after a predetermined period, confirmation of unintentional application launch is made, prompting the execution of a second mitigation operation. This operation includes reversing the application launch and reducing the user device's power consumption to a predetermined level, thereby ensuring optimal device performance and protecting user privacy.

[0020] Figure 1 Example devices are illustrated in some implementations of this technology where unintentional access by an application has an impact.

[0021] In some examples, an application may be opened unintentionally through interaction with user device 102, whether this interaction is made via accidental activation of physical button 108, software button 110, unintentional tapping on display 106 of user device 102, or other unintentional interaction. These interactions may lead to application launch, causing user device 102 to assume that the launch was initiated by intentional user interaction with the intent to participate in the launched application. Such unintentional activation may occur due to the sensitivity of the touchscreen interface on display 106, the outward protrusion of physical button 108 on user device 102, or external factors such as pressure or movement of user device 102 during transport or carrying.

[0022] When unintentional interaction is detected, user equipment 102 may initially take action to prevent application launch. This action may be triggered by initial detection from one or more sensors of user equipment 102, which makes an initial determination that the user is not currently interacting with user equipment 102. Alternatively, user equipment 102 may implement preventative measures to perform verification of user interaction before proceeding with application launch. These measures may involve requiring additional confirmation, such as auxiliary input or authentication, to ensure that the user's intent to launch the application is acknowledged before proceeding. By implementing these proactive measures, user equipment 102 aims to mitigate the risk of unintended application launch.

[0023] However, even with such proactive measures to prevent unintentional app launches, apps sometimes launch anyway, resulting in unnecessary use of device resources such as memory, battery, and cellular data without user intent. For example, in a scenario where a camera app is unintentionally activated while the user device 102 is in a pocket, the user device 102 could perform a detection process to determine whether the activation was initiated by an intentional user interaction.

[0024] During the detection process, the operating system extracts data from sensors (such as proximity sensors, accelerometers, and gyroscopes) to determine the state of user equipment 102. Accelerometers provide inertial measurements, while gyroscopes capture rotational data, thereby identifying different motion patterns and device orientations, particularly indicating the condition of being bagged. Additionally, some user equipment utilizes proximity sensors to detect the proximity of the device (especially a display) to a surface or another object obstructing the view of one or more cameras on user equipment 102.

[0025] In scenarios where the camera's field of view is physically obstructed by an object (such as a hand, finger, or other obstacle), the device's camera system can identify obstructions in real time by analyzing the camera's video feed to detect changes in image features. Upon identifying an obstruction, the system can trigger a response, such as sending a warning or notification to the user, ensuring that the device's camera can capture clear, unobstructed images and maintain feature-based performance.

[0026] When an approaching object is detected, one or more of the sensors in user equipment 102 can deactivate the display and touchscreen to prevent unintentional input. Furthermore, the proximity sensor can work in conjunction with an ambient light sensor to adjust display brightness according to environmental conditions. When the device is brought close to the user's face during a call, the proximity sensor can be used to automatically disable display and touch functionality.

[0027] Figure 1An example of an operating environment for user device 102 is illustrated, in which unintentional application launch may occur. An obstruction 104 caused by unintentional operation of user device 102 (such as picking up or putting down user device 102) can be detected by a proximity sensor. In scenarios where the proximity sensor is triggered by a surface or object during these movements, the sensor may interpret the surface or object as an obstruction, even if user device 102 is not in a pocket or bag. Despite the presence of this temporary obstruction 104, an application may still launch unintentionally. This unintentional detection may occur when user device 102 moves rapidly and the proximity sensor fails to distinguish between intentional operation and unintentional obstruction, resulting in unintended application activation. When an application such as a photo or video capture application is launched, the media stream received from the capture device (e.g., a camera) is analyzed to determine whether the camera is pointing at an obstructed scene, such as inside a pocket. If no identifiable object is detected and obstruction 104 is visible, a determination is made that the application was launched due to unintentional interaction with physical button 108, software button 110, display 106, or other components of user device 102. For example, if obstruction 104 is identified as a pocket, user equipment 102 identifies that the user equipment is placed in the pocket and infers that there is no direct user holding it. In response, one or more mitigation operations may be performed, causing the user equipment to reduce its power state and shutting down the application if it is determined that the application was launched due to unintentional user-device interaction.

[0028] Figure 2 Example architecture 200 is illustrated according to some implementations of the present technology. This example architecture is used to detect unintentional camera launch and to use the result of this detection to determine whether the application should be allowed to launch, or whether the application should remain launched if it has already been launched.

[0029] The Always-On Processor (AOP) 202 can execute the Device Physical Context Service 204 to detect when the user's device is in their pocket before, during, and after application launch. Even when the main processor (CPU) is in a low-power state or asleep, AOP 202 manages specific low-power tasks. Using this Device Physical Context Service 204, AOP 202 can determine whether it should wake up the main processor or manage power delivery to various components when the application is activated or when user interaction is detected. One process handled by AOP 202 includes repeatedly storing a device context framework of motion and proximity states. These states can be stored to trigger actions performed by the device (such as waking the display) or referenced by the application while it is running.

[0030] For example, the Device Physical Context Service 204 on AOP 202 can receive motion or proximity status from proximity sensor 226 and inertial measurement unit (IMU) 228. IMU 228 is a hardware component of the user device consisting of sensors including accelerometers, gyroscopes, and magnetometers. Each of these sensors works together to measure the device's motion, orientation, and rotation in different states, whether physically held by the user or in a stored location. Proximity sensor 226 can be placed on the front and / or back of the user device to detect proximity of the device (typically a display or camera) to a surface such as the user's face, pocket lining, or table surface. In addition to IMU 228, data from proximity sensor 226 can be used by Device Physical Context Service 204 to detect whether the device is in a pocket.

[0031] Before initiating application launch, the operating system's application launch service 206 can utilize motion and / or proximity status provided by the device physical context service 204 to determine whether to proceed with application launch or activate application behavior designed to verify user intent. Signals received from the IMU 228 and proximity sensor 226 act as indicators of the presence of an obstruction near the user device. For example, if the device is detected in a static downward posture, it is determined that the application was launched due to unintentional interaction with the user device's physical button 108, software button 110, display 106, or other components. The application launch service 206 identifies this scenario as accidental and prevents application launch, thereby ensuring that unintentional device interaction does not lead to unwanted application activity. If it is determined that the initiation of launch is expected, or at least no determination is made that the application was launched due to unintentional user-device interaction, the application launch service 206 can launch the application and initiate camera capture service 224. Camera capture service 224 may be part of a camera application.

[0032] In an example where the application has already been launched, the unintentional launch algorithm 230 can determine that the application was launched due to unintentional user-device interaction. This scenario can occur when a user unintentionally activates the camera while placing device 102 in their pocket, causing the camera to launch before it becomes obstructed. Similarly, accidental activation can occur when the device is placed on a table, unintentionally triggering the camera. Therefore, when the device is in the user's pocket, the camera application launches and begins streaming or previewing frames, resulting in unintended media capture. Upon receiving signals from proximity sensor 226 and IMU 228 providing a measure of potential obstruction, device physical context service 204 can determine that the field of view is obstructed and send a signal to camera capture service 224. Camera capture service 224 can activate one or more cameras on the user's device to generate a media stream, allowing verification that the camera is capturing a valid scene or focusing on an object of interest. In some examples, camera capture service can use the object or scene of interest to identify user interactions, including analyzing gestures within the field of view of the media capture device to distinguish between intentional and unintentional interactions. Camera capture service 224 may receive signals from ISP 220 via image signal processor (ISP) driver 222. ISP 220 processes image data captured by one or more cameras of the user device and provides this image data to camera capture service 224. Unintentional capture algorithm 230 may use the image data from ISP 220 to determine that the camera was unintentionally triggered due to accidental activation of the application when the device is placed on a table or in a pocket.

[0033] In some examples, the unintentional capture algorithm 230 can determine that an application is continuously launched due to unintentional activation based on one or more unintentional launch strategies. These strategies can be formulated using sensor-based input and contextual decision-making, including static detection logic, orientation-based logic, and rules for handling multiple or repeated button presses.

[0034] The static detection-based logic is configured to suppress button clicks when the device is identified as stationary. This state is determined using gyroscope and accelerometer sensors that detect no significant movement. To prevent accidental button presses that might occur when the user picks up the device, the suppression continues for a short period after the stationary state ends. If a button press is prevented according to this logic, a message on the screen near the button is displayed to the user, indicating that a second click is required to launch the camera app.

[0035] Orientation-based logic adds subtlety to click suppression by incorporating the device's detected orientation. Using accelerometer and gyroscope data, this logic can identify whether the device is in a longitudinal or lateral orientation. In cases where static detection logic might erroneously suppress intentional clicks, such as when the device is mounted on a tripod, the orientation signal can take over static suppression to allow the camera buttons to function as intended.

[0036] The accidental activation strategy also addresses accidental activation caused by multiple simultaneous button presses. For example, a user might unintentionally press the camera button with their palm while interacting with an adjacent control such as the volume button. In such cases, the logic detects concurrent activation and rejects the camera button press to prevent accidental activation.

[0037] To adapt to user behavior and reduce unintended suppression, the system employs repeated button press detection. If a second press follows a suppressed button click within a short time frame (such as five seconds), this strategy causes the unintentional launch algorithm to identify it as intentional and proceed with launching the camera application.

[0038] In one example, application launch can be initiated by an interaction detected by button 218. Button 218 can be a physical button or a software button. A power management unit (PMU) 216 can receive signals from button 218. PMU 216 is responsible for managing power consumption and distribution within the user device and regulating the voltage and current supplied to different components of the device. PMU 216 can send signals to button driver 214. Events created by the interaction with button 218 can be managed by event handler 212 in application launch service 206. Event handler 212 is configured to provide event handling and input management for user interactions with the device interface (such as touch events, gestures, and other inputs from the user) and relay these user interactions to appropriate application or system components, such as application launch service 206, UI Kit 210, or system overlay 208.

[0039] Figure 3 Example flowcharts illustrating the determination after application launch whether an application is launched due to unintentional user-device interaction, according to some embodiments of the present technology. Although this example system depicts specific system components and their arrangement, this depiction is for the purpose of facilitating the discussion of the present technology and should not be considered limiting unless specified in the appended claims. For example, some components illustrated as separate may be combined with other components, some components may be divided into separate components, some components may be absent or unnecessary, and other components may be present.

[0040] In some implementations, button 218 on the user device is pressed, which triggers the launch 302 of the camera application. Once the camera application has been launched, any active camera on the user device can begin streaming. For example, the front or rear camera (or both) will begin streaming for a certain period 304 while waiting for AE / AF / AWB conversion. This period can be several seconds, such as 1 second, 3 seconds, 6 seconds, 9 seconds, etc. During this period, the camera uses values ​​for auto exposure, auto focus, and auto white balance to stabilize, and the camera application can determine which camera is intended for the user.

[0041] Once the cameras are ready for use after a stabilization period and upon receiving the stream, 308 frame statistics (306) are accumulated before media capture from any of these cameras. Frames extracted from the media stream may exhibit characteristics indicating that the captured content is unimportant (e.g., it's in a pocket, on a table, or not capturing the desired scene). Specifically, the unintentional capture algorithm (230) identifies low frequencies indicating blurry or out-of-focus images (inconsistent with the intended framing). This analysis involves evaluating the frequency components of the frames and establishing thresholds for the frequency band distribution.

[0042] In some implementations, a neural network can be used to analyze frames and determine whether a captured frame is intentional or unintentional (e.g., captured when the device is in a pocket, on a table, or not focused on the desired scene). The neural network can be pre-trained on a large dataset of labeled examples, including intentionally well-framed images and unintentionally captured frames, such as those captured when the device is in a pocket. This network can learn to extract meaningful features from the frames that indicate user intent. Once trained, the neural network can be deployed on the device to analyze each frame and classify it as intentional or unintentional. The neural network extracts frame features and can accumulate frames over a period of time to build stream statistics. The frame features and statistics from this neural network can then be used by another machine learning-based classifier or another neural network to evaluate the importance of the captured content.

[0043] Signals captured from frames include metadata, which typically contains statistics from the image signal processor (ISP). These frame statistics can be analyzed before the actual user capture begins (i.e., after the camera app has been launched, but before the user has pressed the capture UI button) to assess whether the user's app interaction at the start of capture was intentional or unintentional. While some of this metadata is computed at a higher level, a significant portion originates from the lower-level ISP. During frame processing, the focus mechanism provides statistics on focus success, focus methods, and other relevant details such as focus distance and any camera switching due to focus issues. These metadata signals, encompassing stability indicators such as autofocus stability and exposure stability, are aggregated and adopted throughout the camera stack.

[0044] These metadata signals ensure that the camera is in a convergent state before collecting statistics. For example, the stability signal indicates when the camera achieves stability, thus facilitating accurate data collection. Furthermore, metadata signals such as autofocus pose, autofocus position, and the camera in use are utilized during frame processing. If the camera in use encounters focusing difficulties, an alternative camera can be switched to for further focusing attempts. By ensuring the camera is in a convergent state, reliable data can be obtained from the camera when it is determined whether the user equipment is experiencing obstruction. The camera needs to be in a convergent state to obtain reliable data.

[0045] Furthermore, a first-level indication of light intensity derived from parameters such as exposure, noise level, and sensor gain can be utilized. The estimate from this first-level indication allows the ambient light conditions to be correlated with the user device present in the user's pocket. Therefore, each camera can estimate its quality level based on these parameters, providing valuable insight into the dominant light intensity during image capture.

[0046] In some examples, machine learning-based classifiers (such as neural networks) can classify each frame as intentional or unintentional using aggregated metadata signals and combined with frame features. The classifier can also simultaneously utilize metadata and frame features from multiple camera streams (e.g., combining inputs from wide and ultra-wide camera streams) to assess the relevance of captured content and make a more informative determination about the importance of the media. In some examples, the front-facing camera of a user device can be utilized. The front-facing camera can achieve face detection and object detection within its field of view using the projection of lasers via LiDAR. By utilizing LiDAR, the front-facing camera can effectively act as a depth camera and determine distance measurements to a hypothetical object within its field of view. Using the front-facing camera to measure distance makes it possible to determine whether the device is obstructed or placed in a front pocket. Similar to the functionality of proximity sensors, utilizing the depth sensing capabilities of the front-facing camera provides an alternative method for determining device placement.

[0047] In some examples, the front-facing camera may be obstructed, while the rear camera or another camera on the user device may be unobstructed. In this case, further determinations can be made regarding both cameras to determine whether the user device is obstructed and subject to interaction. These determinations can be made based on the individual states of each camera. Furthermore, the device can utilize the LiDAR sensor on the rear camera to map the environment and assess the presence of obstructions or valid scenes, even in low-light or complex environments. The front-facing camera equipped with structured light technology can detect and map 3D objects, enhancing the user device's ability to distinguish between intentional and unintentional captures.

[0048] 308 frames of statistics (including signals, streams, and statistics from the device's cameras and sensors) are analyzed to determine if the device is in a static, face-down pose. If it is determined that the application was launched due to unintentional interaction with the user device's physical button 108, software button 110, display 106, or other components, the application launch service 206 identifies this scenario as accidental. In response, the service prevents the application from launching, thus ensuring that unintentional device interaction does not lead to unnecessary battery drain or unwanted application activity. If the device is not in a static pose and no obstruction is detected, the camera session can proceed as normal, capturing potentially interesting objects or scenes 310.

[0049] Upon determining that an obstacle is being detected, a first mitigation is initiated, during which the user device display is dimmed 312. When the screen dimming feature is activated, the user's device can reduce power consumption. Additionally, the system performs a 5-second verification process to confirm that the button has indeed been interacted with by the user. During this time, the user can still initiate interaction with the button or display 314 before taking any further mitigation action. If the user interacts with the screen to wake it up during the dimming period for a predetermined time period, pocket detection / mitigation can be exited 316.

[0050] However, in the absence of user-initiated interaction, the initial button interaction is considered an interaction. While monitoring user-initiated interactions, further analysis of the media stream can be performed to determine if the camera's viewpoint has continued to be obstructed.

[0051] If no actual button press or user interaction is detected, and the detection algorithm determines that the device is in a static, face-down position on a table or in a pocket, the launch of the camera app is canceled, and the launch of the camera app or its extensions can be canceled as well. Alternatively, if it is determined that further detection is needed, the process is repeated to continuously monitor, detect, and analyze the live stream.

[0052] Figure 4 Example process 400 for detecting application activation due to unintentional user-device interaction, according to some embodiments of the present technology, is illustrated. Although example process 400 depicts a specific sequence of operations, this sequence can be changed without departing from the scope of this disclosure. For example, some of the depicted operations may be performed in parallel or in a different order that does not substantially affect the functionality of process 400. In other examples, different components of the example device or system implementing process 400 may perform functions substantially simultaneously or in a specific order.

[0053] According to some examples, the method includes receiving, at box 402, an indication that an application has been launched and one or more media capture devices on the user device are capturing frames around a region of the user device. For example, Figure 2 The unintentional capture algorithm 230 illustrated herein can receive an indication that an application has been launched and that one or more media capture devices are simultaneously capturing frames of the environment surrounding the user device. This application launch can be indicated by interaction with a button or an application icon displayed on the user device's screen. The one or more media capture devices include one or more cameras whose location spans various parts of the user device.

[0054] In some examples, a verification can be performed to determine if the view of any of the media capture devices can be checked for obstruction. This verification can trigger an unintended capture algorithm 230 to determine if additional action is needed to prevent unintended capture. In some cases, a proximity sensor can provide data to detect the presence of an approaching object, indicating that an application launch interaction has been initiated. Following this, an obstruction detection process is initiated to determine if the application was unintentionally launched due to an obstruction in the device's surrounding environment.

[0055] According to some examples, the method includes analyzing signals received from one or more sensors of the user equipment at box 404 to identify the presence of obstructions in the area surrounding the user equipment. For example, Figure 2 The unintentional capture algorithm 230 illustrated herein can analyze signals received from one or more sensors of the user equipment to identify the presence of obstructions in the area surrounding the user equipment.

[0056] According to some examples, the method includes analyzing one or more streams from one or more media capture devices on the user's device at box 406. For example, Figure 2 The unintentional capture algorithm 230 illustrated herein can analyze one or more streams from one or more media capture devices on a user device. These metrics are derived from the analysis of one or more streams. These streams produce metrics that include metadata linked to one or more media capture devices. The metadata indicates a threshold level associated with the expected scenario for capture. Furthermore, frames containing media captured by the media capture devices are identified, and frequencies are subsequently extracted from the media within the identified frames. These frequencies are examined in detail to determine whether one or more of these frequencies meet one or more thresholds. When at least one of these frequencies is identified as meeting a criterion associated with one or more thresholds, it can be determined that the content in the frame is unimportant (at least from the perspective of photo capture). This determination can indicate that the application was initiated due to unintentional user-device interaction.

[0057] According to some examples, the method includes, at box 408, determining that the application was launched due to unintentional user-device interaction when the metric indicates that one or more of these media capture devices are experiencing an obstruction. For example, Figure 2The unintentional capture algorithm 230 illustrated herein can determine that an application has been initiated due to unintentional user-device interaction when the metric indicates that one or more of these media capture devices are encountering an obstruction. The metric derived from the one or more streams further includes an estimated light intensity, calculated based on various parameters such as exposure, noise level, and sensor gain detected within the stream. These parameters serve to correlate the metric with a quality level indicating the overall performance of the stream. Furthermore, the metrics extracted from the stream involve evaluating whether parameters meet one or more thresholds. When it is identified that one or more of these parameters have met the parameter criteria, it can be determined that an obstruction may be blocking the field of view of the media capture device.

[0058] According to some examples, the method includes performing a first mitigation action at box 410 after determining that the application was enabled due to unintentional user-device interaction. For example, Figure 2 The unintentional capture algorithm 230 illustrated herein can perform a first mitigation operation after determining that an application has been launched due to unintentional user-device interaction. The first mitigation operation includes reducing the user device's power state to a first power state and dimming the user device's display. Reducing the user device's power state to a lower level as part of the first mitigation operation requires adjusting various components and functionalities to operate at a lower power consumption level. This adjustment effectively reduces battery usage and the total battery load applied to the launched application. By optimizing power usage, the user device minimizes the stress on its battery and conserves energy. Therefore, this action helps mitigate the potential impact of unintentionally launched applications on battery life, thereby ensuring efficient power utilization while maintaining necessary device functionality.

[0059] In addition to the first mitigation operation providing the user equipment with information about the power reduction caused by the lower power state, the central processing unit 606 can also, as well as... Figure 6 The display 610 is dimmed to strategically measure user responses when the dimmed display 610 is observed. By dimming the display, the central processing unit 606 aims to detect whether the user will further occupy the device or take action in response to the change in screen brightness. This verification process can help determine the user's interaction level and notify the central processing unit 606 of subsequent actions or adjustments to the device operation performed by the user.

[0060] According to some examples, the method includes: at box 412, after initiating the first mitigation action, monitoring the user interface to identify one or more user interactions indicating a user intent, which is for launching the application or using the application to capture one or more frames. For example, Figure 2The unintentional capture algorithm 230 illustrated herein can monitor the user interface after initiating a first mitigation operation to identify one or more user interactions indicating user intent to launch the application or use the application to capture one or more frames. Furthermore, analysis of one or more streams from the media capture device continues even after the first mitigation operation to detect any appearance of potentially relevant objects or scenes. Confirming that application launch stems from unintentional user-device interaction requires a specified time period of timeout without user interaction, coupled with the absence of the identified object or scene within the stream. Monitoring the stream to identify user interactions includes a detailed examination of gestures detected by the media capture device to distinguish between intentional and unintentional interactions. Simultaneously, the unintentional capture algorithm 230 awaits user interface interactions with physical or software buttons to determine the user's intent to launch the application or use it to capture frames.

[0061] According to some examples, the method includes: at box 414, when the period times out without receiving one or more user interactions, determining that the application may have been launched due to unintentional user-device interaction. For example, when the period times out without receiving one or more user interactions, Figure 2 The unintentional capture algorithm 230 illustrated in the example can determine that an application may be launched due to unintentional user-device interaction.

[0062] According to some examples, the method includes: at box 416, performing a second mitigation action after confirming that the application was launched due to unintentional user-device interaction. For example, Figure 2 The unintentional capture algorithm 230 illustrated here can perform a second mitigation operation after confirming that an application has been launched due to unintentional user-device interaction. The second mitigation operation includes closing the launched application and reducing the power consumption of the user device to a second power state. By closing the application, the user device can reduce the operation of one or more background processes, sensors, and activities related to the application and ISP 220.

[0063] By closing apps as part of a second mitigation measure, user devices mitigate potential battery drain from unintentionally launched apps that consume significant power. In particular, instances of unintentional launch pose a risk of capturing unwanted images and recordings, potentially compromising user privacy. Therefore, by quickly closing these apps, devices can prevent unauthorized image capture and recording, thus protecting user privacy and conserving device battery life.

[0064] Figure 5Example processes for starting a media capture device according to some embodiments of the present technology are illustrated. Although example process 500 depicts a specific sequence of operations, this sequence can be changed without departing from the scope of this disclosure. For example, some of the depicted operations may be performed in parallel or in a different order that does not substantially affect the functionality of process 500. In other examples, different components of the example device or system implementing process 500 may perform their functions substantially simultaneously or in a specific order.

[0065] According to some examples, the method includes receiving an indication that a button on the user's device has been pressed at box 502. For example, Figure 2 The PMU 216 illustrated can receive an indication that a physical button 218 on the user equipment has been pressed, or Figure 6 The central processing unit 606 can determine that the software button has been pressed. Button 218 can be configured to launch the camera application directly from a locked or unlocked screen state, thereby ensuring that the user can quickly capture wonderful moments without delay via one or more media capture devices. After the application launch service 206 launches the camera application, a second click on button 218 can initiate image capture.

[0066] According to some examples, the method includes determining at decision box 504 that a proximity sensor indicates the presence of an object near the proximity sensor. For example, Figure 2 The application launch service 206 illustrated here can determine the presence of an object near the proximity sensor 226 based on data from the proximity sensor 226. The proximity determination indicates the presence of an object that may not be of interest in the scene, and that the object is obstructing the view of one or more media capture devices.

[0067] According to some examples, at decision box 506, the method includes performing analysis to identify whether an object is actually close to the proximity sensor. At box 508, when an object is identified as being too close to the media capture device, application launch service 206 can terminate application launch. At box 510, when no object is found to be too close to proximity sensor 226, application launch service 206 continues to launch the application. For example, Figure 6 The central processing unit 606 illustrated here can enable the application startup service 206 to initiate the startup of the application.

[0068] Once button 218 is touched, application launch service 206 can immediately launch the application, because if the application is launched unintentionally, application launch service 206 can rely on the presence of unintentional capture algorithm 230 to terminate the application. Even with proximity detection, the application may launch unintentionally if the proximity detection determination made at decision box 506 is obstructed, which may happen when the phone is placed in a pocket such that no object is near the proximity sensor at the time the determination is made. Prior to this technology, button activation would require a sufficiently long duration to account for dynamic situations where the proximity sensor may not detect the surface immediately but after a short time interval. Unfortunately, this also delays the launch of camera applications, which is undesirable because photo opportunities may be fleeting.

[0069] Figure 6 This is a system diagram illustrating some embodiments of device 600 according to the present technology. Although this example system depicts specific system components and the arrangement of such components, this depiction is for the convenience of discussing the present technology and should not be considered limiting unless specified in the appended claims. For example, some components illustrated as separate may be combined with other components, some components may be divided into separate components, some components may be absent or unnecessary, and other components may be present.

[0070] Device 600 can perform various operations, including image processing. For this and other purposes, device 600 may include an image sensor 601, a system-on-a-chip 602, a system memory 617, a permanent storage device 616, a motion sensor 619, a display 610, and other components.

[0071] Image sensor 601 is a component for capturing image data and may be embodied, for example, as a complementary metal-oxide-semiconductor (CMOS) active pixel sensor, camera, camcorder, or other device. Image sensor 601 generates raw image data, which is sent to system-on-chip 602 for further processing. In some embodiments, the image data processed by system-on-chip 602 is displayed on display 610, stored in system memory 617, persistent storage device 616, or transmitted to a remote computing device via a network connection. The raw image data generated by image sensor 601 may be in Bayer color filter array (CFA) style (hereinafter also referred to as "Bayer style").

[0072] Flash controller 605 is a component used to control the variable characteristics of flash unit 604. Some adjustable attributes of flash unit 604 profile include flash duration, flash spectrum, and angular profile. For example, some flash unit 604 devices may include flash units with adjustable intensity, and some flash unit devices may have multiple flash units that may have different emission spectra, which can be activated independently to control the angular profile or spectrum of light emitted from the flash units. Angular profile refers to the pattern and spread of light emitted from the flash units over a given area, and how this spread changes at different angles relative to the flash unit. This can include how the intensity and distribution of light change as it moves laterally from a central axis located directly in front of the flash unit.

[0073] Motion sensor 619 is a component or set of components used to sense the motion of device 600. Motion sensor 619 can generate sensor signals indicating the orientation and / or acceleration of device 600. The sensor signals are sent to on-chip system 602 for various operations, such as rotating an image displayed on display 610 and tracking the motion of image sensor 601 during image capture.

[0074] Display 610 is a component used to display images generated by system-on-chip 602. Display 610 may include, for example, a liquid crystal display (LCD) device or an organic light-emitting diode (OLED) device. Based on data received from system-on-chip 602, display 610 may display various images, such as menus, selected operating parameters, images captured by image sensor 601 and processed by system-on-chip 602, and / or other information received from the user interface (not shown) of device 600.

[0075] System memory 617 is a component used to store instructions executed by system-on-chip 602 and to store data processed by system-on-chip 602. System memory 617 can be embodied in any type of memory, including, for example, dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate (DDR, DDR2, DDR3, etc.) RAMBUS DRAM (RDRAM), static RAM (SRAM), or combinations thereof. In some embodiments, system memory 617 can store pixel data or other image data or statistical values ​​in various formats. System memory 617 can be accessed by many components of system-on-chip 602, including but not limited to central processing unit 606, graphics processing unit 612, and neural engine 620.

[0076] Persistent storage device 616 is a component used to store data in a non-volatile manner. Even when power is unavailable, persistent storage device 616 retains the data. Persistent storage device 616 can be embodied as read-only memory (ROM), NAND or NOR flash memory, or other non-volatile random access memory devices.

[0077] The system-on-a-chip (SoC) 602 is embodied as one or more integrated circuit (IC) chips and performs various data processing procedures. The SoC 602 may include an image signal processor 603, one or more central processing units 606, a network interface 607, a sensor interface 608, a display controller 609, one or more graphics processing units 612, a memory controller 613, a video encoder 614, a memory controller 615, one or more neural engines 620, various other input / output (I / O) interfaces 611 and buses 618, and other components. Some components of the SoC 602 may be directly connected to the system memory 617, while other components are connected to other components via the bus 618. The SoC 602 may include more than Figure 6 The components shown have more or fewer components.

[0078] The image signal processor 603 (ISP) is hardware that executes the various stages of the image processing pipeline. In some embodiments, the image signal processor 603 may receive raw image data from the image sensor 601 and process the raw image data into a form that can be used by other sub-components of the system-on-chip 602 or components of the device 600. The image signal processor 603 can perform various image processing operations, such as image panning, horizontal and vertical scaling, color space conversion, and / or image stabilization transformations, as referenced below. Figure 2 Detailed description.

[0079] The Central Processing Unit 606 (CPU) can be embodied using any suitable instruction set architecture and can be configured to execute instructions defined in that instruction set architecture. The CPU 606 can be a general-purpose or embedded processor using any of a variety of instruction set architectures (ISAs), such as x86, PowerPC, SPARC, RISC, ARM, or MIPS ISA, or any other suitable ISA. Although Figure 6 The example illustrates a single CPU, but the System-on-a-Chip 602 can include multiple CPUs. In a multiprocessor system, each CPU may collectively implement the same ISA, but this is not required.

[0080] The graphics processing unit 612 (GPU) is a graphics processing circuit system for executing graphics data. For example, the GPU can render objects to be displayed into a frame buffer (e.g., a frame buffer that includes pixel data for the entire frame). The graphics processing unit 612 may include one or more graphics processors that can execute graphics software to perform some or all of the graphics operations or hardware acceleration of some graphics operations.

[0081] The Always-On Processor (AOP) 622 acts as the component within the device that enables the execution of the real-time operating system (even when the Central Processing Unit 606 is powered off or in sleep mode). As a distinct energy-efficient processor, the AOP operates autonomously relative to the Central Processing Unit 606, ensuring that specific functions remain accessible even when "power off." To manage various sensors and chips within the device, including motion sensors, Bluetooth, and ultra-wideband chips, the AOP facilitates seamless connectivity and operation. The AOP has the ability to wake up the Central Processing Unit 606 as needed, effectively conserving battery life. The AOP 622 is capable of independently performing critical tasks, ensuring uninterrupted functionality.

[0082] The Neural Engine 620 includes one or more processing cores optimized for machine learning tasks, including training and inference. It enables rapid processing of artificial intelligence (AI) and machine learning (ML) operations. Optimized for tasks such as advanced image processing, natural language processing, and pattern recognition, the Neural Engine 620 significantly improves the efficiency and speed of AI-related processes. Its architecture is designed to support a wide range of machine learning models while being highly energy efficient, thereby enhancing the user experience through faster, more responsive applications and functionalities that rely on AI and ML technologies.

[0083] I / O interface 611 is hardware, software, firmware, or a combination thereof used for interfacing with various input / output components in device 600. I / O components may include devices such as keyboards, buttons, audio devices, and sensors such as GPS sensors. I / O interface 611 processes data for sending data to such I / O components or processes data received from such I / O components.

[0084] Network interface 607 enables data to be exchanged between device 600 and other devices via one or more networks (e.g., carrier or proxy devices). For example, video or other image data can be received from other devices via network interface 607 and stored in system memory 617 for subsequent processing (e.g., via networks such as those described below). Figure 2The image signal processor 603 described herein is connected to a back-end interface and a display. The network may include, but is not limited to, a local area network (LAN) (e.g., Ethernet or a corporate network) and a wide area network (WAN). Image data received via network interface 607 can undergo image processing performed by image signal processor 603.

[0085] Sensor interface 608 is a circuit system used to interface with motion sensor 619. Sensor interface 608 receives sensor information from motion sensor 619 and processes the sensor information to determine the orientation or movement of device 600.

[0086] Display controller 609 is a circuit system for sending image data to be displayed on display 610. Display controller 609 receives image data from image signal processor 603, central processing unit 606, graphics processing unit 612 or system memory 617, and processes the image data into a format suitable for display on display 610.

[0087] The memory controller 613 is a circuit system for communicating with the system memory 617. The memory controller 613 can read data from the system memory 617 for processing by the image signal processor 603, the central processing unit 606, the graphics processing unit 612, or other sub-components of the system-on-a-chip 602. The memory controller 613 can also write data received from various sub-components of the system-on-a-chip 602 into the system memory 617.

[0088] The video encoder 614 is hardware, software, firmware, or a combination thereof, used to encode video data into a format suitable for storage in permanent storage device 616, or to pass data to network interface 607 for transmission over a network to another device.

[0089] In some implementations, one or more components of the system-on-a-chip 602, or some functions of these components, may be executed by software components running on the image signal processor 603, the central processing unit 606, or the graphics processing unit 612. Such software components may be stored in system memory 617, permanent storage device 616, or in another device communicating with device 600 via network interface 607.

[0090] Image or video data can flow through various data paths within the system-on-chip 602. In one example, raw image data may be generated by the image sensor 601 and processed by the image signal processor 603 before being sent to the system memory 617. After the image data is stored in the system memory 617, it may be accessed by the graphics processing unit 612, the neural engine 620, and / or the video encoder 614 for encoding or display 610.

[0091] In another example, image data is received from a source other than image sensor 601. For example, video data may be streamed, downloaded, or otherwise communicated to system-on-chip 602 via a wired or wireless network. Image data may be received via network interface 607 and written to system memory 617 via memory controller 613. The image data can then be retrieved from system memory 617 and processed by image signal processor 603, graphics processing unit 612, or neural engine 620. The image data can then be returned to system memory 617.

[0092] Figure 7 An example of a computing system 700 is shown. This computing system can be, for example, any computing device constituting AOP, Application Launch Service 206, or any other component thereof, wherein components of the system communicate with each other using connection 702. Connection 702 can be a physical connection via a bus or a direct connection to processor 704, such as in a chipset architecture. Connection 702 can also be a virtual connection, a networking connection, or a logical connection.

[0093] In some embodiments, computing system 700 is a distributed system, wherein the functions described herein can be distributed across a data center, multiple data centers, a peer-to-peer network, etc. In some embodiments, one or more system components described represent a number of such components, each performing one of the functions described for some or all of the functions. In one embodiment, these components may be physical devices or virtual devices.

[0094] The example computing system 700 includes at least one processing unit (CPU or processor) 704 and a connection 702 that couples various system components (including system memory 708, such as read-only memory (ROM) 710 and random access memory (RAM) 712) to the processor 704. The computing system 700 may include a cache of high-speed memory 708 that is directly connected to, close to, or integrated into the processor 704.

[0095] Processor 704 may include any general-purpose processor and hardware or software services (such as services 716, 718, and 720 stored in storage device 714), which are configured to control processor 704 and, in the case of software instructions being incorporated into the actual processor design, dedicated processors. Processor 704 may essentially be a completely independent computing system, containing multiple cores or processors, buses, memory controllers, caches, etc. Multi-core processors may be symmetric or asymmetric.

[0096] To enable user interaction, the computing system 700 includes an input device 726, which can represent any number of input mechanisms, such as a microphone for voice, a touch-sensitive screen for gesture or graphical input, a keyboard, a mouse, motion input, voice input, etc. The computing system 700 may also include an output device 722, which can be one or more of a plurality of output mechanisms known to those skilled in the art. In some cases, a multi-mode system enables the user to provide multiple types of input / output to communicate with the computing system 700. The computing system 700 may include a communication interface 724, which typically controls and manages user input and system output. Operation is not limited to any particular hardware arrangement; therefore, the basic features described herein can be readily replaced with improved hardware or firmware arrangements after such arrangements are developed.

[0097] Storage device 714 may be a non-volatile memory device and may be a hard disk or other type of computer-readable medium that can store data accessible by a computer, such as magnetic tape cassettes, flash memory cards, solid-state memory devices, digital universal optical discs, cartridges, random access memory (RAM), read-only memory (ROM), and / or some combination of these devices.

[0098] Storage device 714 may include software services, servers, etc., which enable the system to perform functions when the code defining such software is executed by processor 704. In some embodiments, hardware services that perform specific functions may include software components stored in a computer-readable medium and combined with necessary hardware components (such as processor 704, connection 702, output device 722, etc.) to perform functions.

[0099] For clarity, in some cases, this technology may be presented as comprising individual functional blocks, including functional blocks comprising devices, device components, steps or routines in methods embodied in software, or combinations of hardware and software.

[0100] Any of the steps, operations, functions, or processes described herein may be performed or implemented by a combination of hardware and software services, or individually or in combination with other devices. In some embodiments, a service may be software residing in the memory of a user device and / or one or more servers of a content management system and performing one or more functions while the processor executes the software associated with the service. In some embodiments, a service is a program or set of programs that performs a specific function. In some embodiments, a service may be considered a server. The memory may be a non-transitory computer-readable medium.

[0101] In some implementations, computer-readable storage devices, media, and memories may include cables or wireless signals containing bit streams. However, by reference, non-transitory computer-readable storage media explicitly excludes media such as energy, carrier signals, electromagnetic waves, and the signal itself.

[0102] The methods described in the examples above can be implemented using computer-executable instructions stored in or otherwise obtainable from a computer-readable medium. Such instructions may include, for example, instructions and data that cause a general-purpose computer, a special-purpose computer, or a special-purpose processing device, or otherwise configure such device to perform a function or a set of functions. The portion of the computer resources used may be accessible via a network. The executable computer instructions may be, for example, binary, intermediate format instructions such as assembly language, firmware, or source code. Examples of computer-readable media that can be used to store instructions, information used, and / or information generated during the methods according to the examples include disks or optical discs, solid-state storage devices, flash memory, USB devices with non-volatile memory, networked storage devices, etc.

[0103] Devices implementing the methods disclosed herein may include hardware, firmware, and / or software, and may take any of a variety of form factors. Typical examples of such form factors include servers, laptops, smartphones, small form factor personal computers, personal digital assistants, etc. The functionality described herein may also be embodied in peripheral devices or expansion cards. As another example, this functionality may also be implemented in different processes executed between different chips on a circuit board or in a single device.

[0104] Instructions, media for transmitting such instructions, computing resources for executing such instructions, and other structures for supporting such computing resources are components for providing the functionality described in these disclosures.

[0105] For clarity, in some cases, this technology may be presented as comprising individual functional blocks, including devices, device components, steps or routines in methods embodied in software, or combinations of hardware and software.

[0106] Any of the steps, operations, functions, or processes described herein may be performed or implemented by a combination of hardware and software services, or individually or in combination with other devices. In some embodiments, a service may be software residing in the memory of a user device and / or one or more servers of a content management system and performing one or more functions while the processor executes the software associated with the service. In some embodiments, a service is a program or set of programs that performs a specific function. In some embodiments, a service may be considered a server. The memory may be a non-transitory computer-readable medium.

[0107] In some implementations, computer-readable storage devices, media, and memories may include cables or wireless signals containing bit streams. However, by reference, non-transitory computer-readable storage media explicitly excludes media such as energy, carrier signals, electromagnetic waves, and the signal itself.

[0108] The methods described in the examples above can be implemented using computer-executable instructions stored in or otherwise obtainable from a computer-readable medium. Such instructions may include, for example, instructions and data that cause a general-purpose computer, a special-purpose computer, or a special-purpose processing device, or otherwise configure such device to perform a particular function or set of functions. The portion of the computer resources used may be accessible via a network. The computer-executable instructions may be, for example, binary intermediate format instructions, such as assembly language, firmware, or source code. Examples of computer-readable media that can be used to store instructions, information used, and / or information generated during the methods according to the examples include disks or optical discs, solid-state storage devices, flash memory, USB devices with non-volatile memory, networked storage devices, etc.

[0109] Devices implementing the methods disclosed herein may include hardware, firmware, and / or software, and may take any of a variety of form factors. Typical examples of such form factors include servers, laptops, smartphones, small form factor personal computers, personal digital assistants, etc. The functionality described herein may also be embodied in peripheral devices or expansion cards. As another example, this functionality may also be implemented in different processes executed between different chips on a circuit board or in a single device.

[0110] Instructions, media for transmitting such instructions, computing resources for executing such instructions, and other structures for supporting such computing resources are components for providing the functionality described in these disclosures.

[0111] While various examples and other information are used to interpret aspects within the scope of the appended claims, the claims should not be construed as limiting based on specific features or arrangements in such examples, as those skilled in the art will be able to extend these examples to various specific embodiments. Although the subject matter has been described in language specific to structural features and / or method steps, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or behaviors described. For example, such functionality may be distributed or performed differently in components other than those identified herein. Rather, the described features and steps are disclosed as examples of components of systems and methods within the scope of the appended claims.

[0112] Some aspects of this technology include:

[0113] Clause 1. A method for detecting application launch initiated due to unintentional user-device interaction, the method comprising: receiving an indication that an application is launched and one or more media capture devices on a user device are capturing one or more streams or frames surrounding a region of the user device; analyzing the one or more streams from the one or more media capture devices on the user device, wherein a metric is determined from the analyzed one or more streams; determining that the application was launched due to unintentional user-device interaction when the metric indicates that one or more media capture devices are experiencing an obstruction; and executing a first buffer after determining that the application was launched due to unintentional user-device interaction. The mitigation operation includes a first mitigation operation comprising reducing the power state of the user device to a first power state and dimming the display of the user device; after initiating the first mitigation operation, monitoring the user interface to identify one or more user interactions indicating a user intent, which is to launch an application or use the application to capture one or more frames; when a timeout occurs and no such user interaction is received, determining that the application was launched due to unintentional user-device interaction; and after determining that the application was launched due to unintentional user-device interaction, performing a second mitigation operation, the second mitigation operation comprising closing the launched application and reducing the power consumption of the user device to a second power state.

[0114] Clause 2. The method described in Clause 1 further includes: analyzing signals received from one or more sensors of the user equipment to identify the presence of an obstruction in the area surrounding the user equipment.

[0115] Clause 3. The method according to Clauses 1 to 2 further includes: after initiating the first mitigation operation, continuing the analysis of the one or more streams from the one or more media capture devices to detect whether the one or more streams begin to include objects or scenes of potential interest, wherein determining that the application was launched due to unintentional user-device interaction includes: a timeout without receiving user interaction, and the one or more streams not including objects or scenes of potential interest.

[0116] Clause 4. The method described in Clauses 1 to 3, wherein, in addition to analyzing the one or more streams, data from the proximity sensor is determined to indicate that the user device is near an object, which also indicates that the application was launched due to unintentional user-device interaction.

[0117] Clause 5. The method according to Clauses 1 to 4 further comprises: identifying frames of media captured from the one or more streams, including media captured from the one or more media capture devices; extracting frequencies from the media in the identified frames, wherein the frequencies are analyzed to determine whether the frequencies have met the criteria associated with an identification of a potentially relevant object or scene; and, upon determining that the frequencies exceed a threshold indicating that the media in the frame is meaningless, initiating obstruction detection to determine whether an obstruction indicating unintentional activation of the application has been detected.

[0118] Clause 6. The method according to Clauses 1 to 5, wherein the metric extracted from the one or more streams includes one or more sensor metrics, the sensor metrics including an estimated light intensity noise level and a sensor gain detected in the one or more streams; wherein the obstruction is determined to be close to the user equipment after determining that the one or more sensor metrics have met a set of criteria.

[0119] Clause 7. The method according to Clauses 1 to 6 further includes: initiating dimming of the display of the user device in response to the execution of the first mitigation operation; detecting user interaction with the user device in response to the dimming of the display, the interaction including one or more of interaction with a physical button, a software button or an application icon on the display of the user device; and launching the application and exiting the first mitigation operation in response to the user interaction.

[0120] Clause 8. The method according to Clauses 1 to 7, wherein receiving an instruction that the application is launched includes: determining which of the one or more media capture devices is active on the user device; and receiving data from the active media capture device about objects or scenes of potential interest surrounding the user device.

[0121] Clause 9. The method according to Clauses 1 to 8, wherein the metrics extracted from these streams include metadata associated with these media capture devices and one or more of one or more sensors, including proximity sensors, accelerometers, and gyroscopes.

[0122] Clause 10. The method according to Clauses 1 to 9, wherein monitoring the one or more streams to identify user interactions includes analyzing gestures detected by the media capture device to distinguish between intentional and unintentional interactions.

[0123] Clause 11. The method according to Clauses 1 to 10, wherein monitoring of the user interface includes user interaction with physical or software buttons, the user interaction being used to identify user intent to launch the application or to use the application to capture the one or more frames.

[0124] Clause 12. The method according to Clauses 1 to 11 further includes: receiving an indication that a button on the user device has been pressed, wherein the button is a physical button or a software button configured to initiate the launch of the application; causing the application launch service to initiate the launch of the application; determining from a proximity sensor whether there is an object near the proximity sensor or whether there is an obstruction that prevents the capture of one or more images; and after determining that an object is near the proximity sensor, instructing the application launch service to terminate the launch of the application.

Claims

1. A method for detecting application launch initiated due to unintentional user-device interaction, the method comprising: Receive an indication that the application has been launched and one or more media capture devices on the user device are capturing one or more streams or frames around a region of the user device; Analyze the one or more streams from the one or more media capture devices on the user device, wherein a metric is determined from the one or more streams being analyzed; When the metric indicates that one or more of the media capture devices are encountering an obstruction, it is determined that the application was launched due to the unintentional user-device interaction. After determining that the application was launched due to the unintentional user-device interaction, a first mitigation operation is performed, wherein the first mitigation operation includes reducing the power state of the user device to a first power state and dimming the display of the user device. After initiating the first mitigation operation, the user interface is monitored to identify one or more user interactions that indicate a user intent, which is related to launching the application or using the application to capture one or more frames. If the period times out without receiving one or more of the user interactions, it is determined that the application was launched due to the unintentional user-device interaction. as well as After determining that the application was launched due to the unintentional user-device interaction, a second mitigation operation is performed, which includes closing the launched application and reducing the power consumption of the user device to a second power state.

2. The method according to claim 1, further comprising: Analyze signals received from one or more sensors of the user equipment to identify the presence of obstructions in the area surrounding the user equipment.

3. The method according to claim 1, further comprising: After initiating the first mitigation operation, analysis continues on the one or more streams from the one or more media capture devices to detect whether the one or more streams begin to include objects or scenes of potential interest. Determining that the application was launched due to the unintentional user-device interaction includes: the period timed out without receiving user interaction, and one or more streams do not include the object or scenario that may be of interest.

4. The method of claim 1, wherein, in addition to analyzing the one or more streams, data from a proximity sensor is determined to indicate that the user device is near an object, which also indicates that the application was launched due to unintentional user-device interaction.

5. The method according to claim 1, further comprising: The one or more stream identifiers include frames of media captured from the one or more media capture devices; Frequency is extracted from the media in the identified frames, wherein the frequency is analyzed to determine whether the frequency has met the criteria associated with an identifier of a potentially relevant object or scene; and When it is determined that the frequency exceeds a threshold indicating that the media in the frame is meaningless, obstruction detection is initiated to determine whether an obstruction indicating unintentional activation of the application has been detected.

6. The method of claim 1, wherein the metrics extracted from the one or more streams include one or more sensor metrics, the one or more sensor metrics comprising: The estimated light intensity noise level and the sensor gain detected in the one or more streams; in After determining that the measurements of the one or more sensors have met a set of criteria, the obstruction is determined to be close to the user equipment.

7. The method according to claim 1, further comprising: In response to the execution of the first mitigation operation, the dimming of the display of the user equipment is initiated; In response to dimming the display, a user interaction with the user device is detected, the interaction including one or more of an interaction with a physical button, a software button, or an application icon on the display of the user device. In response to the user interaction, the application is launched and the first mitigation operation is exited.

8. The method of claim 1, wherein receiving an indication that the application has been launched comprises: Determine which of the one or more media capture devices is active on the user device; Receive data from the active media capture device about objects or scenes that may be of interest around the user device.

9. The method of claim 1, wherein the metric extracted from the stream includes metadata associated with the media capture device and one or more of one or more sensors, including a proximity sensor, an accelerometer, and a gyroscope.

10. The method of claim 1, wherein monitoring the one or more streams to identify user interactions includes analyzing gestures detected by the media capture device to distinguish between intentional and unintentional interactions.

11. The method of claim 1, wherein the monitoring of the user interface includes user interaction with a physical button or a software button, the user interaction being used to identify user intent regarding launching the application or using the application to capture the one or more frames.

12. The method according to claim 1, further comprising: Receive an indication that a button on the user device has been pressed, wherein the button is a physical button or a software button configured to initiate the launch of the application; Initiate the launch of the application by the application launch service; The proximity sensor determines whether there is an object near the proximity sensor or whether there is an obstruction that prevents one or more images from being captured; as well as After determining that the object is close to the proximity sensor, the application startup service is instructed to terminate the startup of the application.

13. A non-transitory computer-readable medium comprising instructions, said instructions, when executed by a computing system, causing the computing system to: An indication that an application has been launched and one or more media capture devices on the user device are capturing one or more streams or frames around a region of the user device; Analyze the one or more streams from the one or more media capture devices on the user device, wherein a metric is determined from the one or more streams being analyzed; When the metric indicates that one or more of the media capture devices are encountering an obstruction, it is determined that the application was launched due to unintentional user-device interaction. After determining that the application was launched due to the unintentional user-device interaction, a first mitigation operation is performed, wherein the first mitigation operation includes reducing the power state of the user device to a first power state and dimming the display of the user device. After initiating the first mitigation operation, the user interface is monitored to identify one or more user interactions that indicate a user intent, which is related to launching the application or using the application to capture one or more frames. If the period times out without receiving one or more of the user interactions, it is determined that the application was launched due to the unintentional user-device interaction. as well as After determining that the application was launched due to the unintentional user-device interaction, a second mitigation operation is performed, which includes canceling the application launch and reducing the power consumption of the user device to a second power state.

14. The non-transitory computer-readable medium of claim 13, wherein the computer-readable medium further comprises instructions that, when executed by the computing system, cause the computing system to: Analyze signals received from one or more sensors of the user equipment to identify the presence of obstructions in the area surrounding the user equipment.

15. The non-transitory computer-readable medium of claim 13, wherein the computer-readable medium further comprises instructions, which, when executed by the computing system, cause the computing system to: After initiating the first mitigation operation, analysis continues on the one or more streams from the one or more media capture devices to detect whether the one or more streams begin to include objects or scenes of potential interest. Determining that the application was launched due to the unintentional user-device interaction includes: The period times out without receiving user interaction, and one or more streams do not include the object or scenario that may be of interest.

16. The non-transitory computer-readable medium of claim 13, wherein, in addition to analyzing the one or more streams, data from a proximity sensor is determined to indicate that the user device is near an object, which also indicates that the application was launched due to unintentional user-device interaction.

17. The non-transitory computer-readable medium of claim 13, wherein the computer-readable medium further comprises instructions that, when executed by the computing system, cause the computing system to: The one or more stream identifiers include frames of media captured from the one or more media capture devices; Frequency extraction is performed from the media in the identified frames, wherein the frequencies are analyzed to determine whether they meet a set of criteria associated with an identifier of a potentially relevant object or scene; and When it is determined that the frequency has met a set of criteria indicating that the media in the frame is meaningless, obstruction detection is initiated to determine whether an obstruction indicating unintentional activation of the application has been detected.

18. The non-transitory computer-readable medium according to claim 13, wherein: The metrics extracted from the one or more streams include one or more sensor metrics, which include the estimated light intensity noise level and the sensor gain detected in the one or more streams; The obstruction is determined to be close to the user equipment when the one or more sensor measurements meet a set of criteria used to identify objects or scenes of potential interest.

19. The non-transitory computer-readable medium of claim 13, wherein the computer-readable medium further comprises instructions that, when executed by the computing system, cause the computing system to: In response to the execution of the first mitigation operation, the dimming of the display of the user equipment is initiated; In response to dimming the display, a user interaction with the user device is detected, the interaction including one or more of an interaction with a physical button, a software button, or an application icon on the display of the user device. In response to the user interaction, the application is launched and the first mitigation operation is exited.

20. The non-transitory computer-readable medium of claim 13, wherein the computer-readable medium further comprises instructions that, when executed by the computing system, cause the computing system to: Determine which of the one or more media capture devices is active on the user device; Receive data from the active capture device about objects or scenes of interest surrounding the user device.

21. The non-transitory computer-readable medium of claim 13, wherein the monitoring of the user interface includes user interaction with a physical button or a software button, the user interaction being used to identify the user intent to launch the application or to use the application to capture the one or more frames.

22. A computing system, the computing system comprising: processor; and The memory stores instructions that, when executed by the processor, configure the computing system to: An indication that an application has been launched and one or more media capture devices on the user device are capturing one or more streams or frames around a region of the user device; Analyze one or more streams from one or more media capture devices on the user device, wherein a metric is determined from the analyzed one or more streams; When the metric indicates that one or more of the media capture devices are encountering an obstruction, it is determined that the application was launched due to unintentional user-device interaction. After determining that the application was launched due to the unintentional user-device interaction, a first mitigation operation is performed, wherein the first mitigation operation includes reducing the power state of the user device to a first power state and dimming the display of the user device. After initiating the first mitigation operation, the user interface is monitored to identify one or more user interactions that indicate a user intent, which is related to launching the application or using the application to capture one or more frames. If the period times out without receiving one or more of the user interactions, it is determined that the application was launched due to the unintentional user-device interaction. as well as After determining that the application was launched due to the unintentional user-device interaction, a second mitigation operation is performed, which includes canceling the application launch and reducing the power consumption of the user device to a second power state.

23. The computing system of claim 22, further configured to: Analyze signals received from one or more sensors of the user equipment to identify the presence of obstructions in the area surrounding the user equipment.

24. The computing system of claim 23, further configured to: Receive an indication that a button on the user device has been pressed, wherein the button is a physical button or a software button configured to initiate the launch of the application; The proximity sensor determines whether there is an object near the proximity sensor or whether there is an obstruction that prevents the capture of one or more images. The proximity determination indicates that there is no potentially relevant scene to be captured by the one or more media capture devices. as well as If it is determined that the object is not near the proximity sensor, instruct the application to start the service or not to start the application.

Images