Apparatus and method for detecting user intention for image capture or video recording

Abstract

In one embodiment, a method includes, in response to initiating a shooting mode of a camera application on an electronic device, collecting sensor information including at least one of motion data of the electronic device, location data of the electronic device, and image data captured by one or more imaging sensors of the electronic device, the shooting mode represents at least one of a video recording mode and an image capturing mode. The method further includes determining, by using the trained machine learning model, whether the user intends to record a video or capture an image based on features extracted from the sensor information. The method further includes, in response to determining that the user intent is to record the video, regardless of the shooting mode, or in response to determining that the user intent is to capture an image, regardless of the shooting mode, capturing the image.

Description

Technical Field

[0001] The present disclosure relates generally to imaging devices and processes. More specifically, the present disclosure relates to apparatus and methods for detecting user intent to start video recording. Background Art

[0002] Many electronic devices have sophisticated camera capabilities. However, as these capabilities expand and more options become available, cameras become increasingly difficult for users to select and apply the correct tools and features. When trying to capture a particular moment, looking at the tools and settings may cause the particular moment to pass by without being captured. Summary of the invention

[0003] Solution to the problem

[0004] The present disclosure relates to a system and method for detecting user intent to start video recording.

[0005] In an embodiment, a method includes: in response to initiating a shooting mode of a camera application on an electronic device, collecting sensor information, the sensor information including at least one of the following items: motion data of the electronic device, position data of the electronic device, and image data captured by one or more imaging sensors of the electronic device, wherein the shooting mode represents at least one of the following items: a video recording mode and an image capture mode. The method also includes: determining whether a user intends to record a video or capture an image based on features extracted from the sensor information by using a trained machine learning model. The method also includes: in response to determining that the user intends to record a video, recording a video regardless of the shooting mode, or in response to determining that the user intends to capture an image, capturing an image regardless of the shooting mode.

[0006] In an embodiment, an electronic device includes one or more imaging sensors and at least one processing device, the at least one processing device being operably coupled to the one or more imaging sensors. The at least one processing device is configured to: in response to initiating a shooting mode of a camera application on the electronic device, collect sensor information, the sensor information including at least one of the following items: motion data of the electronic device, position data of the electronic device, and image data captured by the one or more imaging sensors of the electronic device, wherein the shooting mode represents at least one of the following items: a video recording mode and an image capture mode. The at least one processing device is also configured to: determine whether a user intends to record a video or capture an image based on features extracted from the sensor information by using a trained machine learning model. The at least one processing device is also configured to: in response to determining that the user intends to record a video, record a video regardless of the shooting mode, or in response to determining that the user intends to capture an image, capture an image regardless of the shooting mode.

[0007] In an embodiment, a machine-readable medium includes instructions that, when executed, cause at least one processor of an electronic device to: in response to initiating a shooting mode of a camera application on the electronic device, collect sensor information, the sensor information including at least one of the following items: motion data of the electronic device, location data of the electronic device, and image data captured by one or more imaging sensors of the electronic device, wherein the shooting mode represents at least one of the following items: a video recording mode and an image capture mode. The non-transitory machine-readable medium also includes instructions that, when executed, cause the at least one processor to: determine whether a user intends to record a video or capture an image based on features extracted from the sensor information by using a trained machine learning model. The non-transitory machine-readable medium also includes instructions that, when executed, cause the at least one processor to: in response to determining that the user intends to record a video, record a video regardless of the shooting mode, or in response to determining that the user intends to capture an image, capture an image regardless of the shooting mode.

[0008] Other technical features may be readily understood by those skilled in the art based on the following drawings, descriptions and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] For a more complete understanding of the present disclosure and its advantages, reference will now be made to the following description taken in conjunction with the accompanying drawings, wherein like reference numerals represent like parts:

[0010] Figure 1 An example network configuration including electronic devices according to the present disclosure is shown;

[0011] Figure 2 An example pipeline for detecting user intent for a capture mode for a camera application according to the present disclosure is shown;

[0012] Figure 3 An example process for automatically capturing the best moment to take a group photo in accordance with the present disclosure is shown;

[0013] Figure 4 An example process for training a machine learning model to determine user intent for a capture mode according to the present disclosure is shown;

[0014] Figure 5 An example user interface for detecting user intent for a capture mode for a camera application according to the present disclosure is shown; and

[0015] Figure 6 An example method for detecting a user's intent to start video recording using an electronic device according to the present disclosure is shown. DETAILED DESCRIPTION

[0016] Before proceeding to the following detailed description, it may be advantageous to set forth definitions of certain words and phrases used throughout this patent document. The terms "send," "receive," and "communicate," and their derivatives, include both direct and indirect communications. The terms "include" and "comprises," and their derivatives, are inclusive and not limiting. The term "or" is inclusive, meaning and / or. The phrase "associated with," and its derivatives, means include, be included within, be interconnected with, include, be contained within, be connected to or connected with, be coupled to or coupled with, be communicable with, collaborate with, be intertwined, be juxtaposed, be close to, be bound to or bound with, have, have the property of, have a relationship to or with, etc.

[0017] In addition, the various functions described below may be implemented or supported by one or more computer programs, each of which is formed by a computer-readable program code and implemented in a computer-readable medium. The terms "application" and "program" refer to one or more computer programs, software components, instruction sets, processes, functions, objects, classes, instances, related data, or parts thereof suitable for implementation in appropriate computer-readable program codes. The phrase "computer-readable program code" includes any type of computer code, including source code, object code, and executable code. The phrase "computer-readable medium" includes any type of medium that can be accessed by a computer, such as a read-only memory (ROM), a random access memory (RAM), a hard drive, a compact disk (CD), a digital video disk (DVD), or any other type of memory. "Non-transitory" computer-readable media do not include wired, wireless, optical, or other communication links that transmit transient electrical or other signals. Non-transitory computer-readable media include media in which data can be permanently stored and media in which data can be stored and subsequently rewritten, such as rewritable optical disks or erasable memory devices.

[0018] As used herein, terms and phrases such as "having", "may have", "include", or "may include" a feature (such as a number, function, operation, or component such as a part) indicate the presence of the feature without excluding the presence of other features. In addition, as used herein, the phrases "A or B", "at least one of A and / or B", or "one or more of A and / or B" may include all possible combinations of A and B. For example, "A or B", "at least one of A and B", or "at least one of A or B" may indicate all of the following situations: (1) including at least one A, (2) including at least one B, or (3) including at least one A and at least one B. In addition, as used herein, the terms "first" and "second" may modify various components regardless of importance and do not limit these components. These terms are only used to distinguish components from each other. For example, a first user device and a second user device may indicate user devices that are different from each other, regardless of the importance or order of the devices. Without departing from the scope of the present disclosure, a first component may be represented as a second component, and vice versa.

[0019] It should be understood that when an element (such as a first element) is referred to as being (operably or communicatively) "coupled to" or "connected to" or "coupled to" or "connected to" another element (such as a second element), the element may be directly coupled to or connected to the other element, or may be coupled to or connected to the other element via a third element. Conversely, it should be understood that when an element (such as a first element) is referred to as being "directly coupled to" or "directly connected to" another element (such as a second element), or "directly coupled to" or "directly connected to" another element (such as a second element), no other element (such as a third element) is interposed between the element and the other element.

[0020] As used herein, the phrase "configured (or set) to" may be used interchangeably with the phrases "suitable for", "capable of", "designed to", "suitable for", "manufactured to", or "capable of...", depending on the scenario. The phrase "configured (or set) to" does not necessarily mean "specially designed in hardware to". On the contrary, the phrase "configured to" may mean that a device is able to perform an operation together with another device or component. For example, the phrase "a processor configured (or set) to perform A, B, and C" may represent a general-purpose processor (such as a CPU or application processor) that can perform operations by executing one or more software programs stored in a memory device, or a special-purpose processor (such as an embedded processor) for performing operations.

[0021] The terms and phrases used herein are provided only to describe some embodiments of the present disclosure, rather than to limit the scope of other embodiments of the present disclosure. It should be understood that the singular forms "one", "an" and "the" include plural references unless the context clearly states otherwise. All terms and phrases used herein, including technical and scientific terms and phrases, have the same meanings as those of ordinary skill in the art to which the present disclosure embodiments belong. It should also be understood that terms and phrases such as those defined in common dictionaries should be interpreted as meanings consistent with their meanings in the context of the relevant technology, and should not be interpreted as ideal or overly formalized meanings, unless clearly defined as such herein. In some cases, the terms and phrases defined herein may be interpreted as excluding embodiments of the present disclosure.

[0022] Examples of "electronic devices" according to embodiments of the present disclosure may include at least one of the following: a smart phone, a tablet personal computer (PC), a mobile phone, a video phone, an e-book reader, a desktop PC, a laptop computer, a netbook computer, a workstation, a personal digital assistant (PDA), a portable multimedia player (PMP), an MP3 player, a mobile medical device, a camera, or a wearable device (such as smart glasses, a head mounted device (HMD), electronic clothing, an electronic bracelet, an electronic necklace, an electronic accessory, an electronic tattoo, a smart mirror, or a smart watch). Other examples of electronic devices include smart home appliances. Examples of smart home appliances may include at least one of the following: a television, a digital video disc (DVD) player, an audio player, a refrigerator, an air conditioner, a vacuum cleaner, an oven, a microwave oven, a washing machine, a dryer, an air purifier, a set-top box, a home automation control panel, a security control panel, a TV box (such as SAMSUNG HOMESYNC, APPLETV, or GOOGLE TV), a smart speaker or a speaker with an integrated digital assistant (such as SAMSUNG GALAXY HOME, APPLE HOMEPOD, or AMAZON ECHO), a game console (such as XBOX, PLAYSTATION, or NINTENDO), an electronic dictionary, an electronic key, a camera, or an electronic photo frame. Still other examples of electronic devices include at least one of the following: various medical devices (such as multifunctional portable medical measuring devices (such as blood sugar measuring devices, heart rate measuring devices, or body temperature measuring devices), magnetic resonance angiography (MRA) devices, magnetic resonance imaging (MRI) devices, computed tomography (CT) devices, imaging devices, or ultrasound devices), navigation devices, global positioning system (GPS) receivers, event data recorders (EDRs), flight data recorders (FDRs), automotive infotainment devices, marine electronic devices (such as marine navigation devices or gyrocompasses), avionics equipment, security equipment, vehicle head units, industrial or home robots, automated teller machines (ATMs), point-of-sale (POS) devices, or Internet of Things (IoT) devices (such as light bulbs, various sensors, electric or gas meters, sprinklers, fire alarms, thermostats, street lights, toasters, fitness equipment, hot water tanks, heaters, or kettles). Other examples of electronic devices include at least a portion of the following: a piece of furniture or a building / structure, an electronic board, an electronic signature receiving device, a projector, or various measuring devices (such as devices for measuring water, electricity, gas, or electromagnetic waves). Note that according to various embodiments of the present disclosure, the electronic device may be one of the devices listed above or a combination thereof. According to some embodiments of the present disclosure, the electronic device may be a flexible electronic device. The electronic device disclosed herein is not limited to the devices listed above, and may include any other electronic device now known or later developed.

[0023] In the following description, according to various embodiments of the present disclosure, an electronic device is described with reference to the accompanying drawings. As used herein, the term "user" may refer to a person using an electronic device or another device (such as an artificial intelligence electronic device).

[0024] Definitions for certain other words and phrases may be provided throughout this patent document. Those of ordinary skill in the art should understand that in many, if not most instances, such definitions apply to prior, as well as future uses of such defined words and phrases.

[0025] The description in this application should not be read as implying that any particular element, step, or function is an essential element that must be included in the claims scope. The scope of the patented subject matter is limited only by the claims.

[0026] The following will be described with reference to the accompanying drawings Figures 1 to 6 However, it should be understood that the present disclosure is not limited to these embodiments, and all changes and / or equivalents or replacements to these embodiments also belong to the scope of the present disclosure.

[0027] As mentioned above, many electronic devices have complex camera functions. However, as these functions expand and more options become available, cameras become increasingly difficult for users to select and apply the correct tools and functions. When trying to capture a specific moment, reviewing the tools and settings may cause the specific moment to pass without being captured. A specific example is image and video capture. For video capture of an impromptu moment, it is necessary to quickly execute a navigation user interface to start video capture to avoid missing some or all of the impromptu moment. On the contrary, if the user intends to capture an image but captures a video in advance, the user may find that the quality of the video that has been captured is lower than the quality of the expected image that should have been captured. Although many user interfaces provide buttons for switching to other image capture settings and small text indicating the camera's shooting mode, it may be difficult for the user to notice these prompts when trying to capture the moment.

[0028] As the number of options for starting a video increases and the user interface becomes more complex, the user may end up missing the event that was intended to be captured. In addition, some user interfaces may be poorly designed and confuse the user.

[0029] Embodiments of the present disclosure provide a machine learning system that allows an electronic device to determine the user's intent to quickly begin capturing with little to no user interaction with a user interface. Manual and "smart" copies of images and videos can be saved, allowing the user to select an appropriate capture after the event has passed in the event that the machine learning system is incorrect. A system can be implemented that recognizes the motion and position of an electronic device to associate a data set including examples of the electronic device used to capture images and videos. The machine learning system can use the data set to determine a high correlation between the user's current actions and known past actions, and determine an expected imaging mode for the camera.

[0030] Figure 1 An example network configuration 100 including an electronic device 101 according to the present disclosure is shown. Figure 1 The embodiment of the network configuration 100 shown is for illustration only. Other embodiments of the network configuration 100 may be used without departing from the scope of the present disclosure.

[0031] According to an embodiment of the present disclosure, an electronic device 101 is included in a network configuration 100. The electronic device 101 may include at least one of a bus 110, a processor 120, a memory 130, an input / output (I / O) interface 150, a display 160, a communication interface 170, and a sensor 180. In some embodiments, the electronic device 101 may not include at least one of these components, or at least one other component may be added. The bus 110 includes circuits for interconnecting components 120 to 180 and transmitting communications (such as control messages and / or data) between these components.

[0032] The processor 120 includes one or more processing devices, such as one or more microprocessors, microcontrollers, digital signal processors (DSPs), application specific integrated circuits (ASICs), or field programmable gate arrays (FPGAs). In some embodiments, the processor 120 includes one or more of a central processing unit (CPU), an application processor (AP), a communication processor (CP), or a graphics processing unit (GPU). The processor 120 is capable of performing control on at least one of the other components of the electronic device 101 and / or performing operations or data processing related to communication or other functions. As described below, the processor 120 can detect a user's intent to start video recording or image capture using an electronic device (such as a handheld phone), as described in more detail below.

[0033] The functions related to artificial intelligence according to the present disclosure are operated by at least one processor 102 and a memory 404. The processor 102 may include one or more processors. In this case, the one or more processors may be a general-purpose processor (such as a CPU, an AP, or a digital signal processor (DSP)), a graphics-specific processor (such as a GPU or a visual processing unit (VPU)), or an artificial intelligence-specific processor (such as an NPU). For example, when one or more processors are artificial intelligence-specific processors, the artificial intelligence-specific processor may be designed as a hardware structure specifically for processing a specific artificial intelligence model.

[0034] The processor 102 controls the input data to be processed according to the predefined operation rules or artificial intelligence models stored in the memory 130. Alternatively, when the processor 102 is an artificial intelligence dedicated processor, the artificial intelligence dedicated processor can be designed as a hardware structure specifically used to process a specific artificial intelligence model.

[0035] The predefined action rule or artificial intelligence model is characterized in that it is created by learning. Here, creation by learning means learning the basic artificial intelligence model using multiple learning data through a learning algorithm, thereby creating a predefined action rule or artificial intelligence model that is set to perform the desired characteristics (or purpose). This learning can be performed in the device itself that executes the artificial intelligence according to the present disclosure, or by a separate server and / or system. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the above examples.

[0036] The artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values, and the neural network operation is performed by the operation between the operation result of the previous layer and the multiple weight values. The multiple weights possessed by the multiple neural network layers can be optimized by the learning results of the artificial intelligence model. For example, multiple weights can be updated so that the loss value or cost value obtained from the artificial intelligence model is reduced or minimized during the learning process. The artificial neural network may include a deep neural network (DNN), such as a convolutional neural network (CNN), a deep neural network (DNN), a recursive neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recursive deep neural network (BRDNN) or a deep Q network, but is not limited to the above examples.

[0037] The memory 130 may include a volatile memory and / or a non-volatile memory. For example, the memory 130 may store commands or data related to at least one other component of the electronic device 101. According to an embodiment of the present disclosure, the memory 130 may store software and / or a program 140. The program 140 includes, for example, a kernel 141, middleware 143, an application programming interface (API) 145, and / or an application program (or "application") 147. At least a portion of the kernel 141, the middleware 143, or the API 145 may be represented as an operating system (OS).

[0038] The kernel 141 may control or manage system resources (such as the bus 110, the processor 120, or the memory 130) used to perform operations or functions implemented in other programs (such as the middleware 143, the API 145, or the application 147). The kernel 141 provides an interface that allows the middleware 143, the API 145, or the application 147 to access various components of the electronic device 101 to control or manage system resources. The application 147 may include one or more applications that detect user intent to start video recording using an electronic device (such as a handheld phone), etc. These functions may be performed by a single application or by multiple applications, each of which performs one or more of these functions.

[0039] The middleware 143 may act as a relay to, for example, allow the API 145 or the application 147 to communicate data with the kernel 141. A plurality of applications 147 may be provided. The middleware 143 is capable of controlling work requests received from the applications 147, such as by assigning a priority for using system resources (such as the bus 110, the processor 120, or the memory 130) of the electronic device 101 to at least one of the plurality of applications 147. The API 145 is an interface that allows the application 147 to control functions provided by the kernel 141 or the middleware 143. For example, the API 145 includes at least one interface or function (such as a command) for file control, window control, image processing, or text control.

[0040] The I / O interface 150 functions as an interface capable of transmitting commands or data input from a user or other external devices to other components of the electronic device 101, for example. The I / O interface 150 may also output commands or data received from other components of the electronic device 101 to the user or other external devices.

[0041] The display 160 includes, for example, a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a quantum dot light emitting diode (QLED) display, a micro-electromechanical system (MEMS) display, or an electronic paper display. The display 160 may also be a depth perception display, such as a multi-focal display. The display 160 can, for example, display various contents (such as text, images, videos, icons, or symbols) to the user. The display 160 may include a touch screen and may receive, for example, a touch, gesture, proximity, or hovering input by using an electronic pen or a body part of the user.

[0042] The communication interface 170 can, for example, establish communication between the electronic device 101 and an external electronic device (such as the first electronic device 102, the second electronic device 104, or the server 106). For example, the communication interface 170 can be connected to the network 162 or 164 through wireless or wired communication to communicate with the external electronic device. The communication interface 170 can be a wired or wireless transceiver or any other component for sending and receiving signals.

[0043] Wireless communication can use, for example, long term evolution (LTE), long term evolution-advanced (LTE-A), fifth generation wireless system (5G), millimeter wave or 60 GHz wireless communication, wireless USB, code division multiple access (CDMA), wideband code division multiple access (WCDMA), universal mobile telecommunications system (UMTS), wireless broadband (WiBro) or global system for mobile communications (GSM) as at least one of the cellular communication protocol. The wired connection can include, for example, at least one of universal serial bus (USB), high-definition multimedia interface (HDMI), recommended standard 232 (RS-232) or plain old telephone service (POTS). Network 162 or 164 includes at least one communication network, such as a computer network (such as a local area network (LAN) or a wide area network (WAN)), the Internet, or a telephone network.

[0044] The electronic device 101 also includes one or more sensors 180, which can measure physical quantities or detect the activation state of the electronic device 101 and convert the measured or detected information into electrical signals. For example, the sensor 180 may include one or more cameras or other imaging sensors that can be used to capture images of scenes. The sensor 180 may also include one or more buttons for touch input, one or more microphones, gesture sensors, gyroscopes or gyroscope sensors, air pressure sensors, magnetic sensors or magnetometers, speed sensors, acceleration sensors or accelerometers, grip sensors, proximity sensors, color sensors (such as red, green, and blue (RGB) sensors), biophysical sensors, temperature sensors, humidity sensors, illumination sensors, ultraviolet (UV) sensors, electromyography (EMG) sensors, electroencephalography (EEG) sensors, electrocardiography (ECG) sensors, infrared (IR) sensors, ultrasonic sensors, iris sensors, or fingerprint sensors. The sensor 180 may also include an inertial measurement unit, which may include one or more accelerometers, gyroscopes, and other components. In addition, the sensor 180 may include a control circuit for controlling at least one sensor included here. Any of these sensors 180 may be located within the electronic device 101 .

[0045] The first external electronic device 102 or the second external electronic device 104 may be a wearable device or a wearable device (such as an HMD) in which the electronic device can be installed. When the electronic device 101 is installed in the electronic device 102 (such as an HMD), the electronic device 101 may communicate with the electronic device 102 through the communication interface 170. The electronic device 101 may be directly connected to the electronic device 102 to communicate with the electronic device 102 without involving a separate network. The electronic device 101 may also be an augmented reality wearable device including one or more cameras, such as glasses.

[0046] The first external electronic device 102 and the second external electronic device 104 and the server 106 may each be a device of the same or different type as the electronic device 101. According to certain embodiments of the present disclosure, the server 106 includes a group of one or more servers. In addition, according to certain embodiments of the present disclosure, all or some operations performed on the electronic device 101 may be performed on another other electronic device or multiple other electronic devices (such as electronic devices 102 and 104 or server 106). In addition, according to certain embodiments of the present disclosure, when the electronic device 101 should automatically or upon request perform some functions or services, as an alternative or in addition to performing the function or service by itself, the electronic device 101 may request another device (such as electronic devices 102 and 104 or server 106) to perform at least some functions associated with it. Other electronic devices (such as electronic devices 102 and 104 or server 106) are capable of performing the requested functions or additional functions and transmitting the results of the execution to the electronic device 101. The electronic device 101 may provide the requested function or service by processing the received results as is or in addition. To this end, for example, cloud computing, distributed computing, or client-server computing technology may be used. Although Figure 1 The electronic device 101 is shown to include a communication interface 170 to communicate with an external electronic device 104 or a server 106 via a network 162 or 164 , but according to some embodiments of the present disclosure, the electronic device 101 may operate independently without a separate communication function.

[0047] The server 106 may include components that are the same or similar to the electronic device 101 (or a suitable subset thereof). The server 106 may support driving the electronic device 101 by performing at least one operation (or function) implemented on the electronic device 101. For example, the server 106 may include a processing module or processor that may support the processor 120 implemented in the electronic device 101. In some embodiments of the present disclosure, the server 106 may detect a user's intent to start video recording or image capture using an electronic device (such as a handheld phone), as described in more detail below.

[0048] although Figure 1 An example of a network configuration 100 including an electronic device 101 is shown, but the Figure 1 Various changes may be made. For example, network configuration 100 may include any number of individual components in any suitable arrangement. In general, computing and communication systems have a wide variety of configurations, and Figure 1 The scope of the present disclosure is not limited to any particular configuration. Figure 1 One operating environment is shown in which the various features disclosed in this patent document may be used, but these features may be used in any other suitable system.

[0049] Figure 2 An example pipeline 200 for detecting a user's intent to select a capture mode for a camera application according to the present disclosure is shown. For ease of illustration, the pipeline 200 is described as Figure 1 The network configuration 100 is executed on or supported by the electronic device 101, which in some embodiments may represent a cellular phone or other handheld device. Figure 2 The illustrated pipeline 200 may be used with any other suitable equipment and for any other suitable system.

[0050] like Figure 2 As shown, a camera application can be opened on the electronic device 101, which causes the front camera to be opened in operation 202 and the rear camera to be opened in operation 204. The camera application can display the front camera or the rear camera by default, or display the front camera or the rear camera based on the previous use of the camera application. The rear camera is a camera located on a surface or side of the electronic device 101 opposite to the display. The front camera is a camera located on the same surface or side of the electronic device 101 as the display. Although the following operations are described for capturing images or videos using the rear camera, similar functions can be reversed when capturing images or videos using the front camera.

[0051] In operation 206, the front camera of the electronic device 101 can capture the user's head and the user's face. The machine learning system can identify the user's head and the user's face in the frame captured by the rear camera. The user's head and the user's face can be captured continuously when the camera application is open, the user's head and the user's face can be captured when an input for recording a video or capturing an image is selected, or the user's head and the user's face can be captured at any other suitable time for capture. Based on the completion of video recording or image capture, or when the electronic device moves sufficiently that the subject is lost from the preview image or video, the capture of the user's head and the user's face can be terminated. In various embodiments, the user can be defined as a person using the camera application on the electronic device 101.

[0052] In operation 208, a head pose of the user's head and a facial expression of the user's face may be analyzed using a deep neural network (DNN). The user's face and the user's head may be analyzed to identify specific features of the user's intent for determining a shooting mode. For example, the electronic device 101 may identify eye gaze, mouth features, etc. from the user's face, and a head pose from the user's head.

[0053] In operation 210, the electronic device 101 may perform a user determination of user intent. The user determination of user intent may be performed using artificial intelligence (AI). The AI ​​may be trained to determine whether the user intends to record a video or capture an image based on the user's face and the user's head. Examples of determining user intent may include that the user is not ready to capture a video or image, that the user intends to record a video, and that the user intends to capture an image. For example, the AI ​​may determine that an amount of head movement exceeding a threshold indicates that the user is not ready to record a video or capture an image. The AI ​​may also determine that the user intends to record a video or capture an image based on the degree of eye movement or other facial expressions.

[0054] In operation 212, by using the rear camera, objects in the scene can be identified and tracked. Objects in the scene are typically people, but may also include cars, bicycles, pets, etc. These objects can be identified by the electronic device 101 or selected by the user. In some embodiments, the electronic device 101 can present the objects to the user in a user interface for confirmation. The number of objects to be identified and tracked in the image can be specified, such as the maximum number of objects. Key objects can be determined based on the duration of the scene (such as a minimum amount of time or a continuous amount of captured frames). When the camera application is turned on, key objects can also be determined based on presence within the scene.

[0055] In operation 214, the electronic device 101 may perform object determination of the user's intent. Based on the motion of these objects relative to the person holding the camera, the likelihood of the user's intent to record a video or capture an image may be determined. For example, if one or more persons are detected and the one or more persons are standing still while looking at the camera, the likelihood of the user's intent to capture an image may be greater than an image capture threshold. However, if one or more persons are detected and the one or more persons are moving in the scene, the likelihood of the user's intent to record a video may be greater than a video recording threshold. Object determination of the user's intent for a capture mode may be performed heuristically or using machine learning.

[0056] In addition, in operation 216, when the rear camera is turned on, the camera posture of the rear camera can be tracked. The camera posture can be tracked using a sensor of the electronic device 101 or using image analysis of frames captured by the rear camera. For example, the electronic device 101 may include a gyroscope, a gravity sensor, an acceleration sensor, a GPS sensor, a proximity sensor, etc. that can be used to track the camera posture. Image analysis can be used to detect whether the scene is moving when the detected object remains in the frame.

[0057] In operation 218, the electronic device 101 may perform a camera gesture determination for the user's intent. The camera gesture may be used to determine the user's intent based on the relative motion of the camera or the electronic device 101. For example, when the camera is stationary, the likelihood of the user's intent is image capture. However, if the movement of the camera exceeds a jitter threshold, it may be determined that the user's intent is a video recording mode. The jitter threshold may distinguish between shaky motion and actual motion.

[0058] In operation 220, based on the determination of the user's intent, the electronic device 101 may determine the overall user intent of the camera shooting mode. The overall determination may use one or more of the user determination, the object determination, and the camera posture determination to determine the overall intent of the camera's shooting mode. The user determination, the object determination, and the camera posture determination may each have a normalized relative likelihood score, which may be used to determine the camera shooting mode. In some embodiments, the user determination, the object determination, and the camera posture determination may each be weighted differently depending on the different levels of importance of each determination by machine learning and AI recognition. The determination of the user's intent may also include determining the settings of the identified shooting mode.

[0059] In operation 222, based on the overall determination, the electronic device 101 can execute the determined shooting mode. Once the overall determination has been completed, the determined shooting mode can be automatically executed. When the determined shooting mode is not the current shooting mode of the camera application, the determined shooting mode can also be used in addition to the current shooting mode. For example, when the video recording shooting mode is determined when in the image capture mode, the camera application can capture an image and automatically start recording a video. When in the image capture mode, the recorded video can be saved in a separate folder so that video recording and image capture are automatically performed. The automatically recorded video and captured image can also have information stored as metadata of the file, which indicates that the video or image is automatically captured. In some embodiments, the camera application can provide a prompt on the display about the shooting mode intended by the user. Image capture or video recording can also be performed using the camera settings determined based on the overall determination in operation 220.

[0060] although Figure 2 An example pipeline 200 for detecting user intent for a capture mode for a camera application is shown, but may be used for Figure 2 For example, although shown as a series of operations, Figure 2 The various operations in can overlap, occur in parallel, occur in a different order, or occur any number of times.

[0061] Figure 3An example process 300 for automatically capturing the best moment to take a group photo according to the present disclosure is shown. For ease of illustration, the process 300 is described as Figure 1 The network configuration 100 is executed on or supported by the electronic device 101, which in some embodiments may represent a cellular phone or other handheld device. Figure 3 The illustrated process 300 may be used with any other suitable device and for any other suitable system.

[0062] like Figure 3 As shown, the electronic device 101 can use a machine learning model 302 to determine when to automatically capture a photo or record a video based on facial expressions, postures, etc. Typically, when taking a photo of a group of people or a family, everyone is smiling and facing the camera with their eyes open. It may be difficult for the user to perfectly schedule the image capture time using these features, especially when the number of people in the image increases. Therefore, the machine learning model 302 can be trained to recognize these features and other features in order to automatically capture images. The machine learning model 302 can be trained using a prepared data set and updated based on the currently captured image.

[0063] The electronic device 101 may receive one or more camera viewfinder frames 304. The viewfinder frame 304 may be received from one or more optical sensors in the electronic device 101. The viewfinder frame 304 may be initialized when a camera application is opened on the electronic device 101. The viewfinder frame 304 may also be referred to as a preview image or preview image feed.

[0064] The machine learning model 302 may be used to perform a person detection and tagging operation 306. This may include, for example, the processor 120 of the electronic device 101 identifying people in the viewfinder frame using the machine learning model 302. The identification of people may be performed using any person identification method. The machine learning model 302 may be used to individually tag each of the identified people in the image.

[0065] Once identified and tagged, a person tracking operation 308 may be performed using the machine learning model 302. This may include, for example, the processor of the electronic device 101 using the machine learning model 302 to track each person simultaneously or in a determined order. In some embodiments, once an individual is identified and tagged, the individual may be tracked while other individuals are still being detected and tagged. For example, a user who is closer to the camera may be detected and identified more quickly using the machine learning model 302 than a person who is farther away in the scene. In some embodiments, the user may select or deselect tracked people on the display of the electronic device 101.

[0066] For example, for each person tracked in the personnel tracking operation 308, the processor 120 of the electronic device 101 may perform a behavior detection operation 310 using the machine learning model 302. The behavior detection operation 310 may include a face detection operation 314, a facial expression detection operation 316, a smile detection operation 318, a head posture detection operation 320, a portrait mode detection operation 322, and the like. One or more of the detection operations 314 to 322 may be performed using the machine learning model 302. The behavior detection operation 310 is performed for each user tracked in the personnel tracking operation 308. Operations 306 to 310 may be performed as part of or simultaneously with operation 212.

[0067] In a face detection operation 314, the user's face may be recognized using the machine learning model 302. The face detection operation 314 may be performed using a face detection algorithm. This may include, for example, the processor 120 of the electronic device 101 using the machine learning model 302 to identify whether facial features are visible in the viewfinder frame 304 for a specified user.

[0068] When a face is detected in the face detection operation 314, the machine learning model 302 may be used to perform a facial expression detection operation 316. When a face is not detected, the machine learning model 302 may skip the facial expression detection operation 316. The facial expression detection operation 316 may identify facial expressions based on detected eyes, cheeks, nose, etc. The facial expression detection operation 316 may be performed using any suitable face detection or extraction algorithm.

[0069] A smile detection operation 318 may be performed using the machine learning model 302 to identify the structure of the user's mouth. This may include, for example, the processor 120 of the electronic device 101 identifying the mouth based on the location on the face using the machine learning model 302. When no face is detected, the machine learning model 302 may skip the smile detection operation 318.

[0070] A head pose detection operation 320 may be performed using a machine learning model to determine the pose of the head. The head pose detection operation 320 may be performed before the face detection operation 314. The machine learning model 302 may skip the face detection operation 314 if the head pose detection operation 320 determines that the head pose is facing away from the camera.

[0071] The portrait mode detection operation 322 may be performed after performing the head pose detection operation 320. This may include, for example, the processor 120 of the electronic device 101 detecting the portrait mode based on the user's body not being within the viewfinder frame 304 or based on only the face being within the viewfinder frame 304 by using the machine learning model 302. Based on not detecting a face in the face detection operation 314, the portrait mode detection operation 322 may be skipped.

[0072] After completing the behavior detection operation 310, the processor 120 of the electronic device 101 may also use the machine learning model 302 to perform a state determination operation 312 for each person. When the behavior detection operation 310 is completed for a specific person, the state determination operation 312 may be performed for each person, or after the behavior detection operation 310 has been completed for all tracked persons, the state determination operation 312 may be performed. The state determination operation 312 may include a face toward camera determination operation 324, a smile determination operation 326, a good facial expression determination operation 328, a body posture toward camera determination operation 330, a face portrait mode determination operation 332, and an eye open determination operation 334.

[0073] A face toward camera determination operation 324 may be performed using the machine learning model 302 to determine whether the user is facing the camera. Using the machine learning model 302, information detected in the head pose detection operation 320 and the face detection operation 314 may be utilized to determine whether the user is facing the camera. When the head pose detection operation 320 does not detect a head pose facing the camera or when the face detection operation 314 does not detect a face, the face toward camera determination operation 324 may be used to determine that the user is not ready for image capture.

[0074] The machine learning model 302 may be used to perform a smile determination operation 326 to determine whether the user is smiling. The machine learning model 302 may utilize information detected in the head pose detection operation 320, the face detection operation 314, and the smile detection operation 318 to determine that the user is smiling. When it is determined in the face toward camera determination operation 324 that the user is not facing the camera, the smile determination operation 326 may be skipped. When it is determined that the user is smiling, the machine learning model 302 may determine that the user is ready for image capture. When the user is not smiling, the machine learning model 302 may be used to determine that the user is not ready, or other determinations 328 to 334 may be used to determine whether the user is ready to be photographed.

[0075] The machine learning model 302 may be used to perform a good facial expression determination operation 328 to determine whether the user has a good facial expression. The machine learning model 302 may utilize the information detected in the head pose detection operation 320, the face detection operation 314, and the facial expression detection operation 316 to determine whether the facial expression of the tracked person indicates that the person is ready for image capture. If it is determined in the face toward camera determination operation 324 that the user is not facing the camera, the good facial expression determination operation 328 may be skipped. When it is determined that the user has a good facial expression, the machine learning model 302 may be used to determine that the user is ready for image capture. When the user does not have a good facial expression, the machine learning model 302 may be used to determine that the user is not ready, or other determination operations 330 to 334 may be used to determine whether the user is ready to be photographed. For example, if the user has an angry or sad facial expression, the machine learning model 302 may be used to determine that the user is not ready for image capture.

[0076] The machine learning model 302 may be used to perform a body pose toward camera determination operation 330 to determine whether the body pose indicates that image capture is ready. The machine learning model 302 may be used to determine whether the user has a body pose facing the camera. When the user's body pose is facing away from the camera, the machine learning model 302 may determine that the user may not be ready to take a picture. In some embodiments, the body pose toward camera determination 330 and the face toward camera determination operation 324 may be used in conjunction as an initial determination before performing the detection operations 326, 328, 332, and 334. When the face toward camera determination identifies that the user is facing the camera, the face toward camera determination operation 324 may be given a greater weight than the body pose toward camera determination operation 330. When it is identified in the face toward camera determination operation 324 that the user is facing the camera, the body pose toward camera determination operation 330 may be skipped.

[0077] A facial portrait mode determination operation 332 may be performed using the machine learning model 302 to determine whether a facial portrait mode is appropriate. The machine learning model 302 may be used to utilize information detected in the head pose detection operation 320, the face detection operation 314, and the portrait mode detection operation 322. When the face toward camera determination operation 324 does not identify that the user's face is toward the camera, the facial portrait mode determination operation 332 may be skipped. When only the user's face or the user's body below the shoulders is not visible in the viewfinder frame 304, the facial portrait mode determination operation 332 may identify that the user is ready to be photographed.

[0078] The machine learning model 302 may be used to perform an eyes open determination operation 334 to determine whether the tracked person has their eyes open. The machine learning model 302 may utilize information detected in the portrait mode detection operation 322 and the face detection operation 314. The eyes open determination operation 334 may be skipped when it is determined in the face toward camera determination operation 324 that the user is facing away from the camera. The machine learning model 302 may also be used to determine that the user is not ready for image capture based on eye closure. When the person's eyes are open, the machine learning model 302 may be used to determine that the user may be ready based on other determinations.

[0079] After it has been determined that one or more tracked people are ready for image capture, the processor 120 of the electronic device 101, for example, can use the machine learning model 302 to perform, for example, a readiness determination operation 336. Ideally, when the entire group of tracked people in the viewfinder frame 304 is ready, the machine learning model 302 is used to capture the image, or the person tracking operation 308, the behavior detection operation 310, and the state determination operation 312 are continued for each person until a photo is captured or the camera application is closed. However, the machine learning model 302 can be used to capture one or more images based on a threshold of the readiness of the person. For example, when taking a photo, some people do not smile or have a happy expression. For a situation where one or two people in a large group do not smile or have a happy expression, the machine learning model 302 can determine that it is appropriate to take a photo. Multiple determinations can be performed for a group.

[0080] When it is determined that all people are ready, the electronic device 101 can perform an automatic capture operation 338. When the machine learning model 302 determines that the tracked users are ready to take pictures, the camera automatically captures the image. When a portion of the people are ready to take pictures, the tracked people who are not ready can be identified in the photo so that the user can manually determine whether the specified user is ready. For example, when the user does not typically smile or does not have a generally happy face, the photo can still be captured and the user can determine whether to keep the photo. Multiple images can be captured each time it is determined that the group is ready.

[0081] although Figure 3 An example process 300 for automatically capturing the best moment to take a group photo is shown, but may be used for Figure 3 For example, although shown as a series of operations, Figure 3 The various operations in can overlap, occur in parallel, occur in a different order, or occur any number of times.

[0082] Figure 4An example process 400 for training a machine learning model to determine user intent for a shooting mode according to the present disclosure is shown. For ease of illustration, the process 400 is described as Figure 1 The network configuration 100 is executed on or supported by the electronic device 101, which in some embodiments may represent a cellular phone or other handheld device. Figure 4 The illustrated process 400 may be used with any other suitable device and for any other suitable system.

[0083] like Figure 4 As shown, the machine learning model 302 can be trained to determine the appropriate shooting mode based on the user's intent. In operation 402, by using the machine learning model 302, the electronic device 101 can detect that the user opens the camera application on the electronic device 101. The camera application can open one or more cameras including the front camera and the rear camera. The camera application can display the front camera or the rear camera by default, or display the front camera or the rear camera based on the previous usage of the camera application. The rear camera is a camera located on the surface or side of the electronic device 101 opposite to the display. The front camera is a camera located on the same surface or side of the electronic device 101 as the display. Although the following operations are described for capturing images or videos using the rear camera, similar functions can be reversed when capturing images or videos using the front camera.

[0084] In operation 404, the machine learning model 302 can be used to identify the current shooting mode of the camera. When the shooting mode changes, the machine learning model 302 can track the shooting mode as a strong indicator that the user actually intends to use the specified shooting mode. The shooting mode can include an image capture mode and a video recording mode.

[0085] In operation 406, the machine learning model 302 may also be used to access information of the front camera or control the front camera to capture an image of a user operating a camera application on the electronic device. After image capture or video recording is completed, the frames from the front camera may be saved or removed.

[0086] In operation 408, a user of the camera application on the electronic device 101 may manually take pictures, change settings, etc. The capture of pictures and the recording of videos may be tracked using the machine learning model 302. The machine learning model 302 may also track the settings of the camera application.

[0087] In operation 410, the data captured in operations 404 to 408 may be used as input to the machine learning model 302. Additional sensor data from the electronic device 101 may be used, including position data, proximity data, posture data, motion data, etc. The data input to the machine learning model 302 may be used to train the machine learning model 302 to recognize the user's intent for the capture mode.

[0088] In operation 412, the user's final action may be captured for user intent determination and machine learning model 302 training. For example, a user deleting a photo and then recording a video may indicate that the input information is indicating that the user intends to enter a video recording mode because the photo was deleted.

[0089] In operation 414, the training data may be stored and used to train the machine learning model 302. The training data may be captured by a large number of devices. Each device may store it specifically, or jointly with another device. For example, a group of devices may be used as initial training data, and a specific user or user group may be used as specific training data associated with the specific user or user group.

[0090] In operation 416, the machine learning model 302 uses the training data to determine the optimal camera settings for the user's intent and the camera application. The machine learning model 302 may use the training data to determine Figure 2 The data of operations 206, 208, 212, and 216 shown and the determinations in operations 210, 214, 218, and 220. The machine learning model 302 can use the input information to develop features to Figure 3 The detection is shown in behavior detection operation 310 and analyzed in state determination operation 312. The machine learning model 302 can apply weights to each of the detection operations 314-322 and determinations 324-334 to determine whether each person in the image is ready for photo capture.

[0091] In operation 418, the machine learning model 302 may be used to identify an expected shooting mode for the current use of the camera application. The machine learning model 302 may also be used to perform or support the performance of operations 202 to 220 when the camera application is opened on the electronic device 101. Based on the training from operation 416, the machine learning model 302 may be used to determine whether to operate the camera application in a video recording mode or an image capture mode.

[0092] In operation 420, when the machine learning model 302 provides a determination that the user intends to enter a video recording mode, the camera application is controlled to begin recording a video. Video can be recorded regardless of the current mode of the camera application. For example, the camera application can be opened in an image capture mode and a determination can be made to switch to the capture mode for display, or to automatically record a video regardless of being in an image capture mode. In some embodiments, a notification can be provided to indicate that a video is being automatically recorded.

[0093] In operation 422, when the machine learning model 302 provides a determination that the user intends to enter an image capture mode, the camera application is controlled to capture an image. The image can be captured regardless of the current shooting mode of the camera application. For example, when the user controls the camera in a video recording mode, the camera application can capture one or more images when it is determined that a person identified in the image is ready or when the user enters a record button. The machine learning model 302 can be used to determine that the user is ready to take a picture based on the readiness determination operation 336.

[0094] In operation 424, the recorded video from operation 420 and the captured image from operation 422 may be stored using an "auto" tag. If stored with user captured video and images, the "auto" tag may allow the user to review the automatically recorded video and captured images more quickly. When in image capture mode, automatically recorded video may be stored on the electronic device 101 along with user captured images. When in video recording mode, automatically captured images may be saved in addition to user captured video.

[0095] although Figure 4 An example process 400 for training a machine learning model to determine user intent for a capture mode is shown, but may be used for Figure 4 For example, although shown as a series of operations, Figure 4 The various operations in can overlap, occur in parallel, occur in a different order, or occur any number of times.

[0096] Figure 5 An example user interface 500 for detecting a user's intent to select a capture mode for a camera application according to the present disclosure is shown. For ease of illustration, the user interface is described as being in Figure 1 The network configuration 100 is provided on the electronic device 101, for example, on the display 160 of the electronic device 101. However, Figure 5 The illustrated user interface 500 may be used with any other suitable device and for any other suitable system.

[0097] like Figure 5As shown, user interface 500 can be opened in an image capture mode of a camera application. A viewfinder or preview image 502 can be displayed on user interface 500. Machine learning model 302 can be used to determine that the user intends to capture an image. The camera application can automatically capture an image 504 corresponding to preview image 502 displayed on user interface 500 in response to the determination that a photo is intended to be captured. Objects in the scene can be determined based on the content currently displayed in the current preview image 502. As shown in preview image 502, the user is zoomed in on objects (e.g., people and cars) in the scene. Machine learning model 302 can be used to determine that the zoomed preview image 502 and other current settings are suitable for automatically captured image 504. However, machine learning model 302 may determine that the user intends to capture the scene at a lower zoom value, and accordingly capture the image with settings for capturing image 504 that are different from the settings currently used for preview image 502.

[0098] although Figure 5 An example user interface 500 is shown for detecting user intent for a capture mode for a camera application, but may be used for Figure 5 Various changes may be made. For example, user interface 500 may be implemented on different electronic devices (such as headphones, cameras, etc.). In addition, the arrangement of automatically captured images 504 may be displayed on different parts of user interface 500 and may not be displayed until the image is actually captured.

[0099] Figure 6 An example method 600 for detecting a user's intention to start video recording using an electronic device according to the present disclosure is shown. For ease of illustration, Figure 6 The method 600 is described as Figure 1 is executed on the electronic device 101 or supported by the electronic device 101. However, Figure 6 The illustrated method 600 may be used with any other suitable device and for any other suitable system.

[0100] like Figure 6As shown, at step 602, the electronic device 101 collects sensor information. The sensor information may be collected in response to a shooting mode of a camera application of the electronic device 101 (such as a cellular phone or a camera). The sensor information may include motion data of the electronic device, position data of the electronic device, and image data captured by one or more image sensors. The shooting mode represents at least one of a video recording mode and an image capture mode. The image data may be captured using a front imaging sensor or a front camera and a rear image sensor or a rear camera. The features extracted from the sensor information may include at least one of the following items: facial expressions, head movements, hand postures, body postures, object identification, positions, movements, etc. of one or more key objects in the image data. The features extracted from the sensor information may also include the posture of the electronic device and the movement of the electronic device. The posture of the electronic device is based on the position data of the electronic device, and the movement of the electronic device is based on the movement data of the electronic device.

[0101] The features extracted from the sensor information may also include at least one of the following items: the speed of one or more key objects in the image data, the acceleration of one or more key objects in the image data, the speed of the electronic device, and the acceleration of the electronic device. In the example, in the following step 604, the electronic device 101 may determine the user's intention based on whether the one or more key objects are moving at a certain speed or higher or whether the speed or acceleration of the one or more key objects is zero. In the example, in the following step 604, the electronic device 101 may determine the user's intention based on whether the electronic device 101 is moving at a certain speed or higher or whether the speed or acceleration of the electronic device 101 is zero.

[0102] At step 604, the electronic device 101 determines the shooting mode intended by the user. The electronic device 101 can determine whether the user intends to record video or capture images based on features extracted from the sensor information. A trained machine learning model (such as the trained machine learning model 302) can be used to determine the user's intention. The determination of the shooting mode can be an overall determination that takes into account user determination, object determination, and camera posture determination. User determination can determine whether the user operating the camera of the electronic device 101 intends to record video or capture images. Object determination can determine whether objects in the scene are ready to be recorded video or captured in an image. Camera posture determination can determine whether to capture images or record video based on the movement and orientation of the camera on the electronic device 101.

[0103] At step 606, the electronic device 101 captures an image or video in the capture mode intended by the user. In response to determining that the user's intent is to record a video, the video can be recorded regardless of the capture mode. In response to determining that the user's intent is to capture an image, the image can be captured regardless of the capture mode. When the machine learning model 302 recognizes when the person in the preview image is ready for photo or video recording, the image or video can be automatically captured.

[0104] although Figure 6 An example of a method 600 for detecting a user's intention to start video recording using an electronic device is shown, but may be used for Figure 6 For example, although shown as a series of steps, Figure 6 The various steps in may overlap, occur in parallel, occur in a different order, or occur any number of times.

[0105] Although the present disclosure has been described by way of exemplary embodiments, various changes and modifications may be suggested to one skilled in the art. The present disclosure is intended to encompass such changes and modifications as fall within the scope of the appended claims.

Claims

1. A method (600) performed by an electronic device (101), the method (600) comprising: In response to initiating a capture mode of a camera application on the electronic device, collecting (602) sensor information, the sensor information comprising at least one of the following: motion data of the electronic device, position data of the electronic device, and image data captured by one or more imaging sensors of the electronic device, wherein the capture mode represents at least one of the following: a video recording mode and an image capturing mode; determining (604) whether the user intends to record a video or capture an image based on features extracted from the sensor information by using the trained machine learning model; and In response to determining that the user intent is to record a video, regardless of the capture mode, a video is recorded (606), or in response to determining that the user intent is to capture an image, regardless of the capture mode, an image is captured (606).

2. The method according to claim 1, wherein: The image data includes image data captured using a front-facing imaging sensor of the electronic device; and The features extracted from the sensor information include at least one of the following: facial expressions, head movements, hand gestures, and body gestures of the user.

3. The method according to any one of claims 1 to 2, wherein: The image data includes image data captured using a rear-mounted imaging sensor of the electronic device; and Features extracted from the sensor information include at least one of: object identification, location, and motion of one or more key objects in the image data.

4. The method according to any one of claims 1 to 3, wherein: The features extracted from the sensor information include a posture of the electronic device based on position data of the electronic device and a motion of the electronic device based on motion data of the electronic device.

5. The method according to any one of claims 1 to 4, further comprising: detecting one or more persons within a field of view of the one or more imaging sensors; For each detected person, determining whether the person is ready for image capture; as well as In response to determining that each detected person is ready for image capture, regardless of the shooting mode, an image is captured.

6. The method according to claim 5, wherein: Determining whether the person is ready for image capture is based on at least one of: a facial expression, head movement, hand posture, and body posture of the person.

7. The method according to any one of claims 5 to 6, wherein: The person is determined to be ready for image capture based on at least one of the following: the person's face is facing the one or more imaging sensors, the person's body is facing the one or more imaging sensors, the person's face makes a desired facial expression, and the person's eyes are open.

8. An electronic device (101), comprising: one or more imaging sensors (108); as well as At least one processing device (120) operably coupled to the one or more imaging sensors and configured to: In response to initiating a capture mode of a camera application on the electronic device, collecting (602) sensor information, the sensor information comprising at least one of the following: motion data of the electronic device, position data of the electronic device, and image data captured by the one or more imaging sensors of the electronic device, wherein the capture mode represents at least one of the following: a video recording mode and an image capturing mode; determining ( 604 ) whether the user intends to record a video or capture an image based on features extracted from the sensor information by using the trained machine learning model; as well as In response to determining that the user intent is to record a video, regardless of the capture mode, a video is recorded (606), or in response to determining that the user intent is to capture an image, regardless of the capture mode, an image is captured (606).

9. The electronic device according to claim 8, wherein: The one or more imaging sensors include a front-facing imaging sensor; The image data includes image data captured using the front-facing imaging sensor; as well as The features extracted from the sensor information include at least one of the following: facial expressions, head movements, hand gestures, and body gestures of the user.

10. The electronic device according to any one of claims 8 to 9, wherein: The one or more imaging sensors include a rear imaging sensor; The image data includes image data captured using the rear imaging sensor; and Features extracted from the sensor information include at least one of: object identification, location, and motion of one or more key objects in the image data.

11. The electronic device according to any one of claims 8 to 10, wherein: The features extracted from the sensor information include a posture of the electronic device based on position data of the electronic device and a motion of the electronic device based on motion data of the electronic device.

12. The electronic device according to any one of claims 8 to 11, wherein: The at least one processing device is configured to: detecting one or more persons within a field of view of the one or more imaging sensors; For each detected person, determining whether the person is ready for image capture; as well as In response to determining that each detected person is ready for image capture, regardless of the shooting mode, an image is captured.

13. The electronic device according to claim 12, wherein: Determining whether the person is ready for image capture is based on at least one of: a facial expression, head movement, hand posture, and body posture of the person.

14. The electronic device according to any one of claims 12 to 13, wherein: The person is determined to be ready for image capture based on at least one of the following: the person's face is facing the one or more imaging sensors, the person's body is facing the one or more imaging sensors, the person's face makes a desired facial expression, and the person's eyes are open.

15. A computer-readable medium comprising instructions which, when executed, cause at least one processor of an electronic device to perform operations corresponding to the method according to any one of claims 1 to 7.

Images