Method for camera control, apparatus, device, storage medium, and program product

Abstract

Embodiments of the present disclosure provide a method for camera control, an apparatus, a device, a storage medium, and a program product. The method comprises: on the basis of motion information collected by a terminal device at a plurality of time points, determining accelerations of the terminal device at the plurality of time points; on the basis of the accelerations of the terminal device at the plurality of time points, determining a target speed of the terminal device at a target time point; on the basis of the target speed at the target time point, determining a predicted value of the terminal device for a camera turn-on requirement at the target time point; and on the basis of the predicted value, determining a turn-on control instruction for a camera of the terminal device, the turn-on control instruction being used for instructing to control whether to turn on the camera.

Description

Methods, apparatus, devices, storage media, and program products for camera control

[0001] This application claims priority to Chinese Patent Application No. 202411397442.4, filed on October 8, 2024, entitled "Method, Apparatus, Device, Storage Medium and Program Product for Camera Control", the entire contents of which are incorporated herein by reference. Technical Field

[0002] The exemplary embodiments disclosed herein generally relate to the field of computers, and particularly to methods, apparatus, devices, computer-readable storage media, and computer program products for camera control. Background Technology

[0003] With the development of artificial intelligence (AI) technology and its integration with smart hardware, AI has gradually become integrated into people's daily lives. From smart home devices to mobile applications, AI technology has significantly improved the intelligence level of devices. Traditional human-computer interaction methods are therefore constantly innovating and iterating. In this context, there is a desire to automatically recognize users' camera activation needs, thereby improving interaction efficiency. Summary of the Invention

[0004] In a first aspect of this disclosure, a method for camera control is provided. The method includes: determining the acceleration of the terminal device at multiple time points based on motion information collected by the terminal device at multiple time points; determining a target velocity of the terminal device at a target time point based on the acceleration of the terminal device at the multiple time points; determining a predicted value of the camera activation requirement of the terminal device at the target time point based on the target velocity at the target time point; and determining an activation control command for the camera of the terminal device based on the predicted value, the activation control command indicating whether the camera should be activated or deactivated.

[0005] In a second aspect of this disclosure, a method for camera control is provided. The method includes: determining the acceleration of the terminal device at multiple time points based on motion information collected by the terminal device at multiple time points; inputting the acceleration at the multiple time points into a trained machine learning model to obtain a prediction result, the prediction result indicating a prediction of camera activation demand, wherein the training samples of the machine learning model during training include positive samples and negative samples, positive samples including motion information collected by the sample device when there is a camera activation demand and a first label indicating the camera activation demand, and negative samples including motion information collected by the sample device when there is no camera activation demand and a second label indicating no camera activation demand; and determining an activation control command for the camera of the terminal device based on the prediction result, the activation control command indicating whether the camera is activated or deactivated.

[0006] In a third aspect of this disclosure, an apparatus for camera control is provided. The apparatus includes: an acceleration determination module configured to determine the acceleration of the terminal device at multiple time points based on motion information collected by the terminal device at multiple time points; a target velocity determination module configured to determine a target velocity of the terminal device at a target time point based on the acceleration of the terminal device at the multiple time points; a prediction value determination module configured to determine a predicted value of the terminal device at the target time point for camera activation requirements based on the target velocity at the target time point; and an activation control module configured to determine an activation control command for the camera of the terminal device based on the predicted value, the activation control command indicating whether the camera is activated or deactivated.

[0007] In a fourth aspect of this disclosure, an apparatus for camera control is provided. The apparatus includes: an acceleration determination module configured to determine the acceleration of a terminal device at multiple time points based on motion information collected by the terminal device at multiple time points; a prediction module configured to input the acceleration at the multiple time points into a trained machine learning model to obtain a prediction result indicating a prediction of camera activation demand, wherein the training samples of the machine learning model during training include positive samples and negative samples, positive samples including motion information collected by the sample device when there is a camera activation demand and a first label indicating the camera activation demand, and negative samples including motion information collected by the sample device when there is no camera activation demand and a second label indicating no camera activation demand; and an activation control module configured to determine an activation control command for the camera of the terminal device based on the prediction result, the activation control command indicating whether the camera is activated or deactivated.

[0008] In a fifth aspect of this disclosure, an electronic device is provided. The device includes at least one processor; and at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor. When executed by the at least one processor, the instructions cause the device to perform the method of the first aspect.

[0009] In a sixth aspect of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores computer-executable instructions that can be executed by a processor to implement the method of the first aspect.

[0010] In a seventh aspect of this disclosure, a computer program product is provided. The computer program product is tangibly stored in a computer storage medium and includes computer-executable instructions that, when executed by a device, cause the device to perform the method of the first aspect.

[0011] It should be understood that the description in this section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0012] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0013] Figure 1 shows a schematic diagram of an example environment in which embodiments of the present disclosure can be implemented;

[0014] Figure 2 shows a flowchart of a camera control process according to some embodiments of the present disclosure;

[0015] Figure 3 shows a schematic diagram illustrating the acquisition of motion information of a terminal device at multiple time points according to some embodiments of the present disclosure;

[0016] Figure 4 illustrates a flowchart of a user camera control process according to some embodiments of the present disclosure;

[0017] Figure 5 shows a schematic diagram of a gravity filtering example according to some embodiments of the present disclosure;

[0018] Figure 6 shows a schematic diagram of an example of a high-pass filtering operation according to some embodiments of the present disclosure;

[0019] Figure 7A shows a schematic diagram of an example waveform of the linear acceleration component before high-pass filtering according to some embodiments of the present disclosure;

[0020] Figure 7B shows a schematic diagram of an example waveform of acceleration after high-pass filtering according to some embodiments of the present disclosure;

[0021] Figure 8A shows a schematic diagram of an example of a velocity waveform before performing leakage filtering using a leaker that is negatively correlated with velocity and acceleration, according to some embodiments of the present disclosure;

[0022] Figure 8B shows a schematic diagram of an example of a velocity waveform after performing leakage filtering using a leaker that is negatively correlated with velocity and acceleration, according to some embodiments of the present disclosure;

[0023] Figure 9A shows a schematic diagram of an example of a velocity waveform before performing leakage filtering using a leaker that is positively correlated with velocity accumulation, according to some embodiments of the present disclosure;

[0024] Figure 9B shows a schematic diagram of an example of a velocity waveform after performing leakage filtering using a leaker that is positively correlated with velocity accumulation, according to some embodiments of the present disclosure;

[0025] Figure 10A illustrates a schematic diagram of an example application scenario for camera control according to some embodiments of the present disclosure;

[0026] Figure 10B illustrates a schematic diagram of an example application scenario for camera control according to some embodiments of the present disclosure;

[0027] Figure 11 illustrates a schematic diagram of an example application scenario for camera control according to some embodiments of the present disclosure;

[0028] Figure 12 shows a schematic diagram of a camera control process according to some embodiments of the present disclosure;

[0029] Figure 13 shows a flowchart of a camera control process according to some embodiments of the present disclosure;

[0030] Figure 14 illustrates a schematic diagram of an example architecture of a machine learning model according to some embodiments of the present disclosure;

[0031] Figure 15 shows a flowchart of an application process for camera control according to some embodiments of the present disclosure;

[0032] Figure 16 shows a block diagram of a device for camera control according to some embodiments of the present disclosure;

[0033] Figure 17 shows a block diagram of a camera control apparatus according to some embodiments of the present disclosure; and

[0034] Figure 18 shows a block diagram of an electronic device capable of implementing one or more embodiments of the present disclosure. Detailed Implementation

[0035] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0036] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0037] In this document, unless explicitly stated otherwise, performing a step in response to A does not mean that the step is performed immediately after A, but may include one or more intermediate steps.

[0038] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition, use, storage or deletion of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0039] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, relevant users should be informed of the type, scope of use, and usage scenarios of the information involved in this disclosure through appropriate means in accordance with relevant laws and regulations, and authorization from relevant users should be obtained. Among them, relevant users may include any type of rights holder, such as individuals, enterprises, and groups.

[0040] For example, in response to receiving an active request from a user, a prompt message is sent to the relevant user to clearly inform the user that the requested operation will require obtaining and using the user's information, thereby enabling the relevant user to choose whether to provide information to the software or hardware such as the electronic device, application, server, or storage medium that performs the operation of the technical solution disclosed herein based on the prompt message.

[0041] As an optional but non-restrictive implementation, in response to a user's active request, a prompt message can be sent to the user, such as a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide information to the electronic device.

[0042] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0043] As used in this paper, the term "model" refers to a model that learns the relationship between inputs and outputs from training data, enabling it to generate corresponding outputs for a given input after training. Model generation can be based on machine learning techniques. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs using multiple layers of processing units. A neural network model is an example of a deep learning-based model. In this paper, "model" may also be referred to as a "machine learning model," "learning model," "machine learning network," or "learning network," and these terms are used interchangeably.

[0044] A neural network is a machine learning network based on deep learning. A neural network processes input and provides a corresponding output, typically consisting of an input layer, an output layer, and one or more hidden layers between the input and output layers. Neural networks used in deep learning applications often include many hidden layers, thus increasing the network's depth. The layers of a neural network are connected sequentially, so that the output of the previous layer is provided as the input to the next layer. The input layer receives the input to the neural network, while the output layer's output serves as the final output. Each layer of a neural network includes one or more nodes (also called processing nodes or neurons), each node processing the input from the layer above.

[0045] Machine learning typically comprises three phases: training, testing, and application (also known as inference). In the training phase, a given model is trained using a large amount of training data, iteratively updating its parameter values ​​until the model can consistently generate inferences that meet the expected goals from the training data. Through training, the model can be considered to have learned the relationship between inputs and outputs (also known as the input-output mapping) from the training data. The parameter values ​​of the trained model are determined. In the testing phase, test inputs are applied to the trained model to test whether it can provide the correct output, thus determining the model's performance. In the application phase, the model can be used to process actual inputs based on the trained parameter values ​​to determine the corresponding output.

[0046] Figure 1 illustrates a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. In this example environment 100, an application 120 is installed on a terminal device 110. A user 140 can interact with the application 120 via the terminal device 110 and / or an attached device of the terminal device 110.

[0047] In embodiments of this disclosure, application 120 may have intelligent dialogue and task processing capabilities. Typically, application 120 can support user 140 in inputting questions in natural language and perform tasks and provide responses based on its understanding of the natural language input and logical reasoning ability. For example, application 120 may support text dialogue services, voice dialogue services, and content dialogues in other modalities with user 140.

[0048] In such an embodiment, if application 120 is active, terminal device 110 may present the user interface 150 of application 120. User interface 150 may include, for example, a dialog interface of application 120 (where current and historical conversations, including text conversation content, may be presented), and so on. In some embodiments, terminal device 110 may present text 152 and play voice in user interface 150. Voice may, for example, include voice from user 140 or voice responses to voice.

[0049] In some embodiments, some hardware and software devices of the terminal device 110 may cooperate with such an application 120 to perform tasks, so that the application 120 can provide a response to the user 140 based on the operation of such hardware and software devices.

[0050] In embodiments of this disclosure, application 120 may also be other applications capable of providing interactive capabilities. For example, application 120 may provide interactive capabilities other than intelligent dialogue. In such embodiments, some hardware and software devices of terminal device 110 may perform tasks based on the terminal device 110's perception of the user 140's behavior and / or the user 140's interactive operations on terminal device 110, without relying on application 120.

[0051] In some embodiments, terminal device 110 or its application 120 may utilize machine learning model 160 (which may include one or more machine learning models, such as machine learning model 160-1, machine learning model 160-2, ..., machine learning model 160-N, etc., where N is a positive integer. For ease of description, the one or more machine learning models are collectively referred to as machine learning model 160 herein) to support interaction with user 140. For example, terminal device 110 or its application 120 may utilize one or more machine learning models 160 to enable some hardware and software devices of terminal device 110 to start working in order to perform tasks.

[0052] In some embodiments, server device 130 communicates with terminal device 110 to provide services to application 120 and support the operation of some hardware and software devices of terminal device 110. For example, server device 130 may invoke machine learning model 160 to support the startup of some hardware and software devices of terminal device 110 based on the output of machine learning model 160.

[0053] Terminal device 110 can be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio broadcast receivers, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. In some embodiments, terminal device 110 may also support any type of user-facing interface (such as "wearable" circuitry).

[0054] Server-side device 130 can be various types of computing systems / servers capable of providing computing power, including but not limited to mainframes, edge computing nodes, computing devices in cloud environments, and so on. Server-side device 130 can, for example, be implemented based on a cloud environment.

[0055] It should be understood that the structure and function of the various elements in environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure.

[0056] As mentioned earlier, with the continuous innovation and iteration of human-computer interaction methods, there is a desire for technologies that can recognize users' camera activation needs. Specifically, the goal is to accurately and automatically determine whether to activate the terminal device's camera, thereby reducing the time and convenience for users to activate the camera and improving interaction efficiency.

[0057] In view of this, embodiments of this disclosure provide an improved scheme for camera control. In this scheme, based on motion information collected by the terminal device at multiple time points, the acceleration of the terminal device at those multiple time points is determined. Based on the acceleration of the terminal device at the multiple time points, a target velocity of the terminal device at a target time point is determined. Based on the target velocity at the target time point, a predicted value for the camera activation requirement of the terminal device at the target time point is determined. Based on the predicted value, an activation control command for the camera of the terminal device is determined, which instructs the camera to be turned on or off.

[0058] In this way, by processing the motion information of the terminal device, data errors can be reduced and data accuracy improved, thereby accurately and efficiently determining the user's intention to use the camera and controlling its activation. This avoids triggering erroneous interactive operations, effectively improving interaction efficiency and user experience.

[0059] It should be understood that the structure and function of the various elements in environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure.

[0060] The following description will continue with reference to the accompanying drawings, which will provide some exemplary embodiments of this disclosure.

[0061] Figure 2 illustrates a flowchart of a camera control process 200 according to some embodiments of the present disclosure. For ease of discussion, these embodiments will be described with reference to the environment 100 of Figure 1. These embodiments can be implemented in the server device 130 of Figure 1.

[0062] In block 210, based on the motion information collected by the terminal device 110 at multiple time points, the server device 130 determines the acceleration of the terminal device 110 at multiple time points. In some embodiments, the motion information collected by the terminal device 110 may be raw IMU (Inertial Measurement Unit) data.

[0063] An inertial measurement unit (IMU) is a device that measures an object's three-axis attitude angles (or angular rates) and acceleration. Typically, an IMU contains three single-axis accelerometers and three single-axis gyroscopes. The accelerometers detect the object's acceleration signals along the three independent axes of the carrier's coordinate system, while the gyroscopes detect the carrier's angular velocity signals relative to the navigation coordinate system. By measuring the object's angular velocity and acceleration in three-dimensional space, the object's attitude can be calculated. Therefore, the raw motion information from an IMU can include data related to acceleration and data related to angular velocity.

[0064] Figure 3 illustrates a schematic diagram of a legend 300 for collecting motion information of a terminal device 110 at multiple time points according to some embodiments of the present disclosure. Legend 300 can be implemented in the environment 100 of Figure 1. Some steps of process 200 can be implemented based on legend 300. Legend 300 may include a legend 310 for representing the motion state of the terminal device 110 at multiple time points, and a legend 320 for the motion trajectory data of the terminal device 110 collected in that motion state. For ease of illustration, a mobile phone can be used as an example of the terminal device 110 in legend 310. In some examples, the motion of the terminal device 110 can be measured using an IMU.

[0065] In Figure 310, it is assumed that the terminal device 110 moves in the environmental space in the direction indicated by arrow 311. Accordingly, in Figure 320, the IMU unit can be used to capture the motion trajectory 321 of the terminal device 110 in the world coordinate system (i.e., the coordinate system of the environmental space of this scene), thereby collecting motion information.

[0066] It should be understood that in other embodiments, other motion measurement units besides the IMU unit may be used to measure the motion of the terminal device 110, thereby obtaining motion information.

[0067] It should be noted that since each frame of raw data from the IMU unit may contain errors, and these errors may accumulate, the data may become inaccurate in actual use, leading to system misjudgment and triggering incorrect interactive effects. Therefore, motion information can be preprocessed.

[0068] In some embodiments, when determining the acceleration of the terminal device 110 at multiple time points, the server device 130 may perform preprocessing on the collected motion information and determine the acceleration of the terminal device 110 at multiple time points based on the preprocessed motion information. The preprocessed motion information may contain higher quality data, thereby facilitating the extraction of the acceleration of the terminal device 110 at multiple time points.

[0069] In some embodiments, preprocessing may include at least one of gravity filtering and high-pass filtering. In embodiments of this disclosure, to improve data quality, preprocessing of motion information may include both gravity filtering and high-pass filtering. It should be understood that in other embodiments, preprocessing of motion information may also include any other suitable operations.

[0070] Figure 4 illustrates a flowchart of a user camera control process 400 according to some embodiments of the present disclosure. Process 400 can be implemented in the environment 100 of Figure 1. Process 400 can be a specific embodiment of process 200. The raw IMU motion information 410 in process 400 can be motion information acquired based on Figure 300.

[0071] In process 400, preprocessing can be performed on the raw IMU motion information 410. Preprocessing may include a gravity filtering operation (415). In some examples, a core motion framework containing a gravity filtering algorithm can be used to perform the preprocessing on the raw IMU motion information 410.

[0072] Figure 5 shows a schematic diagram of a gravity filtering illustration 500 according to some embodiments of the present disclosure. Illustration 500 may be an illustration of the gravity filtering operation (415) in process 400. The terminal device 110 shown in illustration 500 may be the mobile phone in illustration 310. The movement path 510 of the terminal device 110 shown in illustration 500 may be the acceleration-related component in the motion trajectory 321 of illustration 320.

[0073] Specifically, the IMU unit acquires motion information by measuring the acceleration of the terminal device 110 along various axes (typically the X, Y, and Z axes). However, the IMU not only senses actual motion acceleration but is also affected by Earth's gravity. To accurately measure the actual motion state of the terminal device 110, it is necessary to filter out the acceleration component caused by gravity. It should be understood that any appropriate type of gravity filtering operation can be used to filter the gravitational acceleration component in the motion information based on the terminal device 110.

[0074] Referring back to Figure 4, after performing the gravity filtering operation (415), the acceleration 420 of the terminal device 110 is obtained, for example, linear acceleration. Since the IMU unit of the terminal device 110 can only measure data related to displacement acceleration and cannot directly measure displacement velocity, the velocity is obtained by acceleration accumulation. When using acceleration 420 to obtain the velocity of the terminal device 110, the error regarding acceleration will continue to accumulate with the integration process, causing the velocity error to become larger and larger. Therefore, it is necessary to reduce this error to obtain a more accurate velocity. Thus, the acceleration 420 is subjected to the step of acceleration accumulation error correction (425). In step 425, a combination of high-pass filtering and leakage integration is used.

[0075] Through actual testing and observation, it can be found that a portion of the acceleration error is an overall offset. This overall offset signal can be considered a low-frequency signal. Therefore, filtering out the low-frequency acceleration signal can reduce the cumulative velocity error. In this embodiment, a high-pass filter is used to filter out the overall offset signal.

[0076] Figure 6 illustrates a schematic diagram of an example 600 for high-pass filtering operation according to some embodiments of the present disclosure. In example 600, a Butterworth high-pass filter 610 can be used to perform a high-pass filtering operation on acceleration 420. More specifically, a second-order Butterworth high-pass filter can be used.

[0077] As shown in Figure 6, the acceleration 420 includes the true acceleration, overall offset, and other high-frequency noise. When a Butterworth high-pass filter 610 is used, the overall offset can be filtered out, thus obtaining the acceleration 620 of the terminal device 110 at multiple time points. The acceleration 620 may include the true acceleration and other high-frequency noise.

[0078] Figure 7A shows a schematic diagram of waveform example 700A of the linear acceleration component before high-pass filtering according to some embodiments of the present disclosure. Figure 7B shows a schematic diagram of waveform example 700B of the acceleration after high-pass filtering according to some embodiments of the present disclosure. In waveform examples 700A-700B, the horizontal axis may represent time, and the vertical axis may represent the acceleration value.

[0079] In waveform example 700A, the waveform of acceleration 420 without high-pass filtering can be waveform 710. In waveform example 700B, after high-pass filtering of acceleration 420, the resulting waveform can be waveform 720. By comparing waveforms 710 and 720 in waveform example 700B, it can be seen that waveform 720 has a lower overall amplitude compared to waveform 710. Therefore, high-pass filtering can remove the overall offset signal in acceleration 420.

[0080] Referring back to Figure 2, in box 220, based on the acceleration of terminal device 110 at multiple time points, server device 130 determines the target velocity of terminal device 110 at a target time point. In this paper, the target time point is the last of the multiple time points. That is, based on one or more historical time points and the acceleration detected at the current time point, the velocity at the current time point is determined, which can then be used to subsequently determine whether the camera of terminal device 110 should be activated at the current time point.

[0081] Since speed errors accumulate, assuming that terminal device 110 does not continuously move at high speed, and considering the movement of terminal device 110 relative to user 140, but not the overall movement of user 140 and terminal device 110 (such as the overall movement of user 140 carrying terminal device 110 on a vehicle), it can be considered that if a continuous average speed is not zero, then a cumulative speed error has occurred. In some embodiments, as detailed below, speed errors are reduced through a leakage mechanism.

[0082] Referring to Figure 4, in step 425, velocity error correction can be achieved using a leaker that is negatively correlated with velocity and acceleration, and a leaker that is positively correlated with velocity accumulation. Such velocity error correction can be implemented based on leak filtering. The following embodiments will discuss in detail the steps implemented using these two leakers.

[0083] In some embodiments, when determining the target speed of the terminal device 110 at a target time point, the server device 130 can determine the predicted speed of the terminal device 110 at the target time point based on the acceleration of the terminal device 110 at multiple time points. Based on the acceleration and predicted speed of the terminal device 110 at the target time point, the server device 130 can determine a first adjustment rate for the target time point, the first adjustment rate being negatively correlated with both the acceleration and predicted speed at the target time point. Then, the server device 130 can adjust the predicted speed of the terminal device 110 at the target time point based on the first adjustment rate to obtain the target speed.

[0084] Such an embodiment can employ a leaker that is negatively correlated with both velocity and acceleration to achieve error correction. First, based on the conventional relationship between acceleration and velocity, the predicted velocity at the target time point can be determined according to the acceleration of the terminal device 110 at multiple time points. Then, in order to correct the error of the predicted velocity, a first adjustment rate that is negatively correlated with both the acceleration and the predicted velocity at the target time point can be determined, and the predicted velocity can be adjusted using the first adjustment rate to obtain the target velocity.

[0085] In some embodiments, the first adjustment rate can be a first speed reduction rate. This is because when calculating the speed based on IMU data, speed errors accumulate continuously. These accumulated speed errors may be positive, so a leakage mechanism is needed to calculate the speed reduction rate to mitigate the speed errors and obtain a more accurate speed value. In the leakage mechanism, the first speed reduction rate can also be called the first leaked speed.

[0086] Specifically, when acceleration or velocity is high, the terminal device 110 may be performing maneuvering actions (such as rapid movement or violent motion). In this case, the IMU unit error will be lower than the actual signal. This means that the sensor data is more reliable, and therefore the first adjustment rate is small (i.e., the first leakage rate is slow) to avoid affecting the accuracy of motion data.

[0087] Additionally, when both speed and acceleration are low, the terminal device 110 may approach a stationary state, and the sensor's speed value may contain steady-state errors. In this case, the first adjustment rate can be increased (i.e., the first leakage speed can be accelerated) to bring the speed to zero, thereby eliminating the accumulated error in the near-stationary state.

[0088] Figure 8A shows a schematic diagram of a waveform example 800A of velocity before leakage filtering is performed using a leaker that is negatively correlated with velocity and acceleration, according to some embodiments of the present disclosure. Figure 8B shows a schematic diagram of a waveform example 800B of velocity after leakage filtering is performed using a leaker that is negatively correlated with velocity and acceleration, according to some embodiments of the present disclosure. In waveform examples 800A-800B, the horizontal axis may represent time, and the vertical axis may represent velocity values.

[0089] In waveform example 800A, the velocity waveform without leakage filtering can be waveform 810. In waveform example 800B, after performing leakage filtering using a leaker that is negatively correlated with velocity and acceleration, the resulting velocity waveform can be waveform 820. By comparing waveforms 810 and 820 in waveform example 800B, it can be seen that waveform 820 is generally lower in amplitude than waveform 810. Furthermore, after time point 2, the velocity value of waveform 820 increasingly approaches 0. Therefore, performing leakage filtering using a leaker that is negatively correlated with velocity and acceleration can reduce velocity errors.

[0090] In some embodiments, when determining the target speed of the terminal device 110 at a target time point, the server device 130 can determine the predicted speed of the terminal device 110 at multiple time points based on the acceleration of the terminal device 110 at multiple time points, determine the duration for which the predicted speed is greater than zero, and determine a second adjustment rate for the target time point based on the duration, the second adjustment rate being positively correlated with the magnitude of the duration. Then, the server device 130 can adjust the predicted speed of the terminal device 110 at the target time point based on the second adjustment rate to obtain the target speed.

[0091] Such an embodiment could employ a leaker that is positively correlated with the cumulative velocity to achieve error correction. In some embodiments, the second adjustment rate could be a second velocity reduction rate for reasons similar to the first adjustment rate. In the leak mechanism, the second velocity reduction rate could also be referred to as the second leak velocity.

[0092] Specifically, when the speed remains greater than zero for a long period (such as a preset duration threshold), it may be due to accumulated errors. In this case, the speeds are accumulated to calculate a "speed accumulation". If the speed accumulation remains greater than zero, it indicates error accumulation, and the second adjustment rate (i.e., accelerating the leakage speed) can be increased to eliminate the error. It should be understood that in other embodiments, the speed accumulation can also be determined in other ways.

[0093] Additionally, when the speed approaches zero, the cumulative speed value decreases, indicating that the error is being corrected. At this point, the second adjustment rate is reduced (i.e., the second leakage speed is slowed down) to ensure that the speed gradually returns to zero rather than changing abruptly.

[0094] Figure 9A shows a schematic diagram of a waveform example 900A of velocity before leakage filtering is performed using a leaker positively correlated with velocity accumulation, according to some embodiments of the present disclosure. Figure 9B shows a schematic diagram of a waveform example 900B of velocity after leakage filtering is performed using a leaker positively correlated with velocity accumulation, according to some embodiments of the present disclosure. In waveform examples 900A-900B, the horizontal axis may represent time, and the vertical axis may represent velocity values.

[0095] In waveform example 900A, the velocity waveform without leakage filtering is waveform 910. In waveform example 900B, after performing leakage filtering using a leaker positively correlated with the velocity accumulation, the resulting velocity waveform is waveform 920. By comparing waveforms 910 and 920 in waveform example 900B, it can be seen that waveform 920 is generally lower in amplitude than waveform 910. Furthermore, after time point 2, the velocity value of waveform 920 increasingly approaches 0. Therefore, performing leakage filtering using a leaker positively correlated with the velocity accumulation can reduce velocity errors.

[0096] It should be noted that the two methods described above for achieving velocity error correction—one using a leaker negatively correlated with velocity and acceleration, and the other using a leaker positively correlated with velocity accumulation—can be executed in parallel.

[0097] Referring back to Figure 2, in box 230, based on the target velocity at the target time point, server device 130 determines the predicted value of the camera activation requirement of terminal device 110 at the target time point. In some examples, the predicted value may also be referred to as an integral value, which can characterize the camera activation requirement.

[0098] In some embodiments, when determining the predicted value of the camera activation demand of the terminal device 110 at a target time point based on the target speed at the target time point, the server device 130 may determine a third adjustment rate to the historical predicted values, at least based on the target speed at the target time point and historical predicted values ​​of the camera activation demand before the target time point. In some embodiments, the third adjustment rate is negatively correlated with the target speed and positively correlated with the magnitude of the historical predicted values. Subsequently, the server device 130 may update the historical predicted values ​​based on the determined third adjustment rate to obtain the predicted value of the terminal device 110 at the target time point.

[0099] In this embodiment, the predicted value is determined through an integral leakage mechanism. Referring to Figure 4, after obtaining a speed of 430 after speed error correction, an additional predicted value leaker 435 can be used to implement the integral leakage mechanism. In the integral leakage mechanism, the third adjustment rate can be the leakage rate of the historical predicted values ​​that can be obtained at the current moment. According to the integral leakage mechanism, the predicted value used to determine the user's camera activation requirement can be understood as an integral pool. The integral size (i.e., the predicted value) of the integral pool is continuously adjusted through the leakage mechanism, and the determination of whether to activate the camera is based on the comparison between the current integral size (i.e., the currently determined predicted value) and a threshold.

[0100] For the leakers in the predictive value leaker 435 that are positively correlated with the predicted value, the third adjustment rate (also known as the leakage rate corresponding to the integral) can be positively correlated with the magnitude of the historical predicted value. Specifically, the leakage rate can be proportional to the size of the integral pool, meaning the larger the integral (the larger the current predicted value), the faster the leakage. For the leakers in the predictive value leaker 435 that are negatively correlated with the velocity, the third adjustment rate can be negatively correlated with the target velocity. Specifically, the leakage rate can be inversely proportional to the velocity, meaning the lower the velocity, the faster the leakage. Additionally, the magnitude of the historical predicted value is positively correlated with the target velocity; specifically, the integral can be proportional to the magnitude of the velocity.

[0101] In some examples, for the third adjustment rate to be positively correlated with the magnitude of historical forecasts, a dynamic leakage coefficient can be set that is correlated with the current forecast. Assuming λ represents a constant leakage coefficient, the leakage rate can be expressed as λ·I(t), where I(t) can represent the current forecast. Thus, the leakage integral formula can be expressed as: I(t+1)=I(t)·(1-λ·I(t))+x(t+1) (1)

[0102] In the discrete-time case, formula (1) can be transformed into: I[n+1]=I[n]·(1-λ·I[n])+x[n+1] (2)

[0103] Where I[n] can represent the predicted value at time n, λ can represent the constant leakage coefficient, and x[n+1] can represent the input value at time n+1.

[0104] In some examples, for the third adjustment rate which is negatively correlated with the target velocity, it can be assumed that there is a signal x[n], and V[n] is the velocity at the corresponding moment, requiring the leakage rate to be adjusted with the velocity change. The leakage coefficient λ[n] given the velocity V[n] can be expressed as: λ[n] = {V[n] + ε} / k (3)

[0105] Here, ∈ is a very small positive number used to prevent the denominator from being zero when the speed V[n]≈0 (it can usually be taken as a very small value, such as 10). -6 k can be a constant coefficient used to control the proportion of the leakage rate. The leakage integral formula to achieve such a dynamic leakage coefficient can be expressed as: I[n+1]=I[n]·(1-λ[n])+x[n+1] (4)

[0106] Among them, λ[n]={V[n]+ε} / k.

[0107] The leakage integral described by the above formula (4) can be used to calculate the velocity V[n] at each time point n (if this is not directly given, it can be obtained by integrating the acceleration a[n]), then update the leakage coefficient and apply it in formula (4).

[0108] In some embodiments, when determining a third adjustment rate for historical predictions, if the historical predictions exceed a predetermined prediction threshold and the acceleration of the terminal device 110 at multiple time points is greater than a predetermined acceleration threshold, the server device 130 may increase the third adjustment rate.

[0109] In this embodiment, a reward-based points mechanism is used to modify the third adjustment rate. Assuming a predetermined prediction threshold of 0.75, when historical predictions exceed 0.75 and the acceleration is greater than a predetermined acceleration threshold, the points pool can be accelerated, i.e., the third adjustment rate is increased to allow the current prediction to increase more quickly.

[0110] Referring to Figure 4, the predicted value 440 of the terminal device 110 for the camera activation requirement at the target time point is obtained through the leakage mechanism of the prediction value leaker 435.

[0111] Referring back to Figure 2, in box 240, server device 130 determines an enable control command for the camera of terminal device 110 based on a predicted value. The enable control command instructs the camera to be turned on or off. In some embodiments, when determining the enable control command for the camera of terminal device 110 based on the predicted value, server device 130 may determine whether the predicted value is higher than a predetermined upper limit or lower than a predetermined lower limit. If the predicted value is higher than the predetermined upper limit, server device 130 may determine that the enable control command instructs the camera to be turned on. If the predicted value is lower than the predetermined lower limit, server device 130 may determine that the enable control command instructs the camera to be turned off, wherein the predetermined lower limit may include zero.

[0112] Therefore, embodiments of this disclosure can determine whether to control the camera to turn on based on the movement of the terminal device 110. In some scenarios, when the user 140 carries the terminal device 110 (e.g., holds the terminal device 110) at multiple points in time, using embodiments of this disclosure is equivalent to indirectly determining the user 140's intention to turn on the camera at multiple points in time.

[0113] Figure 10A illustrates a schematic diagram of an example application scenario 100A for camera control according to some embodiments of the present disclosure. Figure 10B illustrates a schematic diagram of an example application scenario 100B for camera control according to some embodiments of the present disclosure. Example application scenarios 100A and 100B can be implemented in the environment 100 of Figure 1. Process 200 can be applied to example application scenarios 100A and 100B. Example application scenarios 100A and 100B still use a mobile phone as an example of a terminal device 110.

[0114] In example application scenario 100A, during the process from scenario 1010 to scenario 1020 (e.g., the user is in a flat shooting posture), the user's handheld terminal device 110 moves closer to the target object 1001, and the server device 130 determines the camera activation requirement based on the embodiments of this disclosure.

[0115] In example application scenario 100B, during the transition from scenario 1030 to scenario 1040, the user holds a terminal device 110 and shakes the terminal device 110 left and right in front of the target object 1001. The server device 130 determines the camera activation requirement based on the embodiments of this disclosure.

[0116] In some embodiments, the graphical user interface (GUI) animation presented on the terminal device 110 can be determined based on the predicted value. Referring back to FIG4, when it is determined that the predicted value 440 of the camera activation requirement of the terminal device 110 at a target time point is higher than a predetermined upper limit value, it can be determined to trigger (450) camera activation and present the corresponding GUI event animation. When the predicted value 440 is lower than a predetermined lower limit value (e.g., 0), it means that the user 140's camera activation requirement may have changed or been canceled, and the GUI event can be canceled (460).

[0117] Referring to Figures 10A and 10B, in example application scenarios 100A and 100B, the graphic animation 1002 can serve as an example of a GUI animation. After determining that the user expects to turn on the camera, changes in the graphic animation 1002 (such as the display screen gradually enlarging) can inform the user 140 that the camera is about to be turned on. This further increases interaction efficiency and improves the user experience.

[0118] Figure 11 illustrates a schematic diagram of an example application scenario 1100 for camera control according to some embodiments of the present disclosure. Example application scenario 1100 can be implemented in the environment 100 of Figure 1. Process 200 can be applied to example application scenario 1100. Example application scenario 1100 can be a scenario where the user is in a different shooting posture than in example application scenario 100A of Figure 10A. For example, scenario 1110 could be a user holding the terminal device 110 in a downward shooting posture, intending to photograph an object. Scenario 1120 could be a user holding the terminal device 110 in a downward shooting posture, intending to photograph an object.

[0119] Figure 12 illustrates a schematic diagram of a process 1200 for camera control according to some embodiments of the present disclosure. Process 1200 can be implemented in the environment 100 of Figure 1. Some steps in process 1200 are identical to some steps in process 400. Relative to process 400, process 1200 employs a machine learning model 160 to determine the intention to activate the camera of the terminal device 110 based on acceleration 420, as follows. It should be noted that the machine learning model 160 may include, for example, but not limited to, model structures such as a combination of convolutional neural networks (CNN) and Transformers, a combination of recurrent neural networks (RNN) and Transformers, long short-term memory networks (LSTM), and any suitable variations thereof.

[0120] Figure 13 illustrates a flowchart of a camera control process 1300 according to some embodiments of the present disclosure. For ease of discussion, these embodiments will be described with reference to the environment 100 of Figure 1. These embodiments may be implemented in the server device 130 of Figure 1.

[0121] In block 1310, based on the motion information collected by the terminal device 110 at multiple time points, the server device 130 determines the acceleration of the terminal device 110 at the multiple time points. In some embodiments, when acquiring the motion information collected by the terminal device 110 at multiple time points, the server device 130 may utilize the inertial measurement unit (IMU) of the terminal device 110 to collect the motion information at multiple time points. The specific method for determining the acceleration of the terminal device 110 at multiple time points has been discussed above and will not be repeated here.

[0122] In box 1320, server device 130 inputs acceleration data at multiple time points into a trained machine learning model 160 to obtain predictions indicating a camera activation requirement. In box 1330, server device 130 determines an activation control command for the camera on terminal device 110 based on the prediction results. The activation control command indicates whether the camera should be turned on or off. Thus, the machine learning model 160 can determine the camera activation intention of terminal device 110.

[0123] In box 1320, the training samples for machine learning model 160 during the training process include positive samples and negative samples. Positive samples include motion information collected by the sample device when there is a camera activation requirement and a first label indicating the camera activation requirement, while negative samples include motion information collected by the sample device when there is no camera activation requirement and a second label indicating the absence of a camera activation requirement.

[0124] It should be noted that when collecting positive and negative samples, the object manipulating the sample device (i.e., the sample terminal device) operates the device, either when the camera needs to be turned on or not. Motion information belonging to positive samples and motion information belonging to negative samples are extracted from all motion information recorded by the sample device using timestamp information.

[0125] Figure 14 illustrates a schematic diagram of an example architecture 1400 of a machine learning model 160 according to some embodiments of the present disclosure. Architecture 1400 primarily describes a system for Human Activity Recognition (HAR).

[0126] In architecture 1400, the input to the raw motion information of the IMU (1410) includes three different types of sensor data: gravitational acceleration data (e.g., represented by Grav_ang(i)), gyroscope data (angular velocity, e.g., represented by Gyro(i)), and linear acceleration data (e.g., represented by Linear_acc(i)). This sensor data is provided as input to the subsequent IMU fusion module 1420.

[0127] Furthermore, the IMU fusion module 1420 divides the motion information collected by various sensors in the IMU into two processing branches: a posture branch and a motion branch. The posture branch processes gravitational acceleration and gyroscope data to extract posture features. The motion branch processes linear acceleration data to extract motion features. These features are then concatenated together in the "connection" step 1421 to form a comprehensive feature vector.

[0128] Furthermore, the ConvTransformer 1430 further processes the concatenated features. First, features are extracted using convolutional layers (Convs) to generate convolutional embeddings 1431. Then, a cross-channel interaction encoder 1432 is used for interactive encoding of cross-channel features, enhancing feature expressiveness. Next, two self-attention modules (Self-Attn Block 1 and Module 2) in the Self-Attn Encoder 1433 are used to further extract features. The attention mechanism (Self-Attention) can be used to capture the relationships between different features.

[0129] Finally, the processed features are processed by the self-attention encoder 1433 to generate classification results 1440. These classification results 1440 can be used to identify different types of activities, such as walking, running, standing, etc.

[0130] Figure 15 shows a flowchart of an application process 1500 for camera control according to some embodiments of the present disclosure. The application process 1500 can be implemented in the environment 100 of Figure 1. The application process 1500 can be a specific application of process 400 or process 1200.

[0131] In process 1500, user 140 can activate application 120 on terminal device 110 through a preset operation (such as a long press) to enter multi-channel (such as three-channel) mode (1520). For a long press (1511), the time between the terminal device 110 receiving the user 140's press operation and the release operation can be set to a preset duration (such as 300 milliseconds), which is adjustable. In the three-channel standby mode 1530, user 140 can choose from three modes: voice 1531, voice combined with drawing 1532, and voice combined with video 1533. It should be understood that other modes may also be provided for user 140 to choose from.

[0132] Furthermore, after selecting the voice-video combined mode 1533, the camera control process 400 or process 1200 of this embodiment can be used to determine the intention to turn on the camera of the terminal device 110. If the intention is determined to be to turn on the camera, the camera can be turned on for shooting in conjunction with the GUI event described above (1540).

[0133] Therefore, by applying the intelligent dialogue capability of 120 in combination with the method of determining the camera activation intention of the terminal device 110 in the embodiments of this disclosure, the interaction efficiency can be effectively improved and the user experience can be enhanced.

[0134] Figure 16 shows a schematic structural block diagram of a camera control device 400 according to certain embodiments of the present disclosure. The device 1600 may be implemented as or included in the server device 130. The various modules / components in the device 1600 may be implemented by hardware, software, firmware, or any combination thereof.

[0135] As shown in the figure, the device 1600 includes an acceleration determination module 1610, configured to determine the acceleration of the terminal device at multiple time points based on motion information collected by the terminal device at multiple time points; a target velocity determination module 1620, configured to determine the target velocity of the terminal device at a target time point based on the acceleration of the terminal device at multiple time points; a prediction value determination module 1630, configured to determine the predicted value of the terminal device at the target time point for camera activation requirements based on the target velocity at the target time point; and an activation control module 1640, configured to determine an activation control command for the camera of the terminal device based on the predicted value, the activation control command being used to instruct the camera to be activated or deactivated.

[0136] In some embodiments, the acceleration determination module 1610 is further configured to perform preprocessing on the acquired motion information; and to determine the acceleration of the terminal device at multiple time points based on the preprocessed motion information.

[0137] In some embodiments, preprocessing includes at least one of gravity filtering and high-pass filtering.

[0138] In some embodiments, the target speed determination module 1620 is further configured to determine the predicted speed of the terminal device at a target time point based on the acceleration of the terminal device at multiple time points; determine a first adjustment rate for the target time point based on the acceleration and predicted speed of the terminal device at the target time point, the first adjustment rate being negatively correlated with the acceleration and predicted speed at the target time point; and adjust the predicted speed of the terminal device at the target time point based on the first adjustment rate to obtain the target speed.

[0139] In some embodiments, the target speed determination module 1620 is further configured to determine the predicted speed of the terminal device at multiple time points based on the acceleration of the terminal device at multiple time points; determine the duration for which the predicted speed is greater than zero; determine a second adjustment rate for the target time point based on the duration, the second adjustment rate being positively correlated with the magnitude of the duration; and adjust the predicted speed of the terminal device at the target time point based on the second adjustment rate to obtain the target speed.

[0140] In some embodiments, the prediction value determination module 1630 is further configured to determine a third adjustment rate to the historical prediction value based at least on the target speed at the target time point and the historical prediction value for camera activation demand before the target time point; and to update the historical prediction value based on the determined third adjustment rate to obtain the prediction value of the terminal device at the target time point.

[0141] In some embodiments, the third adjustment rate is negatively correlated with the target velocity and positively correlated with the magnitude of historical forecasts.

[0142] In some embodiments, the prediction value determination module 1630 is further configured to increase the third adjustment rate in response to a historical prediction value exceeding a predetermined prediction threshold and the acceleration of the terminal device at multiple time points being greater than a predetermined acceleration threshold.

[0143] In some embodiments, the activation control module 1640 is further configured to determine whether the predicted value is higher than a predetermined upper limit or lower than a predetermined lower limit; if the predicted value is higher than the predetermined upper limit, an activation control instruction is determined to instruct the camera to be activated; or if the predicted value is lower than the predetermined lower limit, an activation control instruction is determined to instruct the camera to be deactivated, wherein the predetermined lower limit includes zero.

[0144] Figure 17 shows a schematic structural block diagram of a camera control device 1700 according to certain embodiments of the present disclosure. The device 1700 may be implemented as or included in server device 130. The various modules / components in the device 1700 may be implemented by hardware, software, firmware, or any combination thereof.

[0145] As shown in the figure, the device 1700 includes an acceleration determination module 1710, configured to determine the acceleration of the terminal device at multiple time points based on motion information collected by the terminal device at multiple time points; a prediction module 1720, configured to input the acceleration at multiple time points into a trained machine learning model to obtain a prediction result, the prediction result indicating a prediction for camera activation demand, wherein the training samples of the machine learning model during the training process include positive samples and negative samples, positive samples include motion information collected by the sample device when there is a camera activation demand and a first label indicating the camera activation demand, negative samples include motion information collected by the sample device when there is no camera activation demand and a second label indicating no camera activation demand; and an activation control module 1730, configured to determine an activation control command for the camera of the terminal device based on the prediction result, the activation control command being used to indicate whether the camera is activated or not.

[0146] In some embodiments, the device 1700 further includes a motion information acquisition module configured to acquire motion information at multiple time points using an inertial measurement unit (IMU) of the terminal device.

[0147] The units and / or modules included in devices 1600 and 1700 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units and / or modules can be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to or as an alternative to machine-executable instructions, some or all of the units and / or modules in devices 1600 and 1700 can be implemented at least partially by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0148] Figure 18 illustrates a block diagram of an electronic device 1800 in which one or more embodiments of the present disclosure may be implemented. It should be understood that the electronic device 1800 shown in Figure 18 is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. The electronic device 1800 shown in Figure 18 can be used to implement the extended reality device 120 of Figure 1.

[0149] As shown in Figure 18, electronic device 1800 is in the form of a general-purpose computing device. Components of electronic device 1800 may include, but are not limited to, one or more processors or processing units 1810, memory 1820, storage device 1830, one or more communication units 1840, one or more input devices 1850, and one or more output devices 1860. Processor 1810 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 1820. In a multiprocessor system, multiple processors execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 1800.

[0150] Electronic device 1800 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to electronic device 1800, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 1820 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 1830 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data and can be accessed within electronic device 1800.

[0151] Electronic device 1800 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in FIG18, disk drives for reading or writing from removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading or writing from removable, non-volatile optical disks may be provided. In these cases, each drive may be connected to a bus (not shown) via one or more data media interfaces. Memory 1820 may include computer program product 1825 having one or more program modules configured to perform various methods or actions of various embodiments of the present disclosure.

[0152] The communication unit 1840 enables communication with other computing devices via a communication medium. Additionally, the functionality of the components of the electronic device 1800 can be implemented as a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the electronic device 1800 can operate in a networked environment using logical connections to one or more other servers, networked personal computers (PCs), or another network node.

[0153] Input device 1850 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 1860 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 1800 can also communicate with one or more external devices (not shown) via communication unit 1840 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 1800, or with any device that enables electronic device 1800 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0154] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0155] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0156] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0157] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0158] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0159] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for controlling a camera, comprising: Based on motion information collected by the terminal device at multiple time points, the acceleration of the terminal device at those multiple time points is determined. Based on the acceleration of the terminal device at the multiple time points, the target speed of the terminal device at the target time point is determined; Based on the target speed at the target time point, determine the predicted value of the camera activation requirement of the terminal device at the target time point; as well as Based on the predicted value, an activation control command is determined for the camera of the terminal device, which is used to indicate whether the camera is turned on or off.

2. The method according to claim 1, wherein determining the acceleration of the terminal device at the plurality of time points comprises: Preprocessing is performed on the collected motion information; as well as Based on the preprocessed motion information, the acceleration of the terminal device at the multiple time points is determined.

3. The method according to claim 2, wherein the preprocessing includes at least one of gravity filtering and high-pass filtering.

4. The method according to claim 1, wherein determining the target speed of the terminal device at the target time point comprises: Based on the acceleration of the terminal device at the multiple time points, the predicted speed of the terminal device at the target time point is determined; Based on the acceleration of the terminal device at the target time point and the predicted velocity, a first adjustment rate is determined for the target time point, wherein the first adjustment rate is negatively correlated with both the acceleration and the predicted velocity at the target time point; and The predicted speed of the terminal device at the target time point is adjusted based on the first adjustment rate to obtain the target speed.

5. The method according to claim 1, wherein determining the target speed of the terminal device at the target time point further includes: Based on the acceleration of the terminal device at the multiple time points, the predicted speed of the terminal device at the multiple time points is determined; Determine the duration for which the predicted velocity is greater than zero; A second adjustment rate is determined for the target time point based on the duration, and the second adjustment rate is positively correlated with the magnitude of the duration; as well as The predicted speed of the terminal device at the target time point is adjusted based on the second adjustment rate to obtain the target speed.

6. The method of claim 1, wherein determining the predicted value of the camera activation requirement of the terminal device at the target time point based on the target speed at the target time point comprises: A third adjustment rate for the historical predictions is determined based at least on the target velocity at the target time point and the historical predictions of camera activation demand prior to the target time point. as well as The historical prediction value is updated based on the determined third adjustment rate to obtain the prediction value of the terminal device at the target time point.

7. The method of claim 6, wherein the third adjustment rate is negatively correlated with the target velocity and the third adjustment rate is positively correlated with the magnitude of the historical predicted value.

8. The method of claim 6, wherein determining the third adjustment rate for the historical forecast values ​​comprises: In response to the historical predicted value exceeding a predetermined prediction threshold and the acceleration of the terminal device at the multiple time points being greater than a predetermined acceleration threshold, the third adjustment rate is increased.

9. The method according to claim 1, wherein determining the camera activation control command of the terminal device based on the predicted value comprises: Determine whether the predicted value is higher than a predetermined upper limit or lower than a predetermined lower limit; If the predicted value is higher than the predetermined upper limit value, it is determined that the activation control command is used to instruct the camera to be activated; or If the predicted value is lower than the predetermined lower limit, the activation control command is determined to instruct the camera not to be activated, wherein the predetermined lower limit includes zero.

10. A method for controlling a camera, comprising: Based on motion information collected by the terminal device at multiple time points, the acceleration of the terminal device at those multiple time points is determined. The acceleration at the multiple time points is input into a trained machine learning model to obtain a prediction result, which indicates a prediction for camera activation demand. The training samples of the machine learning model during training include positive samples and negative samples. The positive samples include motion information collected by the sample device when there is a camera activation demand and a first label indicating the camera activation demand. The negative samples include motion information collected by the sample device when there is no camera activation demand and a second label indicating no camera activation demand. as well as Based on the prediction results, an activation control command is determined for the camera of the terminal device. The activation control command is used to indicate whether the camera is turned on or off.

11. The method according to claim 10, wherein acquiring motion information collected by the terminal device at multiple time points includes: The inertial measurement unit (IMU) of the terminal device is used to collect motion information at multiple time points.

12. A device for camera control, comprising: An acceleration determination module is configured to determine the acceleration of the terminal device at multiple time points based on motion information collected by the terminal device at multiple time points. The target speed determination module is configured to determine the target speed of the terminal device at a target time point based on the acceleration of the terminal device at multiple time points; The prediction value determination module is configured to determine, based on the target speed at the target time point, the predicted value of the terminal device for the camera activation requirement at the target time point; as well as The activation control module is configured to determine an activation control command for the camera of the terminal device based on the predicted value, the activation control command being used to instruct whether the camera is activated or deactivated.

13. A device for camera control, comprising: An acceleration determination module is configured to determine the acceleration of the terminal device at multiple time points based on motion information collected by the terminal device at multiple time points. The prediction module is configured to input the acceleration at the plurality of time points into a trained machine learning model to obtain a prediction result indicating a prediction for camera activation demand. The training samples of the machine learning model during training include positive samples and negative samples. The positive samples include motion information collected by the sample device when there is a camera activation demand and a first label indicating the camera activation demand. The negative samples include motion information collected by the sample device when there is no camera activation demand and a second label indicating no camera activation demand. as well as The activation control module is configured to determine an activation control command for the camera of the terminal device based on the prediction result, the activation control command being used to instruct whether the camera is activated or deactivated.

14. An electronic device comprising: At least one processor; as well as At least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, the instructions causing the electronic device to perform the method according to any one of claims 1 to 9 or 10 to 11 when executed by the at least one processor.

15. A computer-readable storage medium having stored thereon computer-executable instructions that can be executed by a processor to implement the method according to any one of claims 1 to 9 or 10 to 11.

16. A computer program product tangibly stored in a computer storage medium and comprising computer-executable instructions that, when executed by a device, cause the device to perform the method according to any one of claims 1 to 9 or 10 to 11.

Images