Device control method and apparatus, electronic device, and storage medium

Abstract

The application discloses a device control method and device, an electronic device and a storage medium, and relates to the technical field of device control. The method comprises the following steps: acquiring a target gesture; in response to the target gesture, acquiring a control operation corresponding to the target gesture, the control operation being used for controlling adjustment of a state corresponding to the electronic device; and executing the control operation. The method can control adjustment of the state corresponding to the device through the gesture, so that the device is more convenient to control through the gesture, the intelligence of the device is improved, and the user experience is improved.

Description

Technical Field

[0001] This application relates to the field of equipment control technology, and more specifically, to an equipment control method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the development of electronic device technology, more and more electronic devices support users to control related functions of electronic devices through gestures. However, the related gesture control methods are not convenient enough, which affects the user experience. Summary of the Invention

[0003] This application proposes a device control method, apparatus, electronic device, and storage medium to improve the above-mentioned problems.

[0004] In a first aspect, embodiments of this application provide a device control method applied to an electronic device, the method comprising: acquiring a target gesture; in response to the target gesture, acquiring a control operation corresponding to the target gesture, the control operation being used to control and adjust a corresponding state of the device; and executing the control operation.

[0005] Secondly, embodiments of this application provide a device control apparatus operating on an electronic device. The apparatus includes: a gesture acquisition module for acquiring a target gesture; an operation acquisition module for acquiring a control operation corresponding to the target gesture in response to the target gesture, the control operation being used to control and adjust the corresponding state of the device; and a device control module for executing the control operation.

[0006] Thirdly, embodiments of this application provide an electronic device, including: one or more processors and a memory; one or more application programs are stored in the memory and configured to be executed by the one or more processors, the one or more application programs being configured to perform the device control method provided in the first aspect above.

[0007] Fourthly, embodiments of this application provide a computer-readable storage medium storing program code, wherein the device control method provided in the first aspect is executed when the program code is run by a processor.

[0008] This application provides a device control method, apparatus, electronic device, and storage medium. The method acquires a target gesture; then, in response to the target gesture, acquires a control operation corresponding to the target gesture, the control operation being used to control and adjust the corresponding state of the electronic device; and then executes the control operation. Thus, this method enables the adjustment of the device's state through gesture control, making gesture control of the device more convenient, improving the device's intelligence, and enhancing the user experience. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 The diagram illustrates an application scenario of the device control method provided in the embodiments of this application.

[0011] Figure 2 A flowchart of a device control method provided in an embodiment of this application is shown.

[0012] Figure 3 An example diagram of a static gesture provided in an embodiment of this application is shown.

[0013] Figure 4 Another example diagram of static gestures provided in an embodiment of this application is shown.

[0014] Figure 5 Another example diagram of static gestures provided in the embodiments of this application is shown.

[0015] Figure 6 An example diagram of a dynamic gesture provided in an embodiment of this application is shown.

[0016] Figure 7 Another example diagram of the dynamic gestures provided in the embodiments of this application is shown.

[0017] Figure 8 This illustration shows yet another example of the dynamic gestures provided in the embodiments of this application.

[0018] Figure 9 An example diagram is shown, illustrating a gesture captured within a specified shooting range of a camera, as provided in an embodiment of this application.

[0019] Figure 10 A flowchart of a device control method provided in another embodiment of this application is shown.

[0020] Figure 11 A flowchart of a device control method provided in another embodiment of this application is shown.

[0021] Figure 12 A flowchart of a device control method provided in another embodiment of this application is shown.

[0022] Figure 13 This illustration shows an example of a process for adjusting the state of a device based on the adjustment amount determined by the movement parameters of a dynamic gesture, according to an embodiment of this application.

[0023] Figure 14 This illustration shows another example of a process for adjusting the state of a device based on the adjustment amount determined by the movement parameters of a dynamic gesture, as provided in an embodiment of this application.

[0024] Figure 15 This illustration shows another example of a process for adjusting the state of a device based on the adjustment amount determined by the movement parameters of a dynamic gesture, as provided in an embodiment of this application.

[0025] Figure 16 This illustration shows another example of a process for adjusting the state of a device based on the adjustment amount determined by the movement parameters of a dynamic gesture, as provided in an embodiment of this application.

[0026] Figure 17 This illustration shows another example of a process for adjusting the state of a device based on the adjustment amount determined by the movement parameters of a dynamic gesture, as provided in an embodiment of this application.

[0027] Figure 18 A flowchart of a device control method provided in another embodiment of this application is shown.

[0028] Figure 19 A structural block diagram of a device control apparatus provided in an embodiment of this application is shown.

[0029] Figure 20 A structural block diagram of an electronic device provided in an embodiment of this application is shown.

[0030] Figure 21 An embodiment of the present application shows a storage unit for storing or carrying program code that implements the device control method according to an embodiment of the present application. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0032] With the popularization of smart terminals, users have higher and higher requirements for consumer products such as smart TVs. It's not just about sound quality, picture quality, and aesthetics; the control methods of smart terminals, especially the control methods that smart terminals can support, are also gradually becoming an important standard for judging smart devices.

[0033] However, existing smart devices typically require a remote control for operation, such as muting, playing or pausing videos. Furthermore, controlling these functions via the remote often involves multiple steps; and when the remote is unavailable or out of power, this increases the complexity of control and negatively impacts the user experience.

[0034] Through long-term research, the inventors discovered that a target gesture can be acquired; then, in response to the target gesture, a corresponding control operation can be acquired, which is used to control and adjust the state of the electronic device; and then the control operation can be executed. This method enables gesture-based control of the device's state, making gesture control more convenient, improving device intelligence, and enhancing the user experience.

[0035] Therefore, in order to improve the above problems, the inventors have proposed the device control method, apparatus, electronic device and storage medium provided in this application, which allows the device to be adjusted according to the corresponding state by gesture control, making it more convenient to control the device by gesture, improving the intelligence of the device and enhancing the user experience.

[0036] The application scenarios involved in the device control method provided in the embodiments of this application will be described below with reference to the accompanying drawings.

[0037] Please see Figure 1 The diagram illustrates an application scenario of the device control method provided in this embodiment. Figure 1 As shown, the electronic device is equipped with a camera (the specific type of camera is not limited, for example, it can be a monocular camera, a binocular camera, etc.), or the electronic device is connected to an image acquisition device such as a camera (e.g., an external camera). The user makes a gesture, which is recognized and confirmed by the electronic device, allowing the user to control the device to perform the corresponding control operation. Different gestures are pre-configured to correspond to different control operations. For example, as an example, suppose the "clenched fist" gesture is configured to control the electronic device to turn off. Then, as follows... Figure 1 As shown, when a user makes a "fist-clasped" gesture towards an electronic device, the device can be turned off (at this time, the screen of the electronic device can display the words "Your device is about to turn off" to remind the user that the electronic device is about to turn off, that is, the electronic device has responded to the "fist-clasped" gesture).

[0038] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0039] Please see Figure 2This document illustrates a flowchart of a device control method according to an embodiment of this application. This embodiment provides a device control method applicable to electronic devices. The electronic device in this embodiment can be a mobile communication device with network connectivity, such as a mobile phone, computer, tablet, or television. The electronic device in this embodiment is equipped with an image acquisition device, or is electrically connected to an image acquisition device (in this case, the image acquisition device can transmit the acquired image to the electronic device). The image acquisition device can be a camera, and the specific type of image acquisition device is not limited. This embodiment uses a camera as an example to illustrate the method, which includes:

[0040] Step S110: Obtain the target gesture.

[0041] In this embodiment of the application, the target gesture represents a control gesture captured by a camera. The target gesture may include a static gesture or a dynamic gesture. A static gesture can be understood as a gesture made by one or both hands that remains stationary relative to the camera (where slight hand tremors can be ignored, i.e., slight hand tremors can be considered as remaining stationary relative to the camera). A dynamic gesture can be understood as a gesture made by one or both hands that moves along a certain trajectory.

[0042] As one implementation method, static gestures may include gestures such as the "clenched fist" gesture, the "V" sign, and the "six" gesture. See also... Figure 3 , Figure 4 as well as Figure 5 Examples of static gestures are shown below. For example, ... Figure 3 As shown, the "clasped fist" gesture can be Figure 3 The left hand shown in the image on the left, with the left palm on top of the right fist, can also be considered... Figure 3 The image on the right shows a position with the right palm on top of the left fist, the left hand clenched into a fist. Figure 4 As shown, the "V" sign can be Figure 4 The image on the left shows the left hand with the index and middle fingers spread out, the little finger, ring finger, and thumb tucked in, and the thumb pressing down on the little and ring fingers. This shape can also be considered... Figure 4 The image on the right shows the right index and middle fingers spread out, while the little finger, ring finger, and thumb are tucked in. The thumb presses down on the little and ring fingers, or the thumb is wrapped around the little and ring fingers. Figure 4 (The shape is not shown in the diagram on the right.) For example... Figure 5 As shown, the "number six" gesture can be... Figure 5 The left-hand image shows the little finger and thumb spread out, while the index, middle, and ring fingers are tucked in; this can also be interpreted as... Figure 5 The image on the right shows the right hand with the little finger and thumb spread out, and the index, middle, and ring fingers tucked in.

[0043] As one implementation, dynamic gestures may include gestures such as a "moving palm" gesture, a "moving fist" gesture, and a "moving rocking gesture." See also... Figure 6 , Figure 7 as well as Figure 8 Examples of dynamic gestures are shown below. For example, ... Figure 6 As shown, the "palm" in the "move palm" gesture can be... Figure 6 The upper image shows the left hand with its five fingers spread or in a natural position. In this case, the "moving the palm" gesture can be understood as moving the left palm from... Figure 6 Move to the position shown on the right side of the upper image. Figure 6 The position shown on the left side of the upper part of the image; it can also be... Figure 6 The lower image shows the right hand with its five fingers together or in a natural position. In this case, the "moving palm" gesture can be understood as moving the left palm from... Figure 6 The position shown on the lower left of the image is moved to... Figure 6 The position shown on the right side of the lower part of the image. (See the image below.) Figure 7 As shown, the "fist" in the "move fist" gesture can be... Figure 7 The upper image shows the left hand with its five fingers tucked in, the thumb either wrapped around or pressing down on the index finger. In this case, the "moving fist" gesture can be understood as moving the left fist from... Figure 7 Move to the position shown on the left side of the upper image. Figure 7 The position shown on the right side of the upper part of the image; it can also be... Figure 7 The lower image shows the right hand with its five fingers tucked in, the thumb either wrapped around or pressing down on the index finger. In this case, the "moving fist" gesture can be understood as moving the left fist from... Figure 7 Move to the position shown on the right side of the lower image. Figure 7 The position shown on the lower left of the image. Figure 8 As shown, the "rock gesture" in "moving rock gesture" can be... Figure 8 The upper image shows the right hand with the little finger and index finger spread out, the ring finger, middle finger, and thumb tucked in, and the thumb resting on the ring and middle fingers. In this position, the "moving rock hand gesture" can be understood as a variation of the right-hand rock hand gesture... Figure 8 Move to the position shown on the left side of the upper image. Figure 8 The position shown on the right side of the upper part of the image; or it can be Figure 8 The lower image shows the left hand with the little finger and index finger spread out, the ring finger, middle finger, and thumb tucked in, and the thumb resting on the ring and middle fingers. In this position, the "moving rock hand gesture" can be understood as extending the left-hand rock hand gesture from... Figure 8 Move to the position shown on the right side of the lower image. Figure 8The position shown on the left side of the lower part of the image.

[0044] It should be noted that, in this embodiment, the direction of movement of the dynamic gesture is relative to the movement direction of the electronic device. Specifically, it can be the direction of movement of the dynamic gesture relative to the camera of the electronic device. The specific direction of movement is not limited; for example, it can be moving to the left, right, up, down, forward, or backward relative to the electronic device, etc., which will not be listed here.

[0045] It should be noted that the static gestures and dynamic gestures in the embodiments of this application can have the same form. In this way, when a static gesture moves, it can be regarded as a dynamic gesture.

[0046] One approach is to acquire gestures within a specified shooting range of the camera; then, preset gestures whose morphological similarity to the captured gestures is greater than a preset threshold are used as target gestures. The specified shooting range can be the camera's shooting angle, shooting distance, or shooting angle. In this embodiment, the camera's shooting angle can be any angle between 0° and 360°, and the shooting distance can be 1 meter, 3 meters, 5 meters, or other values. Another implementation method is to store a set of gestures, including dynamic and static gestures, as preset gestures in the electronic device. When acquiring gestures within the specified shooting range of the camera, the acquired gestures may not have a preset gesture standard. In this approach, the morphological similarity between the acquired gesture and the preset gestures can be acquired. The specific principle and implementation process for acquiring morphological similarity are not detailed here. When acquiring the morphological similarity between the acquired gesture and the preset gestures, one or more preset gestures with roughly the same or similar morphology can be pre-judged. Then, the morphological similarity between the acquired gesture and one or more preset gestures is calculated. Finally, preset gestures whose morphological similarity to the acquired gesture is greater than a preset threshold are used as target gestures. The specific value of the preset threshold can be set according to actual conditions.

[0047] For example, in a specific application scenario, such as Figure 9 As shown, assuming Figure 9 The gesture shown is a gesture within the specified shooting range of the camera. If calculated... Figure 9 The gestures shown are Figure 4 If the morphological similarity between the gestures shown in the right-hand image is greater than a preset threshold, then it can be... Figure 4 The gesture shown in the right-hand image is the target gesture.

[0048] In this embodiment, the target gesture can be a frontal gesture or a back gesture corresponding to the frontal gesture. That is, when a user controls the device by making gestures, the user can make gestures with their hand facing the camera of the electronic device, or with the back of their hand facing the camera. Furthermore, the control operation corresponding to the frontal gesture of the same gesture is the same as the control operation corresponding to the back gesture. As one approach, the electronic device can pre-store both the frontal and back gestures of the target gesture; alternatively, it can only store the frontal gesture of the target gesture and then obtain the back gesture based on the stored frontal gesture. For example, if the electronic device pre-stores... Figure 4 The "V" sign shown in the right-hand image (this is a frontal gesture) can be summed with the currently detected gesture (which is currently a non-frontal gesture) and the result. Figure 4 The back gesture corresponding to the "yes" gesture shown in the right figure is matched. If the currently detected gesture matches the obtained back gesture, then the matched gesture (i.e. the currently detected gesture) is taken as the target gesture.

[0049] Step S120: In response to the target gesture, obtain the control operation corresponding to the target gesture, the control operation being used to control and adjust the state of the electronic device.

[0050] In this embodiment, different target gestures can correspond to different control operations. For example, as mentioned above... Figure 3 The "clenched fist" gesture shown corresponds to the control operation of turning off the device, while Figure 4 The "V" sign shown can be used to open the camera app on an electronic device. Since users often make the "V" sign when taking photos, they have already established an association between the "V" sign and "taking a photo." For ease of memorization, the control associated with the "V" sign can be set to open the camera app on an electronic device.

[0051] In some embodiments, the control operations corresponding to the same target gesture can be different, for example... Figure 4 The "V" gesture shown can be used to control either opening the camera app on an electronic device or to activate the camera app's photo-taking function. These control operations are used to adjust the device's status.

[0052] This involves pre-storing different target gestures and their corresponding control operations in electronic devices, establishing a mapping relationship between them. An associative relationship exists between the target gestures and their corresponding control operations. For example, the "clasped fist" gesture is commonly used in daily life to indicate "see you later" or "goodbye." Using this gesture to turn off the TV can be understood as saying goodbye to the television, which is both easy to associate and remember, and also has a sense of ritual. Another example is using the "six" gesture to answer a phone call, enabling convenient and quick call answering. By establishing a natural associative relationship between gestures and control operations, users can easily remember them, reducing the learning curve.

[0053] In one approach, given a target gesture, a control operation corresponding to the target gesture can be obtained in response to the target gesture, enabling convenient control of the device by executing the control operation through an electronic device.

[0054] As one implementation, if the target gesture is a static gesture, the state may include at least one of the following: the opening and closing of a device (e.g., a television or mobile phone), the opening and closing of hardware components in the device (e.g., a display screen, a camera, or other sensors), the opening and closing of an application configured in the device, the opening and closing of application functions configured in the device, and the execution of an application function configured in the device. For example, if the static gesture is as described above... Figure 3 The "clenched fist" gesture shown can be used to control the TV to turn off and the TV screen to turn off. In other words, the states include the TV being on and off and the TV screen being on and off.

[0055] If the target gesture is a dynamic gesture, the state can include at least one of volume state, image quality state, and playback state. Image quality state can refer to the image quality effect during video or image playback on the electronic device, such as brightness and contrast. Similarly, playback state can include playback progress state and playback speed (frequency) state. For example, when the interface of the electronic device is a video or audio playback interface, the adjustment amount corresponding to the movement direction of the dynamic gesture "moving the palm" can be used to adjust the playback time or playback speed; the adjustment amount corresponding to the movement direction of the dynamic gesture "moving the fist" can be used to adjust the playback volume; and the adjustment amount corresponding to the movement direction of the dynamic gesture "moving the rock gesture" can be used to adjust the playback image quality.

[0056] In a specific application scenario, suppose the target gesture is to move the aforementioned... Figure 6The hand gesture shown represents the control operation corresponding to the target gesture: if the hand is moved in the first direction, the TV volume is increased; if the hand is moved in the second direction (the second direction is different from the first direction, for example, the first direction could be moving to the right, and the second direction could be moving upwards), the TV screen brightness is increased. In this mode, the state includes both the TV volume state and the brightness state of the TV screen. Step S130: Execute the control operation.

[0057] The device's state can be adjusted by executing control operations corresponding to the defined target gesture, without the need for remote controls or other control devices. This reduces the complexity of device control and enhances the user's convenience.

[0058] The device control method provided in this embodiment acquires a target gesture; then, in response to the target gesture, acquires a control operation corresponding to the target gesture, the control operation being used to control and adjust the corresponding state of the electronic device; and then executes the control operation. Thus, the above method enables the adjustment of the device's corresponding state through gesture control, making gesture control of the device more convenient, improving the device's intelligence, and enhancing the user experience.

[0059] Please see Figure 10 The diagram illustrates a flowchart of a device control method according to another embodiment of this application. This embodiment provides a device control method applicable to electronic devices, and the method includes:

[0060] Step S210: Obtain the target gesture.

[0061] Step S220: In response to the target gesture, obtain the control operation corresponding to the target gesture, the control operation being used to control and adjust the state of the electronic device.

[0062] Step S230: If the target gesture is a static gesture, obtain the duration of the static gesture.

[0063] Understandably, when the target gesture is a static gesture, directly executing the control operation corresponding to that static gesture may lead to misoperation. For example, suppose the electronic device is a television, the static gesture is a "palm" gesture, and the control operation corresponding to the "palm" gesture is to turn off the television. When a user is sitting in front of the television with a friend, watching television and talking, they may also make the palm gesture. In this case, if the television is turned off when the "palm" gesture is recognized, it may affect the user's normal use of the television.

[0064] To address the aforementioned issues, one approach is to obtain the duration of the static gesture if it is determined to be a static gesture; that is, to determine how long the user maintained the static gesture. Specifically, a timer can be started when the target gesture is identified as static to track its duration.

[0065] Step S241: If the holding time reaches the first preset time, execute the control operation.

[0066] The first preset duration represents the critical duration for triggering the control operation corresponding to the static gesture. The specific value of the first preset duration is not limited. For example, the first preset duration can be 1 second, 2 seconds, 3 seconds, or other values.

[0067] In one approach, if the duration of a static gesture reaches a first preset duration, the control operation is executed. For example, if the first preset duration is 2 seconds and the static gesture is a "clenched fist" gesture, if the duration of the "clenched fist" gesture is equal to or greater than 2 seconds, the control operation corresponding to the "clenched fist" gesture can be executed.

[0068] Step S242: If the holding time does not reach the first preset time, the control operation is not performed.

[0069] As one approach, if the duration of a static gesture is less than a first preset duration, then the control operation corresponding to that static gesture will not be executed. Continuing with the example above, if the duration of the "clenched fist" gesture is less than 2 seconds, then the control operation corresponding to that "clenched fist" gesture will not be executed.

[0070] The device control method provided in this embodiment acquires a target gesture; then, in response to the target gesture, acquires a control operation corresponding to the target gesture, the control operation being used to control and adjust the corresponding state of the electronic device; if the target gesture is a static gesture, the duration of the static gesture is acquired, and if the duration reaches a first preset duration, the control operation is executed; if the duration does not reach the first preset duration, the control operation is not executed. Thus, the above method enables the adjustment of the device's corresponding state through gesture control, making gesture control more convenient, improving device intelligence, and enhancing the user experience. Furthermore, by recognizing the target gesture as a static gesture and ensuring that the duration of the static gesture reaches the first preset duration, the corresponding control operation is triggered; otherwise, the corresponding control operation is not triggered, thereby reducing the possibility of misoperation and improving the accuracy of gesture control of the device.

[0071] Please see Figure 11The diagram illustrates a flowchart of a device control method according to another embodiment of this application. This embodiment provides a device control method applicable to electronic devices, and the method includes:

[0072] Step S310: Obtain the target gesture.

[0073] Step S320: In response to the target gesture, if the target gesture is a static gesture, obtain the current page type of the device.

[0074] In this embodiment, if the target gesture is a static gesture, the control operation corresponding to the static gesture may differ depending on the current page type of the device. Therefore, to avoid accidental operation, in response to the target gesture and upon determining that the target gesture is a static gesture, the current page type of the device can be obtained. This page type can be any page of the electronic device.

[0075] Different pages can be configured with corresponding page identifiers. For example, the page identifier of application A can be configured as "a (this is just an example and does not represent the actual identifier)," the page identifier of application B can be configured as "b," and the page identifier of application function A1 in application A can be configured as "a1." In this way, the current page type of the device can be obtained by identifying the page identifier of the page the device is currently on.

[0076] Step S331: If the page type is a specified application page, the first control operation is used as the control operation corresponding to the static gesture.

[0077] Step S341: Execute the first control operation.

[0078] The specified application page can be understood as the page for a specified application (the specific type of the specified application is not limited; for example, the specified application can be a multimedia application such as audio or video).

[0079] In one implementation, if the current page type of the device is determined to be a specified application page, the first control operation can be used as the control operation corresponding to the static gesture described above in this embodiment. The first control operation is used to control the activation (launch) of functions within the specified application.

[0080] For example, in a specific application scenario, assuming the specified application page is the camera application interface, and the target gesture is the "V" gesture (i.e., a static gesture), if the current page of the device is the camera application page, then the control operation corresponding to the "V" gesture can be to control the camera application to take a picture.

[0081] Step S332: If the page type is not a specified application page, the second control operation is used as the control operation corresponding to the static gesture, and the first control operation and the second control operation are different control operations.

[0082] Step S342: Execute the second control operation.

[0083] In another implementation, if the current page type of the device is not the specified application page, the second control operation can be used as the control operation corresponding to the static gesture described in this embodiment. The second control operation is used to control the opening of the specified application, and it is different from the first control operation. In this embodiment, responding to different current page types of the device, even the same gesture will trigger different device state control operations.

[0084] For example, continuing with the above example, if the target gesture is the "V" gesture (i.e., a static gesture), and the current page of the device is not the camera application page, then the control operation corresponding to the "V" gesture can be to control the opening of the camera application.

[0085] As one approach, if the current page type of the device is not the specified application page, and the duration of the static gesture exceeds a first preset duration, then the aforementioned second control operation and first control operation can be used as the control operations corresponding to the static gesture; or if the current page type of the device is not the specified application page, and the static gesture occurs twice, with each occurrence lasting for a duration exceeding the first preset duration, then the aforementioned second control operation and first control operation can be used as the control operations corresponding to the static gesture. The second control operation is used to open the specified application, and the first control operation is used to activate functions within the specified application.

[0086] For example, continuing with the above example, if the target gesture is a "V" sign (i.e., a static gesture), and if the current page the device is on is not the camera application page, and the duration of the "V" sign is longer than the first preset duration, then it can be considered that two control operations corresponding to the "V" sign are triggered. It should be noted that the two control operations corresponding to the "V" sign in this case are the second control operation and the first control operation, respectively. Similarly, if the target gesture is a "V" sign (i.e., a static gesture), and the current page the device is on is not the camera application page, and the "V" sign appears twice (it can appear twice consecutively at a certain interval), and the duration of each "V" sign appearance reaches the first preset duration, then it can be considered that two control operations corresponding to the "V" sign are triggered. The two control operations corresponding to the "V" sign in this case are the second control operation and the first control operation, respectively.

[0087] As one implementation, if the second control operation and the first control operation are taken as control operations corresponding to static gestures, the second control operation can be executed first, followed by the first control operation. That is, in the example above, if the target gesture is a "V" gesture (i.e., a static gesture), and if the current page of the device is not the camera application page, when two "V" gestures occur and the duration of each "V" gesture reaches the first preset duration, the operation to open the camera application corresponding to the "V" gesture can be executed first, followed by the operation to open the camera function within the camera application (i.e., triggering the photo).

[0088] The device control method provided in this embodiment acquires a target gesture; then, in response to the target gesture, if the target gesture is a static gesture, it acquires the current page type of the device; if the page type is a specified application page, it uses a first control operation as the control operation corresponding to the static gesture; if the page type is not a specified application page, it uses a second control operation as the control operation corresponding to the static gesture, wherein the first control operation and the second control operation are different control operations; and then executes the control operation. Thus, the above method enables the adjustment of the device's state through gesture control, making gesture control of the device more convenient, improving the device's intelligence, and enhancing the user experience. Simultaneously, by determining whether the current page type of the device is a specified application page, and then determining the control operation corresponding to the target gesture based on the determination result—that is, if the page type is a specified application page, the first control operation is used as the control operation corresponding to the static gesture; if the page type is not a specified application page, the second control operation is used as the control operation corresponding to the static gesture—more accurate device control through gestures can be achieved.

[0089] Please see Figure 12 The diagram illustrates a flowchart of a device control method according to another embodiment of this application. This embodiment provides a device control method applicable to electronic devices, and the method includes:

[0090] Step S410: Obtain the target gesture.

[0091] Step S420: In response to the target gesture, obtain the control operation corresponding to the target gesture, the control operation being used to control and adjust the corresponding state of the electronic device.

[0092] Step S430: If the target gesture is a dynamic gesture, obtain the movement parameters of the dynamic gesture.

[0093] In this embodiment, the movement parameters may include at least one of the following: movement direction, movement frequency, movement amplitude, movement distance, and movement angle of the dynamic gesture.

[0094] As one approach, when the target gesture is determined to be a dynamic gesture, the movement parameters of the dynamic gesture can be obtained. Specifically, gesture recognition technology and corresponding functional sensors can be used to obtain the movement parameters of the dynamic gesture. For example, gesture recognition technology and a distance sensor can be used to obtain the movement distance of the dynamic gesture, and gesture recognition technology and an infrared sensor can be used to obtain the movement direction of the dynamic gesture. The implementation principle and process of obtaining movement parameters will not be elaborated here.

[0095] Step S440: Determine the adjustment amount for the state based on the movement parameters.

[0096] Different movement parameters for dynamic gestures can result in different adjustment amounts.

[0097] In one implementation, under the same interface, different movement parameters of the same dynamic gesture can correspond to different adjustment amounts. For example, when the interface of the electronic device is a video or audio playback interface, the adjustment amount corresponding to the movement direction of the dynamic gesture "palm" can be used to adjust the playback time progress, and the adjustment amount corresponding to the movement frequency of the "palm" gesture can be used to adjust the playback speed.

[0098] One approach is to obtain the change in movement parameters before and after a dynamic gesture, and then determine the adjustment amount for the corresponding device state based on this change. For example, when the movement parameter is the movement distance, the movement distance before and after the dynamic gesture can be obtained, and then, based on the ratio between this movement distance and the video playback duration, it can be determined how much to speed up or slow down the video playback duration (i.e., the adjustment amount). The ratio between the movement distance of the dynamic gesture and the adjustment amount of the video playback duration can be set according to actual needs, and will not be elaborated upon here.

[0099] Step S450: Adjust the state based on the adjustment amount.

[0100] In one implementation, such as Figure 13 As shown, electronic device 100 is playing a video with a total playback time of 52 minutes and 30 seconds. At 6 minutes and 6 seconds into the video, the screen displays the following: Figure 13 The image above shows screen 101. At this time, if the user makes a dynamic hand gesture "move palm" gesture 12 towards the camera of the electronic device, the direction of movement is... Figure 13 The direction of movement is indicated by the dotted line in the upper image. When the palm moves to... Figure 13 When the position shown in the lower image is (i.e.) Figure 13The "move palm" gesture (position 12) shown in the lower image can be used to adjust the playback time of the video from 6 minutes and 6 seconds to 38 minutes and 2 seconds based on the distance the palm moves. At this time, the video screen changes as follows. Figure 13 The lower image shows screen 101. The "moving palm" gesture 12 is a non-linear movement, and all dynamic gestures in this embodiment can be non-linear movements. By obtaining the movement distance of the "moving palm" gesture, the video playback time can be adjusted based on this distance without requiring manual touch control by the user, thus improving the intelligence of device control.

[0101] In one implementation, such as Figure 14 As shown, in Figure 13 Based on the content shown in the upper image, volume bar 103 is the volume bar of electronic device 100. If the user makes a dynamic gesture towards the camera of the electronic device, such as a "moving fist" gesture 12, the direction of movement is... Figure 14 The direction of movement is indicated by the dotted line in the upper image. When the palm moves to... Figure 14 When the position shown in the lower image is reached, the volume of the electronic device can be controlled based on the distance the palm moves. Figure 14 The volume shown in the upper image is adjusted to 105. Figure 14 The volume is 105 as shown in the lower image.

[0102] In another implementation, such as Figure 15 As shown, and Figure 14 The difference is, Figure 15 In the scenario shown, when the palm is... Figure 15 Position 12 shown in the upper image, moved to Figure 15 When at position 12 as shown in the lower image, the playback volume of the video can be controlled based on the distance the palm moves. Figure 15 The volume shown in the upper image is adjusted to 105. Figure 15 The volume is 105 as shown in the lower image.

[0103] In this embodiment, the orientation of the volume bar of the electronic device and the volume bar of the playback software is not limited, and the orientation of the volume bar can be adapted to the movement direction of the dynamic gesture. For example, if the orientation of the volume bar (or brightness bar) is horizontal, the volume (brightness) status can be adjusted by moving the dynamic gesture left and right; if the orientation of the volume bar (or brightness bar) is vertical, the volume (brightness) status can be adjusted by moving the dynamic gesture up and down.

[0104] In one implementation, on the same video (or audio) playback interface, different directions of movement of dynamic gestures can correspond to different adjustment amounts. For example, as Figure 16 As shown, in Figure 13 Based on the content shown in the upper image, if the movement direction of the dynamic gesture "move palm" gesture 12 is... Figure 13 If the direction indicated by the dotted line in the upper image is such that the adjustment amount corresponding to the "moving palm" gesture can be the playback time progress of the video (or audio), then the direction of movement of the dynamic gesture "moving palm" gesture 12 is... Figure 16 If the direction indicated by the dotted line in the upper image is the direction, then the adjustment amount corresponding to the "moving palm" gesture is the volume of the electronic device (or video (or audio)).

[0105] In one implementation, such as Figure 17 As shown, electronic device 100 is playing a video with a total playback time of 52 minutes and 30 seconds. At 6 minutes and 6 seconds into the video, the screen displays the following: Figure 17 The image above shows screen 101. At this time, if the user makes a dynamic "moving rock gesture" 12 towards the camera of the electronic device, the direction of movement is... Figure 17 The direction of movement is indicated by the dotted line in the upper image. When the palm moves to... Figure 17 When the position shown in the lower image is (i.e.) Figure 17 As shown in the lower image (at position 12 of the "moving rock gesture"), the brightness of the video frame 101 can be controlled based on the distance the rock gesture moves. Figure 17 The brightness of the upper image in the middle is adjusted to Figure 17 The brightness of the lower image in the image.

[0106] The device control method provided in this embodiment acquires a target gesture; then, in response to the target gesture, acquires a control operation corresponding to the target gesture, the control operation being used to control and adjust the corresponding state of the device; then, if the target gesture is a dynamic gesture, acquires the movement parameters of the dynamic gesture; determines the adjustment amount of the state based on the movement parameters; and adjusts the state based on the adjustment amount. Thus, the above method enables the adjustment of the device's corresponding state through gesture control, making gesture control of the device more convenient, improving the device's intelligence, and enhancing the user experience. Simultaneously, by determining the movement parameters of the dynamic gesture and then adjusting the corresponding state of the device based on the adjustment amount determined by the movement parameters, the accuracy and reliability of the device's state adjustment are improved.

[0107] Please see Figure 18The diagram illustrates a flowchart of a device control method according to another embodiment of this application. This embodiment provides a device control method applicable to electronic devices, and the method includes:

[0108] Step S510: Obtain the target gesture.

[0109] Step S520: In response to the target gesture, obtain the control operation corresponding to the target gesture, the control operation being used to control and adjust the state of the electronic device.

[0110] Step S531: If the target gesture is a dynamic gesture and the dynamic gesture is a low-frequency gesture, then execute the control operation.

[0111] In this embodiment, dynamic gestures can include low-frequency gestures and high-frequency gestures. Low-frequency gestures represent gestures that occur at a frequency below a frequency threshold within a specified period, while high-frequency gestures represent gestures that occur at a frequency above a frequency threshold within a specified period. The specific value of the frequency threshold is not limited, and the specified period can be a week, a month, or other durations, with no specific value required. High-frequency gestures can be understood as gestures commonly used by the user or by consumers of all electronic devices. For example, gestures like OK, raising a hand, and palm gestures are commonly used by the user or by all users, while gestures like the scissor gesture, fist gesture, and rock gesture are gestures that are not commonly used by the user or by all users.

[0112] It is understandable that the dynamic gestures made by the user can be gestures used to control the state of electronic devices or gestures made during conversation. To avoid misidentifying gestures made during conversation as gestures used to control the state of electronic devices, as a method, if the target gesture is a dynamic gesture, and the dynamic gesture is a low-frequency gesture, then the control operation corresponding to that dynamic gesture can be directly executed.

[0113] Step S532: If the target gesture is a dynamic gesture and the dynamic gesture is a high-frequency gesture, and the dynamic gesture continues to move after the holding time before the movement reaches the second preset time, then the control operation is executed; if the holding time before the movement does not reach the second preset time, then the control operation is not executed.

[0114] As another approach, if the target gesture is a dynamic gesture and the dynamic gesture is a high-frequency gesture, in one implementation, if the dynamic gesture continues to move after the holding time before moving reaches a second preset time, the control operation corresponding to the dynamic gesture can be executed; while in another implementation, if the target gesture is a dynamic gesture and the dynamic gesture is a high-frequency gesture, if the holding time before moving does not reach the second preset time, the control operation corresponding to the dynamic gesture will not be executed.

[0115] In other words, if the dynamic gesture is determined to be a high-frequency gesture, the control operation corresponding to the dynamic gesture will only be executed if the dynamic gesture appears and continues to move after a second preset duration. If the dynamic gesture appears but does not continue to move after the second preset duration, it means that the dynamic gesture may be a gesture during the user's conversation. In this case, not executing the control operation corresponding to the dynamic gesture can avoid misidentification and misoperation.

[0116] In some embodiments of this application, the values ​​of the first preset duration and the second preset duration may be the same; for example, both the first preset duration and the second preset duration may be 2 seconds. In other embodiments, the values ​​of the first preset duration and the second preset duration may be different, which is not limited here.

[0117] The device control method provided in this embodiment acquires a target gesture; then, in response to the target gesture, acquires a control operation corresponding to the target gesture, the control operation being used to control and adjust the corresponding state of the electronic device; then, if the target gesture is a dynamic gesture and the dynamic gesture is a low-frequency gesture, the control operation is executed; if the target gesture is a dynamic gesture and the dynamic gesture is a high-frequency gesture, the control operation is executed if the dynamic gesture continues to move after the holding time before moving reaches a second preset time; if the holding time before moving does not reach the second preset time, the control operation is not executed. Thus, the above method enables the adjustment of the device's corresponding state through gesture control, making gesture control of the device more convenient and improving the user experience. Simultaneously, by determining whether the dynamic gesture is a high-frequency or low-frequency gesture, and then deciding whether to execute the corresponding control operation in a timely manner based on the determination result, misidentification and misoperation can be avoided, thereby improving the accuracy and reliability of gesture control of the device.

[0118] Please see Figure 19 This is a structural block diagram of a device control apparatus provided in an embodiment of this application. This embodiment provides a device control apparatus 600 that can operate on an electronic device. The apparatus 600 includes: a gesture acquisition module 610, an operation acquisition module 620, and a device control module 630.

[0119] The gesture acquisition module 610 is used to acquire the target gesture.

[0120] In this embodiment, the electronic device is equipped with an image acquisition device or is connected to an image acquisition device. As one approach, the gesture acquisition module 610 can specifically be used to acquire gestures within a specified shooting range of the image acquisition device; and to select a preset gesture whose morphological similarity to the gesture is greater than a preset threshold as the target gesture.

[0121] The operation acquisition module 620 is used to acquire a control operation corresponding to the target gesture in response to the target gesture, and the control operation is used to control and adjust the corresponding state of the electronic device.

[0122] Wherein, if the target gesture is a static gesture, the state includes at least one of the following: the device being turned on or off, the hardware devices in the device being turned on or off, the application configured in the device being turned on or off, the application functions configured in the device being turned on or off, and executing the application functions configured in the device; if the target gesture is a dynamic gesture, the state includes at least one of the following: volume state, image quality state, and playback state.

[0123] In this embodiment, the static gesture and the dynamic gesture can have the same form. In this way, when the static gesture moves, the static gesture is regarded as the dynamic gesture.

[0124] In one approach, the operation acquisition module 620 can be specifically used to: if the target gesture is a static gesture, acquire the current page type of the device; if the page type is a specified application page, use the first control operation as the control operation corresponding to the static gesture; if the page type is not a specified application page, use the second control operation as the control operation corresponding to the static gesture, wherein the first control operation and the second control operation are different control operations.

[0125] Optionally, the operation acquisition module 620 can also be used to: if the page type is not a specified application page and the duration of the static gesture is greater than a first preset duration, use the second control operation and the first control operation as control operations corresponding to the static gesture; or if the page type is not a specified application page and the static gesture appears twice with each appearance lasting for a duration equal to the first preset duration, use the second control operation and the first control operation as control operations corresponding to the static gesture. The second control operation is used to open the specified application, and the first control operation is used to activate functions within the specified application. It should be noted that, in this method, executing the control operation may include: if the second control operation and the first control operation are used as control operations corresponding to the static gesture, executing the second control operation first, and then executing the first control operation.

[0126] The device control module 630 is used to perform the control operation.

[0127] In one embodiment, the device 600 may further include a duration determination module, which, before executing the control operation, if the target gesture is a static gesture, obtains the duration of the static gesture; if the duration reaches a first preset duration, executes the control operation; if the duration does not reach the first preset duration, does not execute the control operation.

[0128] In one manner, the device control module 630 can also be used to execute the control operation if the target gesture is a dynamic gesture and the dynamic gesture is a low-frequency gesture; if the target gesture is a dynamic gesture and the dynamic gesture is a high-frequency gesture, and the dynamic gesture continues to move after the holding time before the movement reaches a second preset time, then the control operation is executed; if the holding time before the movement of the dynamic gesture does not reach the second preset time, then the control operation is not executed.

[0129] In one approach, the device control module 630 can be used to, if the target gesture is a dynamic gesture, acquire the movement parameters of the dynamic gesture; determine the adjustment amount of the state based on the movement parameters; and adjust the state based on the adjustment amount.

[0130] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described device and module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0131] In the several embodiments provided in this application, the coupling between modules can be electrical, mechanical, or other forms of coupling.

[0132] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0133] Please see Figure 20 Based on the aforementioned device control method and apparatus, this application also provides an electronic device 100 capable of executing the aforementioned device control method. The electronic device 100 includes a memory 102 and one or more (only one shown in the figure) processors 104 coupled to each other, with a communication line connecting the memory 102 and the processors 104. The memory 102 stores a program capable of executing the contents of the aforementioned embodiments, and the processors 104 can execute the program stored in the memory 102.

[0134] The processor 104 may include one or more processing cores. The processor 104 connects to various parts within the electronic device 100 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 102, and by calling data stored in the memory 102. Optionally, the processor 104 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 104 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 104 and may be implemented separately using a communication chip.

[0135] The memory 102 may include random access memory (RAM) or read-only memory (ROM). The memory 102 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 102 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the aforementioned embodiments. The data storage area may also store data created by the electronic device 100 during use (such as phonebook data, audio and video data, chat log data, etc.).

[0136] Please refer to Figure 21 This diagram illustrates a structural block diagram of a computer-readable storage medium provided in an embodiment of this application. The computer-readable medium 700 stores program code that can be called by a processor to execute the methods described in the above method embodiments.

[0137] The computer-readable storage medium 700 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 700 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 700 has storage space for program code 710 that performs any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code 710 may, for example, be compressed in a suitable form.

[0138] In summary, the device control method, apparatus, electronic device, and storage medium provided in this application acquire a target gesture; then, in response to the target gesture, acquire a control operation corresponding to the target gesture, the control operation being used to control and adjust the corresponding state of the electronic device; and then execute the control operation. Thus, the above method enables the adjustment of the device's corresponding state through gesture control, making gesture control of the device more convenient, improving the device's intelligence, and enhancing the user experience.

[0139] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A device control method, applied to electronic equipment, characterized in that, The method includes: Acquire the target gesture; In response to the target gesture, a control operation corresponding to the target gesture is obtained, and the control operation is used to control and adjust the state of the electronic device. If the target gesture is a dynamic gesture, and the dynamic gesture is a low-frequency gesture, then the control operation is executed; If the target gesture is a dynamic gesture, and the dynamic gesture is a high-frequency gesture, the control operation is executed if the dynamic gesture continues to move after the holding time before moving reaches the second preset time; otherwise, the control operation is not executed.

2. The method according to claim 1, characterized in that, If the target gesture is a static gesture, the state includes at least one of the following: the opening and closing of the electronic device, the opening and closing of the hardware devices in the device, the opening and closing of the application configured in the electronic device, the opening and closing of the application function configured in the device, and the execution of the application function configured in the device. If the target gesture is a dynamic gesture, the state includes at least one of volume state, image quality state, and playback state.

3. The method according to claim 2, characterized in that, The method further includes: If the target gesture is a static gesture, obtain the duration of the static gesture; If the duration of holding reaches the first preset duration, the control operation is executed; If the duration of holding the device does not reach the first preset duration, the control operation will not be performed.

4. The method according to claim 2, characterized in that, The step of acquiring the control operation corresponding to the target gesture includes: If the target gesture is a static gesture, obtain the current page type of the device; If the page type is a specified application page, the first control operation will be used as the control operation corresponding to the static gesture; If the page type is not a specified application page, the second control operation will be used as the control operation corresponding to the static gesture, and the first control operation and the second control operation will be different control operations.

5. The method according to claim 4, characterized in that, The method further includes: If the page type is not a specified application page, and the duration of the static gesture is greater than a first preset duration, the second control operation and the first control operation are used as control operations corresponding to the static gesture. The second control operation is used to launch the specified application, and the first control operation is used to launch functions under the specified application; or If the page type is not a specified application page, and the static gesture appears twice with each appearance lasting for a duration equal to a first preset duration, then the second control operation and the first control operation will be used as the control operations corresponding to the static gesture.

6. The method according to claim 2, characterized in that, The execution of the control operation includes: If the target gesture is a dynamic gesture, obtain the movement parameters of the dynamic gesture; The adjustment amount to the state is determined based on the movement parameters; The state is adjusted based on the adjustment amount.

7. The method according to claim 2, characterized in that, The static gesture has the same shape as the dynamic gesture, and the static gesture is regarded as the dynamic gesture when the static gesture moves.

8. The method according to any one of claims 1-7, characterized in that, The electronic device is equipped with an image acquisition device, or is connected to an image acquisition device, and the acquisition of the target gesture includes: Capture gestures within a specified shooting range of the image acquisition device; A preset gesture whose shape similarity to the gesture is greater than a preset threshold is selected as the target gesture.

9. A device control apparatus, operating in electronic equipment, characterized in that, The device includes: The gesture acquisition module is used to acquire the target gesture; An operation acquisition module is used to acquire a control operation corresponding to the target gesture in response to the target gesture, and the control operation is used to control and adjust the corresponding state of the electronic device. The device control module is configured to execute the control operation if the target gesture is a dynamic gesture and the dynamic gesture is a low-frequency gesture; execute the control operation if the target gesture is a dynamic gesture and the dynamic gesture is a high-frequency gesture, and the gesture continues to move after the holding time before the dynamic gesture moves reaches a second preset time; and not execute the control operation if the holding time before the dynamic gesture moves does not reach the second preset time.

10. An electronic device, characterized in that, Includes one or more processors and memory; One or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs being configured to perform the method of any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code, wherein the program code, when executed by a processor, performs the method according to any one of claims 1-8.

Images