Device control methods, electronic devices and computer-readable storage media

Abstract

This application provides a device control method, an electronic device, and a computer-readable storage medium, relating to the field of electronic technology. The method includes: detecting a user's spatial pointing gesture and recording the device pointing order of multiple target devices pointed to by the spatial pointing gesture; and executing corresponding control operations according to the device pointing order. The device control method provided in this application can abstract gestures into specific devices, making them easier for users to remember and use. Furthermore, spatial pointing gestures are relatively easy to design and can support a large number of gestures. In addition, users can use devices in space as coordinate points to implement spatial pointing gestures, thus providing a basis for the gestures and further improving user convenience.

Description

Technical Field

[0001] This application relates to the field of electronic technology, and more particularly to a device control method, electronic device, and computer-readable storage medium. Background Technology

[0002] With the rapid development of electronic technology, more and more intelligent electronic devices are entering people's lives, making their lives more convenient.

[0003] Many electronic devices, such as mobile phones and televisions, now have gesture control capabilities; users can issue commands through various pre-defined gestures to trigger corresponding functions or operations. However, current gestures are not easy for users to remember, thus affecting the convenience of using electronic devices. Summary of the Invention

[0004] In view of this, this application provides a device control method, an electronic device, and a computer-readable storage medium to improve the convenience of users using electronic devices.

[0005] To achieve the above objectives, in a first aspect, embodiments of this application provide a device control method, comprising:

[0006] Detect the user's spatial pointing gesture and record the device pointing order of multiple target devices pointed to by the spatial pointing gesture;

[0007] Execute the corresponding control operations according to the device pointing sequence.

[0008] The device control method provided in this application executes corresponding control operations according to the device pointing order of the target device pointed to by the user's spatial pointing gesture. This abstracts the gesture into a specific device, making it easier for the user to remember and use. Moreover, spatial pointing gestures are relatively easy to design and can support a large number of gestures. In addition, the user can use devices in space as coordinate points to realize spatial pointing gestures, thereby giving the gestures a basis and further improving the convenience of use for the user.

[0009] In one possible implementation of the first aspect, the target devices involved in the device pointing order include a first device and at least one second device, wherein the pointing order of the first device precedes the pointing order of the second device, and the step of performing corresponding control operations according to the device pointing order includes:

[0010] For each second device in the device pointing sequence, control the first device and the second device to perform the corresponding function association operation.

[0011] Through the above implementation method, users can use spatial pointing gestures to achieve functional association control operations between devices.

[0012] In one possible implementation of the first aspect, controlling the first device and the second device to perform corresponding functional association operations includes:

[0013] If the second device is an audio device, control the second device to play the sound information from the first device;

[0014] When the second device is a display device, control the second device to display the target content of the first device, or execute and display the target application of the first device;

[0015] If the second device is a printing device, control the second device to print the target file of the first device.

[0016] In one possible implementation of the first aspect, the method further includes: upon detecting a user's cancellation operation, in response to the cancellation operation, controlling the first device and the second device corresponding to the cancellation operation to perform a function association cancellation operation. This allows the user to cancel the function association operation between the devices.

[0017] In one possible implementation of the first aspect, the cancellation operation is a first spatial pointing gesture, the target device to which the first spatial pointing gesture points includes a second device corresponding to the cancellation operation first and a first device pointed to later;

[0018] Alternatively, the cancellation operation is a combination of a second spatial pointing gesture and a target operation, wherein the second spatial pointing gesture includes a second device corresponding to the cancellation operation, and the target operation is a target cancellation gesture, a target touch operation, or a target button operation.

[0019] In one possible implementation of the first aspect, the execution of the corresponding control operations according to the device pointing sequence includes:

[0020] The third device is controlled to perform authorized operations according to the device pointing order.

[0021] Through the above implementation method, users can perform device access control operations through spatial pointing gestures.

[0022] In one possible implementation of the first aspect, controlling the third device to perform permission functions according to the device pointing order includes:

[0023] If the device pointing order is consistent with the preset device pointing order, control the third device to perform the interface unlocking operation;

[0024] Alternatively, if the device pointing order is consistent with the preset device pointing order, the third device can be controlled to turn on or off.

[0025] In one possible implementation of the first aspect, the device pointing order involves target devices including a fourth device and a fifth device, the fourth device including at least one, and the fifth device being the last target device to be pointed to. The execution of the corresponding control operation according to the device pointing order includes:

[0026] The fifth device controls the fourth device to perform corresponding status control operations.

[0027] Through the above implementation method, users can control the status of the device by using spatial pointing gestures.

[0028] In one possible implementation of the first aspect, controlling the fourth device to perform corresponding state control operations based on the fifth device includes:

[0029] When the fifth device is a switch, it controls the fourth device to turn on or off, or controls the fourth device to adjust brightness, volume, or temperature.

[0030] If the fifth device is a relay device, control the fourth device to connect to or refresh the network;

[0031] The direction of sound playback or shooting direction of the fourth device is controlled according to the position of the fifth device.

[0032] In one possible implementation of the first aspect, detecting the user's spatial pointing gesture includes:

[0033] When a user's spatial pointing gesture is detected, the system detects the user's spatial pointing gesture in response to the gesture. This improves the accuracy of spatial pointing gesture detection and saves processing resources.

[0034] In one possible implementation of the first aspect, the spatial pointing gesture triggering operation is a first target gesture, a touch operation on a target control, or a press operation on a target button.

[0035] In one possible implementation of the first aspect, the execution of the corresponding control operations according to the device pointing sequence includes:

[0036] When the user's spatial pointing gesture ends, in response to the end of the spatial pointing gesture, corresponding control operations are executed according to the device's pointing sequence. This improves the accuracy of spatial pointing gesture detection results.

[0037] In one possible implementation of the first aspect, the spatial pointing gesture end operation is a second target gesture, a touch release operation on a target control, or a button release operation on a target button.

[0038] In one possible implementation of the first aspect, prior to performing the corresponding control operations according to the device pointing sequence, the method further includes:

[0039] Detect and save the correspondence between the device pointing order set by the user and the control operation.

[0040] Through the above implementation methods, users can set their own spatial pointing gestures as needed, which can meet the needs of different users.

[0041] In one possible implementation of the first aspect, the spatial pointing gesture is a pointing operation performed by a handheld electronic device. This allows the user to perform spatial pointing gestures using the handheld electronic device without relying on a camera device, thereby enabling the user to perform more spatial pointing gestures and improving the accuracy of spatial pointing gesture detection results.

[0042] Secondly, embodiments of this application provide a device control apparatus, comprising:

[0043] The detection module is used to detect the user's spatial pointing gestures;

[0044] A recording module is used to record the device pointing order of the multiple target devices pointed to by the spatial pointing gesture;

[0045] The control module is used to execute corresponding control operations according to the order in which the devices are pointed.

[0046] In one possible implementation of the second aspect, the target devices involved in the device pointing order include a first device and at least one second device, wherein the pointing order of the first device precedes the pointing order of the second device, and the control module is specifically used for:

[0047] For each second device in the device pointing sequence, control the first device and the second device to perform the corresponding function association operation.

[0048] In one possible implementation of the second aspect, the control module is specifically used for:

[0049] If the second device is an audio device, control the second device to play the sound information from the first device;

[0050] When the second device is a display device, control the second device to display the target content of the first device, or execute and display the target application of the first device;

[0051] If the second device is a printing device, control the second device to print the target file of the first device.

[0052] In one possible implementation of the second aspect, the detection module is further configured to: detect a user's cancellation operation;

[0053] The control module is further configured to: when the detection module detects a user's cancellation operation, in response to the cancellation operation, control the first device and the second device corresponding to the cancellation operation to perform a function-associated cancellation operation.

[0054] In one possible implementation of the second aspect, the cancellation operation is a first spatial pointing gesture, the target device pointed to by the first spatial pointing gesture includes a second device corresponding to the cancellation operation first and a first device pointed to later;

[0055] Alternatively, the cancellation operation is a combination of a second spatial pointing gesture and a target operation, wherein the second spatial pointing gesture includes a second device corresponding to the cancellation operation, and the target operation is a target cancellation gesture, a target touch operation, or a target button operation.

[0056] In one possible implementation of the second aspect, the control module is specifically used to: control the third device to perform permission operations according to the device pointing order.

[0057] In one possible implementation of the second aspect, the control module is specifically used for:

[0058] If the device pointing order is consistent with the preset device pointing order, control the third device to perform the interface unlocking operation;

[0059] Alternatively, if the device pointing order is consistent with the preset device pointing order, the third device can be controlled to turn on or off.

[0060] In one possible implementation of the second aspect, the target devices involved in the device pointing sequence include a fourth device and a fifth device, the fourth device includes at least one, the fifth device is the last target device to be pointed to, and the control module is specifically used to: control the fourth device to perform corresponding state control operations based on the fifth device.

[0061] In one possible implementation of the second aspect, the control module is specifically used for:

[0062] When the fifth device is a switch, it controls the fourth device to turn on or off, or controls the fourth device to adjust brightness, volume, or temperature.

[0063] If the fifth device is a relay device, control the fourth device to connect to or refresh the network;

[0064] The direction of sound playback or shooting direction of the fourth device is controlled according to the position of the fifth device.

[0065] In one possible implementation of the second aspect, the detection module is specifically used for:

[0066] When a user's spatial pointing gesture is detected, the user's spatial pointing gesture is detected in response to the spatial pointing gesture triggering operation.

[0067] In one possible implementation of the second aspect, the spatial pointing gesture triggering operation is a first target gesture, a touch operation on a target control, or a press operation on a target button.

[0068] In one possible implementation of the second aspect, the control module is specifically configured to: when the detection module detects that the user's spatial pointing gesture has ended, in response to the spatial pointing gesture ending operation, execute corresponding control operations according to the device pointing sequence.

[0069] In one possible implementation of the second aspect, the spatial pointing gesture end operation is a second target gesture, a touch release operation on a target control, or a button release operation on a target button.

[0070] In one possible implementation of the second aspect, the detection module is further configured to: detect and save the correspondence between the device pointing order set by the user and the control operation before the control module executes the corresponding control operation according to the device pointing order.

[0071] In one possible implementation of the second aspect, the spatial pointing gesture is a pointing operation performed by a handheld electronic device.

[0072] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to execute the method described in the first aspect or any embodiment of the first aspect when the computer program is invoked.

[0073] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect or any embodiment of the first aspect.

[0074] Fifthly, embodiments of this application provide a computer program product that, when run on an electronic device, causes the electronic device to perform the method described in the first aspect or any embodiment of the first aspect.

[0075] Sixthly, embodiments of this application provide a chip system including a processor coupled to a memory. The processor executes a computer program stored in the memory to implement the method described in the first aspect or any embodiment thereof. The chip system may be a single chip or a chip module composed of multiple chips.

[0076] It is understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0077] Figure 1 A schematic diagram illustrating an application scenario provided in an embodiment of this application;

[0078] Figure 2 A flowchart illustrating the device control method provided in an embodiment of this application;

[0079] Figure 3 This is a schematic diagram illustrating another application scenario provided by an embodiment of this application;

[0080] Figures 4-6 This application provides further schematic diagrams illustrating various application scenarios.

[0081] Figure 7 This is a schematic diagram of the structure of the device control apparatus provided in the embodiments of this application;

[0082] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0083] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0084] First, we introduce an application scenario based on an embodiment of this application. Please refer to... Figure 1 , Figure 1 This is a schematic diagram illustrating an application scenario provided by an embodiment of this application. For example... Figure 1 As shown, the electronic devices involved in this application scenario (also referred to here as target devices) may include: computer 10, television 20, speaker 30, camera 40, air conditioner 50, router 60, printer 70, light 80 and switch 90, etc., and each electronic device may include one or more.

[0085] It is understood that the above application scenario is only an example. In specific applications, the application scenario may include more or fewer electronic devices than shown in the figure. For example, the application scenario may not involve printer 70. Or, the application scenario may also include refrigerators, water heaters, smart displays and / or environmental monitoring devices, etc.

[0086] In this embodiment, the user can regard each electronic device in the space as a physical coordinate point in the space. During use, the user can point to multiple target devices in the space in sequence to complete the spatial pointing gesture, and realize the corresponding control operation through the spatial pointing gesture.

[0087] Users can perform different spatial pointing gestures by pointing at different devices in different sequences, thereby achieving different control operations. For example, a user can point to computer 10 and TV 20 in sequence to transfer the content displayed on computer 10 to TV 20; a user can also point to TV 20, air conditioner 50 and switch 90 in sequence to turn TV 20 and air conditioner 50 on or off.

[0088] In practice, users can complete spatial pointing gestures with their bare hands or other handheld electronic devices (such as mobile phone 100, smartwatch, or remote control). That is, users can complete spatial pointing gestures with their bare hands or with an electronic device. The figure shows an example of a user completing a spatial pointing gesture by holding a mobile phone 100.

[0089] For situations where spatial pointing gestures are performed without hands, image recognition technology can be used to detect the target device pointed to by the user's spatial pointing gesture. For example, an independent camera 40 or a built-in or external camera 40 of other electronic devices can be used to capture user images. The camera 40 or the electronic device connected to the camera 40 can then identify the target device pointed to by the user's spatial pointing gesture in the user image and determine the device pointing order of the target device.

[0090] For spatial pointing gestures performed via other electronic devices, Ultra Wideband (UWB), Bluetooth, or other indoor positioning technologies can be used to detect the target device being pointed at by the user's gesture. For example, when using UWB technology, both the user's handheld electronic device and each target device can be equipped with a UWB chip. During the user's pointing gesture, each target device can receive the UWB signal sent by the user's handheld electronic device, determine the relevant signal angle and transmission position, and thus determine whether the electronic device is pointing at it. If it determines that it is being pointed at, it can send pointing information back to the electronic device; the electronic device can then determine the pointing order based on the received pointing information.

[0091] In this embodiment, any electronic device in the application scenario can detect the user's spatial pointing gesture and implement corresponding control operations based on the spatial pointing gesture. That is, when controlling the device, any target device in the application scenario and the electronic device held by the user can act as control devices to realize the detection of spatial pointing gestures and corresponding control operations.

[0092] The control device can detect spatial pointing gestures by combining information from other electronic devices. For example, as mentioned above, it can detect spatial pointing gestures based on user images captured by camera 40, or based on UWB or Bluetooth signals fed back by the target device being pointed at. When performing a control operation, if the control device is the controlled electronic device corresponding to the control operation, it can directly control itself to perform the relevant operation; if the control device is not the controlled electronic device corresponding to the control operation, it can send control commands to the controlled electronic devices to control them to perform the relevant operation; if multiple controlled electronic devices are involved, the control device can send control commands to at least one controlled electronic device to control it to perform the relevant operation. For example, a user can point to the TV 20 and then to the speaker 30 to control the speaker 30 to play the sound information from the TV 20. If the control device is the TV 20, it can directly send its own sound information to the speaker 30 for playback. If the control device is the user's mobile phone 100, the mobile phone 100 can send a control command to the TV 20 to control the TV 20 to send the sound information to the speaker 30 for playback. Alternatively, the mobile phone 100 can send a control command to the speaker 30 to control the speaker 30 to obtain the sound information from the TV 20 for playback. Or, the mobile phone 100 can send control commands to both the TV 20 and the speaker 30 to control the speaker 30 to play the sound information from the TV 20.

[0093] For ease of understanding and explanation, this application uses the detection and corresponding control operation of a user's spatial pointing gesture through a handheld electronic device as an example to illustrate the technical solution of this application. The relevant detection and control process of the electronic device will be described in detail below. Unless otherwise specified, the electronic device referred to is a handheld electronic device.

[0094] Please see Figure 2 , Figure 2 This is a flowchart illustrating the device control method provided in an embodiment of this application, as shown below. Figure 2 As shown, the method may include the following steps:

[0095] S110, Detect the user's spatial pointing gesture trigger operation.

[0096] In this embodiment, the user's spatial pointing gesture trigger operation can be detected before detecting the user's spatial pointing gesture to determine whether the user has entered the spatial pointing gesture state, so as to reduce false detections, improve the accuracy of spatial pointing gesture detection results, and save processing resources.

[0097] Specifically, the spatial pointing gesture trigger operation can be a target gesture (referred to here as the first target gesture), such as a raise gesture, meaning the electronic device can start detecting spatial pointing gestures after detecting the user's raise gesture; the spatial pointing gesture trigger operation can also be a user's touch operation on a target control (referred to here as the first target control) on the electronic device, meaning the electronic device can provide a control to indicate the start of a spatial pointing gesture, and the user can trigger the electronic device to start detecting spatial pointing gestures by touching this control; the spatial pointing gesture trigger operation can also be a user's press operation on a target button (referred to here as the first target button), meaning the electronic device (such as a remote control) can provide a mechanical button to indicate the start of a spatial pointing gesture, and the user can trigger the electronic device to start detecting spatial pointing gestures by pressing this mechanical button.

[0098] S120. In response to the detected spatial pointing gesture trigger operation, detect the user's spatial pointing gesture and record the device pointing order of multiple target devices pointed to by the spatial pointing gesture.

[0099] If an electronic device detects a spatial pointing gesture, it means that the user has entered a control pointing gesture state. At this time, the aforementioned image recognition technology or indoor positioning technology can be used to detect the user's spatial pointing gesture, identify the target device pointed to by the spatial pointing gesture, and record the device pointing order of each target device pointed to by the spatial pointing gesture during the detection process.

[0100] In this context, each target device in the device pointing sequence can be represented by a corresponding device identifier. This device identifier can be a unique identification code of the target device, or a network address of the target device, or other identification information that can be used to identify the target device. Electronic devices can identify the target device based on this device identifier.

[0101] S130, Detect the user's spatial pointing gesture to end the operation.

[0102] During the detection of spatial pointing gestures, electronic devices can detect when the user's spatial pointing gesture ends, thus determining whether the user has ended the spatial pointing gesture state and making the spatial pointing gesture detection result more accurate.

[0103] Similar to the spatial pointing gesture trigger operation, the spatial pointing gesture end operation can be a target gesture (referred to here as the second target gesture), such as a user's hand-lowering gesture after raising their hand. That is, the electronic device can determine the end of the spatial pointing gesture after detecting the user's hand-lowering action. The spatial pointing gesture end operation can also be a user's touch operation on a target control (referred to here as the second target control) on the electronic device. That is, the electronic device can provide a control to indicate the end of the spatial pointing gesture, and the user can trigger the electronic device to stop detecting the spatial pointing gesture by touching this control. The spatial pointing gesture end operation can also be a user's pressing operation on a target button (referred to here as the second target button). That is, the electronic device (such as a remote control) can provide a mechanical button to indicate the end of the spatial pointing gesture, and the user can trigger the electronic device to start detecting the spatial pointing gesture by pressing this mechanical button. The first target control and the second target control can be the same control or different controls; the first target button and the second target button can also be the same button or different buttons.

[0104] In this embodiment, the user can also maintain the first touch operation or the first press operation during the spatial pointing gesture, and end the first touch operation or the first press operation after the spatial pointing gesture ends. That is, when a spatial pointing gesture is needed, the user presses and holds the first target control or the first target button, and releases the first target control or the first target button after the spatial pointing gesture ends. In other words, the spatial pointing gesture end operation can also be a touch release operation of the first target control or a button release operation of the first target button.

[0105] S140. In response to the detected spatial pointing gesture ending operation, execute the corresponding control operation according to the recorded device pointing sequence.

[0106] If an electronic device detects that a spatial pointing gesture has ended, it means that the user's spatial pointing gesture has ended. At this point, the electronic device can be controlled to perform the corresponding function based on the recorded device pointing sequence.

[0107] The above control operations are explained below using several implementation scenarios.

[0108] Scenario 1: Perform functional association operations between devices according to the device pointing order.

[0109] In this scenario, the target devices involved in the device pointing order can include a first device and a second device. The pointing order of the first device is earlier than that of the second device. When performing control operations, the first device and the second device can be controlled to perform corresponding functional association operations.

[0110] Specifically, if the second device is an audio device, it can be used as the sound playback device for the first device, meaning the second device can be controlled to play the sound information from the first device. For example, if a user points to the TV (the first device) and then to the speaker (the second device), the speaker can be used as the TV's sound playback device; that is, any sound information the TV needs to play can be sent to the speaker for playback.

[0111] If the second device is a display device, it can be used as a display screen or extended display screen for the first device to display the target content of the first device. For example, if a user points to a camera (first device) and then to a television (second device), the television can be used as a display screen for the camera, displaying the image captured by the camera. Similarly, if a user points to a computer (first device) and then to a television (second device), the television can be used as an extended display screen for the computer, displaying all or part of the computer's display interface.

[0112] When the second device is a display device, applications currently running on the first device can also be copied or transferred to the second device for execution and display. For example, if a user points to a computer (first device) and then to a television (second device), an application currently running on the computer (such as video playback application A) can be transferred to the television for execution. The television may have video playback application A pre-installed. The computer can send the video information to the television, controlling the television to open video playback application A and play the target video based on the received video information. As another example, if a user points to mobile phone 1 (first device) and then to mobile phone 2 (second device), an application currently running on mobile phone 1 (such as video playback application B) can be copied to mobile phone 2 for execution. That is, both mobile phone 1 and mobile phone 2 will run video playback application B to play the video currently playing on mobile phone 1.

[0113] It should be noted that target devices such as computers, mobile phones, and televisions are both audio and display devices. In practical implementation, they can perform function association operations related to audio and / or display functions. For example, if the user points to the computer (first device) and then to the television (second device), the application currently running on the computer (such as video playback application A) can be transferred to the television for execution. When the television is running video playback application A and playing the target video, it simultaneously performs function association operations related to audio and display functions.

[0114] If the second device is a printing device, it can be controlled to print the file currently displayed on the first device (i.e., the target file). This target file can be an image, a text document, or the current frame of a video. For example, if the user points to the computer (the first device) and then to the printer (the second device), the printer can be controlled to print the text document displayed on the computer.

[0115] In this embodiment, a user's spatial pointing gesture can point to two devices as described above, or it can point to more than two devices; that is, there can be multiple second devices. In specific implementation, for each second device in the device pointing sequence, the first device and that second device can be controlled to perform corresponding functional association operations.

[0116] Figure 3 This is a schematic diagram illustrating another application scenario provided by an embodiment of this application, such as... Figure 3 As shown in (a), when a user makes a spatial pointing gesture, pointing sequentially at computer 10, TV 20, and speaker 30, the gesture will work as shown in (a). Figure 3 As shown in (b), the television 20 is used as the display screen of the computer 10, and the television 20 displays the picture A on the computer 10. At the same time, the speaker 30 is used as the sound playback device of the computer 10, and the speaker 30 plays the sound information of the computer 10.

[0117] In this embodiment, the correspondence between device pointing order and function-associated operations can include a system-preset one or a user-set one. That is, the electronic device can detect and save the user-set correspondence between device pointing order and function-associated operations. For example, the user can pre-set the correspondence between device pointing order and function-associated operations on their mobile phone and can synchronize this correspondence to other electronic devices. When the user makes a spatial pointing gesture, the electronic device can execute the corresponding control operation when the device pointing order corresponding to the user's spatial pointing gesture matches the device pointing order in the preset correspondence.

[0118] For example, a user can set the following mapping relationships: Relationship 1: Pointing from the computer to the TV, the TV is used as an extended display for the computer, showing part of the computer's interface; Relationship 2: Pointing from the mobile phone to the computer, the computer is used as an extended display for the mobile phone, synchronously showing the entire interface displayed on the mobile phone; Relationship 3: Pointing from the mobile phone, computer, or TV to the speaker, the speaker is used as the sound playback device for the mobile phone, computer, or TV.

[0119] When a user performs a spatial pointing gesture, if the user points to a mobile phone, a computer, and a speaker in sequence during a single spatial pointing gesture, then corresponding relationships 2 and 3 can be matched. Based on these two relationships, the mobile phone's display interface can be displayed synchronously on the computer, and the mobile phone's sound information can be played through the speaker.

[0120] The above is just an example. In actual implementation, some correspondences can be preset as needed. Users can also set personalized correspondences as needed and can change the system's preset correspondences. This embodiment does not make any special limitations on this.

[0121] To facilitate user operation, in this embodiment, the electronic device can also, in response to the user's cancellation operation, control the first device and the second device corresponding to the cancellation operation to perform a function-associated cancellation operation.

[0122] The cancellation operation can be a spatial pointing gesture (referred to here as the first spatial pointing gesture), and the target device pointed to by the first spatial pointing gesture includes the second device corresponding to the cancellation operation first and the first device pointed to later. Continuing with the above... Figure 3 For example, if a user points to the computer, TV, and speaker in sequence, and wants to cancel the function association between the computer and TV, they can do so as follows: Figure 4 As shown in (a), pointing to TV 20 and then to computer 10, then as shown in (a), Figure 4 As shown in (b), the television 20 no longer serves as the display screen for the computer 10, and the computer 10 continues to display picture A.

[0123] Cancellation can also be a combination of spatial pointing gesture (referred to here as second spatial pointing gesture) and target operation, wherein the second spatial pointing gesture includes the second device corresponding to the cancellation operation.

[0124] The target action can be a target cancellation gesture, meaning the cancellation action can be a combination of a second-space pointing gesture and a target cancellation gesture. For example: when a user cancels... Figure 4 When performing a function association operation between the computer and the TV, you can also point to the TV and then perform a target cancellation gesture, such as a fist gesture, to cancel the function association between the TV and the computer. The target cancellation gesture can be set as needed, and this embodiment does not impose any special limitations on it.

[0125] The target operation can also be a target touch operation; that is, the cancel operation can be a combination of a second-space pointing gesture and a target touch operation. For example, when a user cancels... Figure 4 When performing a function association operation between a computer and a TV, the user can point to the TV to end the spatial pointing gesture. After the electronic device detects that the user is pointing to the TV in a function association state (i.e., as a second device, performing a function association operation with the first device), it can display an option to cancel the function association operation. The user can select this option to cancel the function association relationship between the TV and the computer.

[0126] The target action can also be a target button action; that is, the cancel action can be a combination of a second-space pointing gesture and a target button action. For example: when a user cancels... Figure 4 When performing a function association operation between a computer and a TV, you can point to the TV and then press the cancel button on the electronic device (such as a remote control) to cancel the function association between the TV and the computer.

[0127] The above examples illustrate several possible implementations of the cancellation operation. In actual implementation, the cancellation operation can also be other operations, and this embodiment does not impose any particular limitations on it.

[0128] Scenario 2: Control the third device to perform permission operations based on the device pointing order.

[0129] In this scenario, users can use the device pointing sequence as a password to unlock a third device, enabling it to perform corresponding authorized operations. The correspondence between the device pointing sequence and the authorized operations can be preset by the user.

[0130] Specifically, spatial pointing gestures can be used as passwords to unlock the account login interface. That is, electronic devices can compare the device pointing sequence corresponding to the spatial pointing gesture with the preset device pointing sequence. If the two match, the third device can be controlled to unlock the interface.

[0131] For example, users can pre-set a mapping between the order in which devices are pointed and the corresponding permission operations: pointing sequentially at the camera, air conditioner, TV, router, and printer decrypts the TV's account login interface. Then, when performing spatial pointing gestures, users can... Figure 5 The space shown points to the account login interface for gesture-based TV decryption. Figure 5 This is a schematic diagram illustrating another application scenario provided in the embodiments of this application, such as... Figure 5As shown in (a), the TV displays the account login interface. The user points to camera 40, air conditioner 50, TV 20, router 60, and printer 70 in sequence. The electronic device can detect that the device pointing order matches the device pointing order in the user's preset correspondence, and then... Figure 5 As shown in (b), this unlocks the TV's account login interface. In practice, the electronic device can send the detected device pointing sequence to the TV, or it can directly send the corresponding login password to the TV, instructing the TV to unlock the account login interface.

[0132] In this embodiment, spatial pointing gestures can also be used as passwords to open or close one or more third devices. That is, electronic devices can also control the opening or closing of third devices when the device pointing order corresponding to the spatial pointing gesture is consistent with the preset device pointing order.

[0133] For example, a user can pre-set a mapping between the device pointing sequence and the corresponding permission operations: pointing sequentially at the camera, air conditioner, TV, and printer will turn on the air conditioner, speaker, and printer. When the user completes the spatial pointing gesture by pointing sequentially at the camera, air conditioner, TV, and printer, the electronic device can then turn on the air conditioner, speaker, and printer. In practice, turning on and off the third device can use different device pointing sequences or the same sequence. In this case, the electronic device can determine whether to turn on or off the third device based on its state. For example, when the user completes the spatial pointing gesture by pointing sequentially at the camera, air conditioner, TV, and printer, if the air conditioner, speaker, and printer are on, then they are off; if they are off, then they are on.

[0134] As an optional implementation, the system can also control the activation or deactivation of each target device involved in the device pointing sequence. That is, the device pointing sequence can be arbitrary, and the corresponding permission operation is to activate or deactivate each target device involved in the device pointing sequence. For example, if a user points to a camera, air conditioner, television, and printer in sequence, and completes the spatial pointing gesture, the electronic device can then activate the camera, air conditioner, television, and printer. Similarly, the electronic device can control the activation or deactivation of a third device based on its status.

[0135] Scenario 3: Control the state of the preceding devices based on the last target device pointed to in the device pointing sequence.

[0136] In this scenario, the target devices involved in the device pointing order may include a fourth device and a fifth device. The fourth device includes at least one device, and the fifth device is the last target device to be pointed to. When performing control operations, the fourth device can be controlled to perform corresponding state control operations based on the fifth device.

[0137] Specifically, if the fifth device is a switch, it can control the fourth device to turn on or off. Figure 6 This is a schematic diagram illustrating another application scenario provided in the embodiments of this application, such as... Figure 6 As shown in (a), the user points to the router 60, air conditioner 50, TV 20, light 80 (fourth device), and switch 90 (fifth device) in sequence. After completing the spatial pointing gesture, the user can then... Figure 6 As shown in (b), the router 60, air conditioner 50, TV 20 and light 80 are turned on.

[0138] In practical implementation, electronic devices can also control whether the fourth device is turned on or off based on its status. That is, after the user points to the router, air conditioner, TV, light and switch in sequence to complete the spatial pointing gesture, if these fourth devices are in the off state, the user can control them to turn on; if these fourth devices are in the on state, the user can control them to turn off.

[0139] If there are multiple switching devices, different switches can be set to represent on and off. For example, if a user points to the router, air conditioner, TV, light, and switch 1 in sequence and completes the spatial pointing gesture, the user can control the router, air conditioner, TV, and light to turn on; if a user points to the router, air conditioner, TV, light, and switch 2 in sequence and completes the spatial pointing gesture, the user can control the router, air conditioner, TV, and light to turn off.

[0140] In this embodiment, if the fifth device is a switch, it can also control the fourth device to adjust brightness, volume, or temperature. For example, a user can adjust the brightness of the light by pointing to the lamp and then to the switch; a user can adjust the volume of the speaker by pointing to the speaker and then to the switch; a user can adjust the temperature of the air conditioner by pointing to the air conditioner and then to the switch; a user can also adjust the brightness of the light, the volume of the speaker, and the temperature of the air conditioner simultaneously by pointing to the light, the speaker, and the air conditioner.

[0141] In the case of a single switchgear, the switchgear can be used to indicate an increase or decrease; in the case of multiple switchgear, different switches can be set to indicate an increase or decrease.

[0142] If the fifth device is a relay device, it can control the fourth device to connect to or refresh the network. The relay device can be a router or gateway, etc. For example, if a user points to the TV and then to the router, the TV can be controlled to connect to or refresh the network. In practice, the decision to connect to or refresh the network can also be determined based on the status of the fourth device. If the fourth device is not connected to the network, it can be controlled to connect; if it is already connected, it can be controlled to refresh the network.

[0143] In this embodiment, the sound playback direction or shooting direction of the fourth device can also be controlled according to the position of the fifth device. For example, if the user points to the speaker and then to the printer, the sound playback direction of the speaker can be adjusted to face the printer; if the user points to the camera and then to the computer, the shooting direction of the camera can be adjusted to face the computer.

[0144] Similar to Scenario 1, in this scenario, the correspondence between device pointing order and state control operations can include system presets or user-sets. That is, the electronic device can detect and save the user-set correspondence between device pointing order and state control operations. When the user performs a spatial pointing gesture, the electronic device can execute the corresponding control operation if the device pointing order corresponding to the user's spatial pointing gesture matches the device pointing order in the preset correspondence.

[0145] For example, the user sets the following correspondence: Correspondence 1: Point from device 1 (including at least one of light, TV, air conditioner and router) to switch 1 to turn on device 1; Correspondence 2: Point from light to switch 2 to increase the brightness of the light; Correspondence 3: Point from speaker to switch 2 to increase the volume of the speaker.

[0146] When a user performs a spatial pointing gesture, if the user points to a light, computer, and router and then to switch 1 in one spatial pointing gesture, then corresponding relationship 1 can be matched. Based on this correspondence, the light, computer, and router can be turned on. If the user points to a light and speaker and then to switch 2 in one spatial pointing gesture, then corresponding relationships 2 and 3 can be matched. Based on these two correspondences, the brightness of the light can be adjusted, and the volume of the speaker can be increased at the same time.

[0147] The above is just an example. In actual implementation, some correspondences can be preset as needed. Users can also set personalized correspondences as needed and can change the system's preset correspondences. This embodiment does not make any special limitations on this.

[0148] In specific applications, at least one of the above scenarios can be selected to implement the relevant functions as needed. That is, the electronic device can have the control functions corresponding to at least one of the above scenarios. In specific implementation, they can all be set as needed. This embodiment does not make any special limitations on this.

[0149] Those skilled in the art will understand that the above embodiments are exemplary and not intended to limit this application. Where possible, the execution order of one or more of the above steps can be adjusted, or they can be selectively combined to obtain one or more other embodiments. Those skilled in the art can arbitrarily select and combine the above steps as needed, and all those that do not depart from the essence of this application fall within the protection scope of this application.

[0150] The device control method provided in this embodiment executes corresponding control operations based on the device pointing sequence of the target device pointed to by the user's spatial pointing gesture. This abstracts the gesture into a specific device, making it easier for users to remember and use. Moreover, spatial pointing gestures are relatively easy to design and can support a large number of gestures. In addition, users can set their own exclusive spatial pointing gestures as needed, which can meet the needs of different users. Furthermore, users can use devices in space as coordinate points to realize spatial pointing gestures, thus giving the gestures a basis and further improving the convenience of use.

[0151] Based on the same inventive concept, as an implementation of the above method, this application provides a device control apparatus. This apparatus embodiment corresponds to the aforementioned method embodiment. For ease of reading, this apparatus embodiment will not repeat the details of the aforementioned method embodiment one by one, but it should be clear that the apparatus in this embodiment can correspondingly implement all the contents of the aforementioned method embodiment.

[0152] Figure 7 This is a schematic diagram of the structure of the device control apparatus provided in the embodiments of this application, such as... Figure 7 As shown, the apparatus provided in this embodiment includes:

[0153] Detection module 210 is used to detect the user's spatial pointing gestures;

[0154] The recording module 220 is used to record the device pointing order of the multiple target devices pointed to by the spatial pointing gesture;

[0155] The control module 230 is used to perform corresponding control operations according to the device pointing sequence.

[0156] As one possible implementation of this application, the target devices involved in the device pointing order include a first device and at least one second device, wherein the pointing order of the first device precedes the pointing order of the second device, and the control module 230 is specifically used for:

[0157] For each second device in the device pointing sequence, control the first device and the second device to perform the corresponding function association operation.

[0158] As one possible implementation of this application embodiment, the control module 230 is specifically used for:

[0159] If the second device is an audio device, control the second device to play the sound information from the first device;

[0160] When the second device is a display device, control the second device to display the target content of the first device, or execute and display the target application of the first device;

[0161] If the second device is a printing device, control the second device to print the target file of the first device.

[0162] As one possible implementation of this application embodiment, the detection module 210 is further configured to: detect the user's cancellation operation;

[0163] The control module 230 is further configured to: when the detection module 210 detects a user's cancellation operation, in response to the cancellation operation, control the first device and the second device corresponding to the cancellation operation to perform a function-associated cancellation operation.

[0164] As one possible implementation of this application, the cancellation operation is a first spatial pointing gesture, and the target device pointed to by the first spatial pointing gesture includes the second device corresponding to the cancellation operation first and the first device pointed to later;

[0165] Alternatively, the cancellation operation is a combination of a second spatial pointing gesture and a target operation, wherein the second spatial pointing gesture includes a second device corresponding to the cancellation operation, and the target operation is a target cancellation gesture, a target touch operation, or a target button operation.

[0166] As one possible implementation of this application, the control module 230 is specifically used to: control the third device to perform permission operations according to the device pointing order.

[0167] As one possible implementation of this application embodiment, the control module 230 is specifically used for:

[0168] If the device pointing order is consistent with the preset device pointing order, control the third device to perform the interface unlocking operation;

[0169] Alternatively, if the device pointing order is consistent with the preset device pointing order, the third device can be controlled to turn on or off.

[0170] As one possible implementation of this application, the target devices involved in the device pointing order include a fourth device and a fifth device, the fourth device includes at least one, and the fifth device is the last target device to be pointed to. The control module 230 is specifically used to control the fourth device to perform corresponding state control operations based on the fifth device.

[0171] As one possible implementation of this application embodiment, the control module 230 is specifically used for:

[0172] When the fifth device is a switch, it controls the fourth device to turn on or off, or controls the fourth device to adjust brightness, volume, or temperature.

[0173] If the fifth device is a relay device, control the fourth device to connect to or refresh the network;

[0174] The direction of sound playback or shooting direction of the fourth device is controlled according to the position of the fifth device.

[0175] As one possible implementation of this application embodiment, the detection module 210 is specifically used for:

[0176] When a user's spatial pointing gesture is detected, the user's spatial pointing gesture is detected in response to the spatial pointing gesture triggering operation.

[0177] As one possible implementation of this application, the spatial pointing gesture triggering operation is a first target gesture, a touch operation on a target control, or a press operation on a target button.

[0178] As one possible implementation of this application, the control module 230 is specifically used to: when the detection module 210 detects that the user's spatial pointing gesture has ended, in response to the spatial pointing gesture ending operation, execute corresponding control operations according to the device pointing sequence.

[0179] As one possible implementation of this application, the spatial pointing gesture end operation is a second target gesture, a touch release operation on a target control, or a button release operation on a target button.

[0180] As one possible implementation of this application, the detection module 210 is further configured to: detect and save the correspondence between the device pointing order set by the user and the control operation before the control module 230 executes the corresponding control operation according to the device pointing order.

[0181] As one possible implementation of this application, the spatial pointing gesture is a pointing operation performed by a handheld electronic device.

[0182] The apparatus provided in this embodiment can execute the above method embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.

[0183] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0184] Based on the same inventive concept, embodiments of this application also provide an electronic device. Figure 8 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application, such as... Figure 8 As shown, the electronic device may include a processor 110, an external memory interface 120, an internal memory 121, a Universal Serial Bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a Subscriber Identification Module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0185] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0186] Processor 110 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0187] The controller can serve as the nerve center and command center of an electronic device. Based on the instruction opcode and timing signals, the controller generates operation control signals to control the fetching and execution of instructions.

[0188] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0189] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an Inter-Integrated Circuit (I2C) interface, an Inter-Integrated Circuit Sound (I2S) interface, a Pulse Code Modulation (PCM) interface, a Universal Asynchronous Receiver / Transmitter (UART) interface, a Mobile Industry Processor Interface (MIPI) interface, a General-Purpose Input / Output (GPIO) interface, a Subscriber Identity Module (SIM) interface, and / or a Universal Serial Bus (USB) interface, etc.

[0190] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C buses. The processor 110 can couple to the touch sensor 180K, charger, flash, camera 193, etc., through different I2C bus interfaces. For example, the processor 110 can couple to the touch sensor 180K through the I2C interface, enabling the processor 110 and the touch sensor 180K to communicate through the I2C bus interface, thereby realizing the touch function of the electronic device.

[0191] The I2S interface can be used for audio communication. In some embodiments, the processor 110 may include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface to enable the function of answering phone calls through a Bluetooth headset.

[0192] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via the PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering phone calls through a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0193] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface to enable music playback through Bluetooth headphones.

[0194] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a Camera Serial Interface (CSI) and a Display Serial Interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to enable the electronic device to capture images. The processor 110 and the display screen 194 communicate via the DSI interface to enable the electronic device to display images.

[0195] The GPIO interface can be configured via software. It can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to a camera 193, a display screen 194, a wireless communication module 160, an audio module 170, a sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0196] USB port 130 is a USB standard compliant interface, which can be a Mini USB port, Micro USB port, USB Type-C port, etc. USB port 130 can be used to connect a charger to charge electronic devices, and can also be used for data transfer between electronic devices and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices.

[0197] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a limitation on the structure of the electronic device. In other embodiments of this application, the electronic device may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0198] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via a USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the electronic device. While charging the battery 142, the charging management module 140 can also supply power to the electronic device via the power management module 141.

[0199] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, external memory, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.

[0200] The wireless communication function of electronic devices can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0201] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0202] The mobile communication module 150 can provide solutions for wireless communication applications including 2G / 3G / 4G / 5G in electronic devices. The mobile communication module 150 may include at least one filter, switch, power amplifier, low-noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0203] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0204] The wireless communication module 160 can provide solutions for wireless communication applications in electronic devices, including Wireless Local Area Networks (WLANs) (such as Wireless Fidelity (Wi-Fi) networks), Bluetooth (BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), and Infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0205] In some embodiments, antenna 1 of the electronic device is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling the electronic device to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division-Synchronous Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GNSS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or Satellite Based Augmentation Systems (SBAS).

[0206] Electronic devices implement display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0207] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), an Active-Matrix Organic Light-Emitting Diode (AMOLED), a Flexible Light-Emitting Diode (FLED), a MiniLED, a MicroLED, a Quantum Dot Light-Emitting Diode (QLED), etc. In some embodiments, the electronic device may include one or N displays 194, where N is a positive integer greater than 1.

[0208] Electronic devices can achieve shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0209] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0210] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device may include one or N cameras 193, where N is a positive integer greater than 1.

[0211] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when an electronic device is selecting a frequency, a DSP can perform a Fourier transform on the frequency energy.

[0212] Video codecs are used to compress or decompress digital video. Electronic devices can support one or more video codecs. This allows the electronic device to play or record video in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0213] NPU stands for Neural Network (NN) computing processor. By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0214] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0215] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of the electronic device by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of the electronic device (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, Universal Flash Storage (UFS), etc.

[0216] Electronic devices can implement audio functions such as music playback and recording through audio modules 170, speakers 170A, receivers 170B, microphones 170C, headphone jacks 170D, and application processors.

[0217] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0218] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. Electronic devices can listen to music or make hands-free calls through the speaker 170A.

[0219] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When an electronic device answers a phone call or voice message, the receiver 170B can be brought close to the ear to hear the voice.

[0220] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. Electronic devices can have at least one microphone 170C. In some embodiments, electronic devices can have two microphones 170C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, electronic devices can have three, four, or more microphones 170C, enabling sound signal collection, noise reduction, sound source identification, and directional recording, among other functions.

[0221] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0222] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be disposed on display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 180A, the capacitance between the electrodes changes. The electronic device determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 194, the electronic device detects the intensity of the touch operation based on pressure sensor 180A. The electronic device can also calculate the touch position based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example, when a touch operation with an intensity less than a first pressure threshold is applied to the SMS application icon, a command to view an SMS is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the SMS application icon, a command to create a new SMS is executed.

[0223] The gyroscope sensor 180B can be used to determine the motion attitude of an electronic device. In some embodiments, the gyroscope sensor 180B can determine the angular velocity of the electronic device around three axes (i.e., the x, y, and z axes). The gyroscope sensor 180B can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the electronic device's shake, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to counteract the shake of the electronic device by moving in the opposite direction, thus achieving image stabilization. The gyroscope sensor 180B can also be used in navigation and motion-sensing gaming scenarios.

[0224] The barometric pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device calculates altitude using the air pressure value measured by the barometric pressure sensor 180C to assist in positioning and navigation.

[0225] The magnetic sensor 180D includes a Hall sensor. The electronic device can use the magnetic sensor 180D to detect the opening and closing of the flip cover. In some embodiments, when the electronic device is a flip phone, the electronic device can detect the opening and closing of the flip cover using the magnetic sensor 180D. Then, based on the detected opening and closing state of the cover or the flip cover, features such as automatic flip unlocking can be configured.

[0226] The 180E accelerometer can detect the magnitude of acceleration in various directions (typically three axes) of an electronic device. When the electronic device is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the posture of electronic devices, and is applicable to screen orientation switching, pedometers, and other applications.

[0227] A distance sensor 180F is used to measure distance. Electronic devices can measure distance using infrared or laser. In some embodiments, during a shooting scene, the electronic device can utilize the distance sensor 180F to measure distance for rapid focusing.

[0228] The proximity sensor 180G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The LED may be an infrared LED. The electronic device emits infrared light outward through the LED. The electronic device uses the photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that an object is near the electronic device. When insufficient reflected light is detected, the electronic device can determine that no object is near the electronic device. The electronic device can use the proximity sensor 180G to detect when a user holds the electronic device close to their ear for a call, so as to automatically turn off the screen to save power. The proximity sensor 180G can also be used in holster mode and pocket mode for automatic unlocking and locking of the screen.

[0229] The ambient light sensor 180L is used to detect ambient light levels. Electronic devices can adaptively adjust the brightness of their displays (194) based on the detected ambient light. The ambient light sensor 180L can also be used to automatically adjust white balance when taking photos. Furthermore, the ambient light sensor 180L can work in conjunction with the proximity sensor 180G to detect whether electronic devices are in a pocket, preventing accidental touches.

[0230] The fingerprint sensor 180H is used to collect fingerprints. Electronic devices can utilize the characteristics of the collected fingerprints to achieve fingerprint unlocking, app access locks, fingerprint photography, fingerprint answering of calls, etc.

[0231] Temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device uses the temperature detected by temperature sensor 180J to execute a temperature handling strategy. For example, when the temperature reported by temperature sensor 180J exceeds a threshold, the electronic device reduces the performance of a processor located near temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is below another threshold, the electronic device heats battery 142 to prevent abnormal shutdown of the electronic device due to low temperature. In still other embodiments, when the temperature is below yet another threshold, the electronic device boosts the output voltage of battery 142 to prevent abnormal shutdown due to low temperature.

[0232] Touch sensor 180K, also known as a "touch panel," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touchscreen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of the electronic device, in a different position than display screen 194.

[0233] The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire vibration signals from the vibrating bone segments of the human vocal cords. The bone conduction sensor 180M can also contact the human pulse to receive blood pressure signals. In some embodiments, the bone conduction sensor 180M can also be incorporated into headphones to form bone conduction headphones. The audio module 170 can parse the voice signals from the vibrating bone segments of the vocal cords acquired by the bone conduction sensor 180M to realize voice functionality. The application processor can parse heart rate information from the blood pressure signals acquired by the bone conduction sensor 180M to realize heart rate detection functionality.

[0234] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. The electronic device can receive button input and generate key signal inputs related to user settings and function control of the electronic device.

[0235] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. For example, different vibration feedback effects can correspond to touch operations performed on different applications (such as taking photos, playing audio, etc.). Motor 191 can also correspond to different vibration feedback effects for touch operations performed on different areas of the display screen 194. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0236] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0237] The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation with the electronic device. The electronic device can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 195 simultaneously. The multiple cards can be of the same or different types. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The electronic device interacts with the network through the SIM card to achieve functions such as calls and data communication. In some embodiments, the electronic device uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device and cannot be separated from it.

[0238] The electronic device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so they will not be described again here.

[0239] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the methods described in the above-described method embodiments.

[0240] This application also provides a computer program product that, when run on an electronic device, causes the electronic device to implement the method described in the above-described method embodiments.

[0241] This application also provides a chip system including a processor coupled to a memory. The processor executes a computer program stored in the memory to implement the method described in the above-described method embodiments. The chip system may be a single chip or a chip module composed of multiple chips.

[0242] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, or magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0243] Those skilled in the art will understand that implementing all or part of the processes in the above embodiments can be accomplished by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium can include various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0244] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0245] In the embodiments provided in this application, it should be understood that the disclosed apparatus / devices and methods can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0246] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0247] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0248] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0249] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0250] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0251] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A device control method, characterized in that, include: Detect the user's spatial pointing gesture and record the device pointing order of multiple target devices pointed to by the spatial pointing gesture; Execute the corresponding control operations according to the device pointing sequence; The step of executing corresponding control operations according to the device pointing sequence includes: When the target devices involved in the device pointing sequence include a fourth device and a fifth device, the fourth device includes at least one, and the fifth device is the last target device to be pointed to, the fourth device is controlled to perform corresponding state control operations based on the device type or location of the fifth device. The control of the fourth device to perform corresponding status control operations based on the device type or location of the fifth device includes at least one of the following: When the fifth device is a switch, it controls the fourth device to turn on or off, or controls the fourth device to adjust brightness, volume, or temperature. If the fifth device is a relay device, control the fourth device to connect to or refresh the network; The direction of sound playback or shooting direction of the fourth device is controlled according to the position of the fifth device.

2. The method according to claim 1, characterized in that, The step of executing corresponding control operations according to the device pointing sequence includes: If the target devices involved in the device pointing sequence include a first device and at least one second device, and the pointing order of the first device is prior to the pointing order of the second device, then for each second device in the device pointing sequence, the first device and the second device are controlled to perform corresponding functional association operations.

3. The method according to claim 2, characterized in that, The control of the first device and the second device to perform corresponding function association operations includes: If the second device is an audio device, control the second device to play the sound information from the first device; When the second device is a display device, control the second device to display the target content of the first device, or execute and display the target application of the first device; If the second device is a printing device, control the second device to print the target file of the first device.

4. The method according to claim 2, characterized in that, The method further includes: Upon detecting a user's cancellation operation, in response to the cancellation operation, the first device and the second device corresponding to the cancellation operation are controlled to perform a function-associated cancellation operation.

5. The method according to claim 4, characterized in that, The cancellation operation is a first spatial pointing gesture, and the target device pointed to by the first spatial pointing gesture includes the second device corresponding to the cancellation operation first and the first device pointed to later; Alternatively, the cancellation operation is a combination of a second spatial pointing gesture and a target operation, wherein the second spatial pointing gesture includes a second device corresponding to the cancellation operation, and the target operation is a target cancellation gesture, a target touch operation, or a target button operation.

6. The method according to claim 1, characterized in that, The step of executing corresponding control operations according to the device pointing sequence includes: If the device pointing order is consistent with the preset device pointing order, control the third device to perform the permission operation.

7. The method according to claim 6, characterized in that, The control of the third device to perform authorized operations includes: Control the third device to unlock the interface; Alternatively, control the third device to turn on or off.

8. The method according to claim 1, characterized in that, The detection of the user's spatial pointing gesture includes: When a user's spatial pointing gesture is detected, the user's spatial pointing gesture is detected in response to the spatial pointing gesture triggering operation.

9. The method according to claim 8, characterized in that, The spatial pointing gesture trigger operation is a first target gesture, a touch operation on the target control, or a press operation on the target button.

10. The method according to claim 1, characterized in that, The step of executing corresponding control operations according to the device pointing sequence includes: When the user's spatial pointing gesture ends, in response to the end of the spatial pointing gesture, the corresponding control operation is executed according to the device pointing sequence.

11. The method according to claim 10, characterized in that, The spatial pointing gesture end operation is a second target gesture, a touch release operation on the target control, or a button release operation on the target button.

12. The method according to claim 1, characterized in that, Before executing the corresponding control operations according to the device pointing sequence, the method further includes: Detect and save the correspondence between the device pointing order set by the user and the control operation.

13. The method according to any one of claims 1-12, characterized in that, The spatial pointing gesture refers to the pointing operation performed by a handheld electronic device.

14. An electronic device, characterized in that, include: A memory and a processor, the memory being used to store a computer program; the processor being used to execute the method as described in any one of claims 1-13 when the computer program is invoked.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-13.

16. A chip system, characterized in that, The chip system includes a processor coupled to a memory, the processor executing a computer program stored in the memory to implement the method as described in any one of claims 1-13.

Images