Device control method, electronic device, storage medium, and program product

Abstract

A device control method, an electronic device, a storage medium, and a program product. The device control method is applied to a wearable device, the wearable device comprising multiple haptic feedback structures, the haptic feedback structures being used to provide haptic feedback to a user wearing the wearable device. The method comprises: after a wearable device is connected to an electronic device, acquiring an instruction for the wearable device from the electronic device; on the basis of the instruction, controlling working states of the multiple haptic feedback structures.

Description

Equipment control methods, electronic devices, storage media, and software products

[0001] This disclosure claims priority to Chinese patent application No. 202410794511.9, filed on June 18, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to the field of wearable device technology, and more particularly to a device control method, electronic device, storage medium, and program product. Background Technology

[0003] With the development of technology, wearable devices such as smart bracelets and smart rings have demonstrated their unique value in the experience of virtual scenes such as augmented reality (AR) and virtual reality (VR). For example, based on the sensors equipped to recognize user gestures, users can perform human-computer interaction operations such as moving, grasping, or rotating virtual objects in the virtual scene based on gestures.

[0004] Currently, smart bracelets, smart rings, and other devices can also provide users with visual or vibration feedback. Summary of the Invention

[0005] On one hand, a device control method is provided, applied to a wearable device. The wearable device includes multiple haptic feedback structures, which provide haptic feedback to a user wearing the wearable device. The device control method includes:

[0006] After the wearable device is connected to the electronic device, the instructions from the electronic device to the wearable device are obtained.

[0007] The operating state of the plurality of haptic feedback structures is controlled according to the instructions.

[0008] On the other hand, a device control apparatus is provided for use in a wearable device. The wearable device includes multiple haptic feedback structures for providing haptic feedback to a user wearing the wearable device. The device control apparatus includes: an acquisition module and a control module;

[0009] The acquisition module is used to acquire the instructions from the electronic device to the wearable device after the wearable device is connected to the electronic device.

[0010] The control module is used to control the working state of the plurality of haptic feedback structures according to the instructions.

[0011] On another front, a device control method is provided, applied to a wearable device. The wearable device includes multiple haptic feedback structures, which are used to acquire information about the user's actions on the wearable device. The device control method includes:

[0012] Obtain pressure information from the multiple haptic feedback structures corresponding to the user's gestures;

[0013] Based on the pressure information from the multiple tactile feedback structures, the type information of the gesture is identified;

[0014] The type information of the gesture action is sent to the electronic device, wherein the type information of the gesture action is used to trigger the electronic device to execute a control command corresponding to the type information of the gesture action.

[0015] On another front, a device control apparatus is provided for use in a wearable device. The wearable device includes multiple haptic feedback structures, which are used to acquire information about the user's actions on the wearable device. The device control apparatus includes: an acquisition module, an identification module, and a transmission module.

[0016] The acquisition module is used to acquire pressure information of the multiple tactile feedback structures corresponding to the user's gesture actions;

[0017] The recognition module is used to identify the type information of the gesture based on the pressure information of the plurality of tactile feedback structures;

[0018] The sending module is used to send the type information of the gesture to the electronic device, and the type information of the gesture is used to trigger the electronic device to execute the control command corresponding to the type information of the gesture.

[0019] In another aspect, an electronic device is provided. The electronic device includes: a memory and a processor; wherein the memory and the processor are coupled; the memory is used to store instructions executable by the processor; and the processor executes the instructions to perform the device control method described in any of the preceding aspects.

[0020] In another aspect, a computer-readable storage medium is provided. This computer-readable storage medium stores computer instructions that, when executed on a computer, cause the computer to perform the device control method described in any of the preceding aspects.

[0021] In another aspect, a computer program product is provided. This computer program product includes a computer program that, when executed by a processor, implements the device control method described in any of the preceding aspects. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in this disclosure, the accompanying drawings used in some embodiments of this disclosure will be briefly described below. Obviously, the drawings described below are merely drawings of some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings.

[0023] Figure 1 is a system architecture diagram according to some embodiments of the present disclosure.

[0024] Figure 2 is a schematic diagram of the structure of a wearable device according to some embodiments of the present disclosure.

[0025] Figure 3 is a schematic diagram of a wearable device including a first haptic feedback structure according to some embodiments of the present disclosure.

[0026] Figure 4 is a schematic diagram of a wearable device including a second haptic feedback structure according to some embodiments of the present disclosure.

[0027] Figure 5 is a schematic diagram of the structure of another wearable device according to some embodiments of the present disclosure.

[0028] Figure 6 is a schematic flowchart of a device control method according to some embodiments of the present disclosure.

[0029] Figure 7 is a schematic flowchart of another device control method according to some embodiments of the present disclosure.

[0030] Figure 8 is a flowchart illustrating another device control method according to some embodiments of the present disclosure.

[0031] Figure 9 is a schematic diagram of a gesture type according to some embodiments of the present disclosure.

[0032] Figure 10 is a schematic diagram of a wearable device providing directional prompts according to some embodiments of the present disclosure.

[0033] Figure 11 is a schematic diagram of a wearable device providing distance prompts according to some embodiments of the present disclosure.

[0034] Figure 12 is a schematic diagram of an execution progress prompt in a wearable device according to some embodiments of the present disclosure.

[0035] Figure 13 is a schematic diagram of the structure of a device control apparatus according to some embodiments of the present disclosure.

[0036] Figure 14 is a schematic diagram of the structure of another device control apparatus according to some embodiments of the present disclosure.

[0037] Figure 15 is a schematic diagram of the structure of another device control apparatus according to some embodiments of the present disclosure. Detailed Implementation

[0038] To enable those skilled in the art to better understand the technical solutions of the embodiments of this disclosure, the technical solutions of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0039] It should be noted that in this disclosure, the terms "exemplary / exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary / exemplary" or "for example" in this disclosure should not be construed as being more preferred or advantageous than other embodiments or designs in this disclosure. Specifically, the use of terms such as "exemplary / exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0040] In the following text, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with terms such as "first," "second," etc., may explicitly or implicitly include one or more of that feature.

[0041] In the description of this disclosure, unless otherwise stated, the symbol “ / ” means “or”, for example, A / B can mean A or B. The term “and / or” in this document is merely a description of the relationship between related objects, indicating that three relationships can exist; for example, A and / or B can mean: only A, only B, and A and B. Furthermore, “at least one” means one or more, and “multiple” means two or more.

[0042] With the rapid development of technology, wearable devices such as smart rings, smart bracelets, and watches, with their small and portable characteristics, not only play a role in health and fitness tracking, but also demonstrate their unique value in virtual scene experiences such as AR and VR.

[0043] Smart bracelets and watches are typically equipped with sensors that can recognize users' gestures and movements. In AR / VR environments, these smart bracelets or watches work in conjunction with AR glasses or VR headsets, serving as an input device for these displays. Users can interact with virtual objects through simple gestures, performing actions such as grasping, moving, or rotating, achieving a more natural and intuitive human-computer interaction. Simultaneously, these smart bracelets or watches can provide feedback to users through visual cues, vibration alerts, and other means, offering a more realistic experience and enhancing user immersion.

[0044] However, in virtual scenarios such as VR and AR, the feedback prompts to users through wearable devices such as smart rings, smart bracelets, and watches are mainly achieved through relatively simple forms such as visual information or simple vibrations, resulting in a poor user experience.

[0045] To address the aforementioned problems, this disclosure provides a device control method applied to a wearable device. The wearable device includes multiple haptic feedback structures that provide haptic feedback to a user wearing the device. The device control method includes: after the wearable device is connected to an electronic device, acquiring instructions from the electronic device to the wearable device; and controlling the operating state of the multiple haptic feedback structures according to the instructions. Thus, by controlling the operating state of the multiple haptic feedback structures through instructions from the electronic device, the multiple haptic feedback structures can provide haptic feedback to the user, enhancing the user's immersion and improving the user experience.

[0046] The device control method provided in this disclosure can be applied to the device control system shown in FIG1. ​​As shown in FIG1, the device control system includes: a wearable device 101 and an electronic device 102. The wearable device 101 and the electronic device 102 are connected.

[0047] Wearable device 101 includes a housing 1011 and a haptic feedback structure 1012. The haptic feedback structure 1012 is used to provide haptic feedback to the user wearing the device; or to receive instructions sent by electronic device 102 and control the plurality of haptic feedback structures based on the instructions; or to send the user's gesture information to electronic device 102.

[0048] Electronic device 102 is used to generate and manage extended reality scenes; or to send instructions to wearable device 101; or to receive gesture information sent by wearable device.

[0049] It should be noted that Figure 1 is only an exemplary framework diagram, and the number of devices included in Figure 1 and the names of each device are not limited.

[0050] Figure 2 is a schematic diagram of the structure of a wearable device according to some embodiments of the present disclosure. The wearable device includes a haptic feedback structure 201, an information acquisition module 202, a user input module 203, a calculation and processing module 204, and an execution and control module 205.

[0051] In some embodiments, multiple haptic feedback structures 201 may exist.

[0052] In some embodiments, the haptic feedback structure 201 can be a first haptic feedback structure, which provides haptic feedback to the user through its own deformation. The first haptic feedback structure includes a flexible cavity.

[0053] For example, the first tactile feedback structure includes an airbag and an inflation device for inflating the airbag.

[0054] For example, FIG3 is a schematic diagram of a wearable device including a first tactile feedback structure according to some embodiments of the present disclosure, including a wearable device in a default state (without deformation), a wearable device with a single cavity deformation, a wearable device with multiple cavity deformations, and a wearable device with all cavities deformed, wherein the wearable device includes multiple flexible cavities and a shell.

[0055] In some embodiments, the haptic feedback structure 201 can be a second haptic feedback structure, which provides haptic feedback to the user through its own displacement. The second haptic feedback structure includes a rigid cavity.

[0056] For example, as shown in FIG4, this disclosure provides a schematic diagram of a wearable device including a second tactile feedback structure, including a wearable device in a default state (no displacement), a wearable device with a single cavity displacement, a wearable device with multiple cavity displacements, and a wearable device with all cavity displacements, wherein the wearable device includes multiple rigid cavities and a shell.

[0057] In some embodiments, the multiple haptic feedback structures of the wearable device are evenly distributed.

[0058] For example, for wearable devices without clasps, such as wristbands or rings, multiple haptic feedback structures on the wearable device can be arranged in a uniformly distributed manner.

[0059] In some embodiments, the multiple haptic feedback structures of the wearable device are non-uniformly distributed.

[0060] For example, in wearable devices such as watches where the tightness of the fit can be adjusted by adjusting the position of the clasp, multiple haptic feedback structures are arranged in the non-adjustable area of ​​the wearable device. Alternatively, in cases where the wearable device contains sensors that need to be in close contact with the skin, multiple haptic feedback structures are arranged in an area outside the sensor layout area. For instance, Figure 5 is a schematic diagram of the structure of another wearable device according to some embodiments of this disclosure. The wearable device includes a sensor 501 that is in close contact with the skin, a size adjustment area 502, and multiple haptic feedback structures 503. Figure 5 shows the unfolded state and the worn state of the wearable device.

[0061] It should be noted that each of the multiple haptic feedback structures can be controlled independently, providing haptic feedback to the user without interfering with each other.

[0062] In one implementation, the accuracy of haptic feedback varies among different wearable devices due to the different numbers of haptic feedback structures they include. The more haptic feedback structures a wearable device includes, the higher its accuracy in providing haptic feedback.

[0063] The information acquisition module 202 includes various sensors, such as: a pressure sensor, used to acquire the pressure on each haptic feedback structure and convert the pressure into an electrical signal; a gyroscope, used to acquire the orientation of the haptic feedback structure to calibrate the relative position between the haptic feedback structure and the operable elements in the extended reality scene; a communication sensor, used to establish a communication connection with the electronic device that manages the extended reality and acquire the position information or progress information of elements (or objects) in the extended reality scene in real time; and a positioning sensor, used to acquire the position information of the wearable device.

[0064] In some embodiments, the positioning sensor or communication sensor may employ one or more of the following technologies: wireless fidelity (Wi-Fi), Bluetooth low energy (BLE), infrared, ultra-wideband (UWB), radio frequency identification (RFID), etc.

[0065] User input module 203 is used to acquire user commands such as voice input, screen operation input, and gesture input. The voice input user input module 203 includes a voice command microphone or microphone array, a speech recognition engine for semantic recognition, and other auxiliary components (such as a noise reduction module). To ensure sound reception and device size limitations, the voice recognition user input module is placed in an electronic device (AR / VR host) or other device that manages the extended reality scene (e.g., a smart speaker connected to an AR / VR host). The screen operation input user input module 203 includes a screen that supports clicks, swipes, and other operations. This screen can be the interactive display screen of the wearable device itself (e.g., a wristband, watch, head-mounted wearable device). The gesture input user input module 203 includes a camera, motion sensor, gesture recognition algorithm, touch sensing module, etc. By analyzing the pressure changes experienced by various tactile feedback structures during user gesture operations, it can accurately recognize user gestures and execute the corresponding operations.

[0066] The calculation and processing module 204 is used to calculate and process the position information of the wearable device, the orientation information of the wearable device, the position information of the elements in the extended real scene, and the progress information of the elements in the extended real scene obtained by the sensor, to obtain the control parameters of multiple haptic feedback structures, and send the control parameters to the execution and control module 205.

[0067] The execution and control module 205 is used to control multiple haptic feedback structures to provide haptic feedback (or control the haptic feedback structures to squeeze the area in contact with the user) based on the control parameters of multiple haptic feedback structures; or to execute user-inputted instructions.

[0068] The application scenarios in this disclosure are not limited. The system architecture and business scenarios described in this disclosure are for the purpose of more clearly illustrating the technical solutions of this disclosure and do not constitute a limitation on the technical solutions provided in this disclosure. As those skilled in the art will know, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in this disclosure are also applicable to similar technical problems.

[0069] The device control method provided in the embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0070] The device control method provided in this disclosure can be applied to the device control system shown in FIG1. ​​FIG6 shows a schematic flowchart of a device control method, as shown in FIG6, the device control method includes the following S601-S602.

[0071] S601. After the wearable device is connected to the electronic device, obtain the instructions from the electronic device to the wearable device.

[0072] Wearable devices can serve as output devices for electronic devices. As output devices, wearable devices can receive commands from electronic devices.

[0073] In some embodiments, the electronic device can be a host device for an extended reality scene, used to generate and manage the extended reality scene. The extended reality scene may include virtual elements and / or entity elements (or virtual objects and entity objects). The management of the extended reality scene by the electronic device includes operations such as generating, deleting, and moving virtual elements or entity elements.

[0074] In some embodiments, extended reality scenes include virtual reality scenes, augmented reality scenes, and mixed reality (MR) scenes.

[0075] In some embodiments, the instructions from the electronic device to the wearable device include at least one of the following: the execution progress of the task to be performed, the relative distance between the wearable device and the operable element in the extended reality scene, and the relative orientation between the wearable device and the operable element.

[0076] For example, the task to be performed may include one or more of the following: a download task, a task in an interactive game, or a timed task. The executable operations corresponding to operable elements in extended real-world scenarios include one or more of the following: move, rotate, delete (or demolish), resize, and play effects.

[0077] S602. Control the working state of multiple haptic feedback structures according to the instructions.

[0078] In some embodiments, instructions from the electronic device to the wearable device are used to indicate information to be fed back. Based on the information to be fed back in the instructions, the wearable device controls the operating state of multiple haptic feedback structures, enabling these structures to provide haptic feedback to the user, thereby prompting the user for the information to be fed back and completing the human-computer interaction. Thus, controlling the operating state of multiple haptic feedback structures through instructions from the electronic device allows these structures to provide haptic feedback to the user, enhancing the user's immersion and improving the user experience.

[0079] In some embodiments, the operating state of multiple haptic feedback structures is adjusted by at least one of the following parameters: the number of target haptic feedback structures, the identifier of the target haptic feedback structure, and the pressure parameter of the target haptic feedback structure. The target haptic feedback structure is a haptic feedback structure that creates a squeezing sensation in the area contacted by the user; the pressure parameter is used to determine the pressure of the squeezing sensation.

[0080] In some embodiments, the instructions include the execution progress of the task to be performed or the relative distance between the wearable device and the operable element, and the operating states of the plurality of haptic feedback structures satisfy at least one of the following relationships:

[0081] The execution progress of the task to be performed or the relative distance between the wearable device and the operable element and the number of target tactile feedback structures satisfy the first preset relationship;

[0082] The execution progress of the task to be performed or the relative distance between the wearable device and the operable element and the pressure parameters of the target tactile feedback structure satisfy the second preset relationship.

[0083] In some embodiments, the first preset relationship can be a positive correlation. A positive correlation between the execution progress of the task to be executed and the number of target haptic feedback structures means that the greater the execution progress, the greater the number of target haptic feedback structures; a positive correlation between the execution progress of the task to be executed and the pressure parameters of the target haptic feedback structures means that the greater the execution progress, the greater the pressure parameters of the target haptic feedback structures.

[0084] In some embodiments, the second preset relationship can be a negative correlation. A negative correlation between the relative distance between the wearable device and the operable element and the number of target haptic feedback structures means that the greater the relative distance, the fewer the target haptic feedback structures. Similarly, a negative correlation between the relative distance between the wearable device and the operable element and the pressure parameter of the target haptic feedback structure means that the greater the relative distance, the smaller the pressure parameter of the target haptic feedback structure.

[0085] In some embodiments, the instructions include the relative orientation between the wearable device and the operable element, and the haptic feedback structure that is closest to the operable element among a plurality of haptic feedback structures is designated as the target haptic feedback structure.

[0086] It should be understood that the line connecting the haptic feedback structure closest to the operable element and the center point of the wearable device points to the operable element. That is, the relative orientation between the wearable device and the operable element can be determined based on the haptic feedback structure closest to the operable element. In this case, the wearable device will take the haptic feedback structure closest to the operable element among multiple haptic feedback structures as the target haptic feedback structure.

[0087] In conjunction with the description of the above embodiments, in some embodiments, S602 controls the working state of multiple haptic feedback structures according to instructions. This can be achieved in the following ways:

[0088] Based on the execution progress and positive correlation of the tasks to be executed in the instructions, determine the Q target haptic feedback structures participating in this haptic feedback and the pressure parameters corresponding to each of the Q target haptic feedback structures; based on the number of target haptic feedback structures and the pressure parameters, control the Q target haptic feedback structures to work based on the corresponding pressure parameters, where Q is a positive integer.

[0089] Alternatively, based on the relative distance and negative correlation between the wearable device and the operable element included in the instruction, determine the number M of target haptic feedback structures and the pressure parameters among multiple haptic feedback structures; operate based on the pressure parameters of M target haptic feedback structures, where M is a positive integer.

[0090] Alternatively, based on the relative orientation between the wearable device and the operable element included in the instructions, the identifier and pressure parameters of the haptic feedback structure closest to the operable element among multiple haptic feedback structures are determined, and based on the identifier and pressure parameters, the closest haptic feedback structure is controlled to operate according to the pressure parameters.

[0091] Alternatively, based on the execution progress and positive correlation of the tasks to be executed included in the instruction, determine the number and pressure parameters of the target haptic feedback structure among the multiple haptic feedback structures; and control the working state of the multiple haptic feedback structures based on the number and pressure parameters of the target haptic feedback structure.

[0092] For example, the execution progress of the task to be performed and the number of target haptic feedback structures satisfy the following formula (1): M=m×P (1)

[0093] Where M represents the number of target haptic feedback structures, m represents the number of haptic feedback structures in the wearable device, and P represents the execution progress of the task to be performed. When the execution progress is 0, the number of target haptic feedback structures is 0; when the execution progress is 100%, the number of target haptic feedback structures is m.

[0094] In some embodiments, the instructions include a relative position between the wearable device and the operable element, the relative position including a relative distance and a relative orientation. When the wearable device receives the relative position, it determines a haptic feedback structure as the extension center among a plurality of haptic feedback structures based on the relative orientation.

[0095] As the wearable device is moved, when the relative distance is detected to decrease to a first distance threshold, the wearable device controls the haptic feedback structure, which acts as the extension center, to squeeze the area in contact with the user to indicate the location of the controllable element.

[0096] When the relative distance is detected to decrease to less than a first distance threshold but greater than a second distance threshold (the second distance threshold is less than the first distance threshold), the wearable device uses a haptic feedback structure to squeeze the area in contact with the user based on the relative distance to determine the number of targets.

[0097] In some embodiments, the wearable device preferentially controls the haptic feedback structures on both sides of the haptic feedback structure serving as the extension center to squeeze the area in contact with the user.

[0098] For example, the relative distance and the number of targets satisfy the following formula (2):

[0099] Where N is the number of targets, S is the relative distance between the wearable device and the operable element, S1 is the first distance threshold, S2 is the second distance threshold, and n is the number of haptic feedback structures in the wearable device.

[0100] In some embodiments, the relative distance between the wearable device and the operable element is the smallest relative distance among the relative distances between each of the multiple haptic feedback structures in the wearable device and the operable element.

[0101] When the relative distance is detected to decrease to the second distance threshold, the wearable device controls all haptic feedback structures to squeeze the area in contact with the user. At this time, the wearable device can collect pressure information of the user's gesture to obtain information about the type of the user's gesture.

[0102] As shown in Figure 7, this disclosure also provides a device control method applied to a wearable device. The wearable device includes multiple haptic feedback structures, which are used to acquire information about the user's actions on the wearable device. The device control method includes steps S701-S703.

[0103] S701. Obtain pressure information from multiple haptic feedback structures corresponding to the user's gesture.

[0104] It should be understood that haptic feedback structures are used to acquire information about the user's actions on the wearable device, including pressure information from multiple haptic feedback structures corresponding to the user's gestures. These multiple haptic feedback structures are the haptic feedback structures within the wearable device that are compressed due to the user's gestures.

[0105] In some embodiments, each haptic feedback structure has a pressure sensor (or pressure sensor). The pressure sensor on a haptic feedback structure is used to detect pressure information from the area in contact with the user that affects the haptic feedback structure.

[0106] S702: Based on pressure information from multiple haptic feedback structures, identify the type of hand gesture.

[0107] It is understandable that, since different gestures exert different pressures on the haptic feedback structures of wearable devices, the type of gesture can be identified by the pressure information of multiple haptic feedback structures corresponding to the user's gesture.

[0108] S703, Send the type information of the gesture action to the electronic device.

[0109] The type information of the gesture is used to trigger the electronic device to execute the control command corresponding to the type information of the gesture.

[0110] It should be understood that the type information of the gesture is determined based on the pressure information of multiple tactile feedback structures corresponding to the user's gesture.

[0111] Understandably, wearable devices can act as sensing devices, acquiring pressure information from multiple tactile feedback structures corresponding to hand gestures, and identifying the type of the user's gesture based on this pressure information. Subsequently, the wearable device can function as an input device for an electronic device, sending the gesture type information to the electronic device. The electronic device can then execute control commands corresponding to the gesture type information, completing the control of operable elements that the user wishes to achieve with that gesture. This completes the closed loop of the human-computer interaction process.

[0112] In some embodiments, the wearable device includes a communication module for enabling data exchange between the wearable device and an electronic device.

[0113] In some embodiments, control commands are used to control operable elements in an extended reality scenario managed by the electronic device. Referring to the embodiment shown in FIG7, as shown in FIG8, the above-described S701 involves acquiring pressure information from multiple haptic feedback structures corresponding to the user's gesture, including S801-S802.

[0114] S801, Obtain the relative distance between the wearable device and the operable element.

[0115] In some embodiments, the wearable device includes a positioning sensor, a communication sensor (or communication module), and a computing processing module. The positioning sensor is used to determine the position of the wearable device; the communication sensor is used to communicate with the electronic device to obtain the position of the operable element; the computing processing module calculates the relative distance between the wearable device and the operable element in the extended reality scene based on the position of the wearable device and the position of the operable element.

[0116] S802. When the relative distance is less than a preset distance threshold, obtain the pressure information of multiple tactile feedback structures corresponding to the user's gesture.

[0117] It should be understood that when the relative distance is less than a preset distance threshold, it means that the wearable device is within the range corresponding to the preset distance threshold of the operable element (this range can be referred to as the operable range of the operable element). At this time, the wearable device acquires pressure information from multiple haptic feedback structures corresponding to the user's gesture, thereby determining the type of the user's gesture. Thus, acquiring the user's gesture only when the wearable device is within the range corresponding to the preset distance threshold ensures that the user's gesture is directed at the operable element. This avoids accidental touches or other misoperations, ensuring the reliability of human-computer interaction through the wearable device and improving the user experience.

[0118] In some embodiments, when the relative distance is less than a preset distance threshold, the wearable device activates the pressure sensor on each haptic feedback structure to achieve the above-described S802.

[0119] For example, Figure 9 is a schematic diagram of gesture types according to some embodiments of the present disclosure. Taking a wearable device as a ring as an example, the wearable device is worn on the first knuckle of the user's index finger. Assuming that the gesture types include the "OK" gesture, the fist gesture, and the "heart" gesture, the wearable device includes multiple evenly distributed airbags. Figure 9 shows the pressure conditions of the OK gesture, the fist gesture, the heart gesture, and the multiple tactile feedback structures corresponding to each gesture.

[0120] When the user makes an "OK" gesture, their index finger and thumb touch, and the bent index finger compresses the air bladders on both sides of the ring (the part of the ring on the palm side is defined as "bottom," and the part on the back of the palm as "top"). When the user makes a fist gesture, their index and middle fingers squeeze the ring, causing the air bladder between their index and middle fingers to be compressed. When the user makes a heart gesture, the area between their thumb and index finger compresses the ring.

[0121] The following embodiments of this disclosure provide an example of a user performing remote operation through a wearable device, including: wearing and calibration, device connection, positioning and feedback, proximity and function activation, user operation, operation completion and exit.

[0122] Wearing and Calibration: After the user wears the wearable device, a pressure calibration is performed first to confirm the reference pressure of the haptic feedback structure. The reference pressure serves as the reference value without prompts.

[0123] In some embodiments, the haptic feedback structure is a flexible cavity, such as an airbag. The reference pressure of the haptic feedback structure is confirmed by: in response to a user's pressure calibration operation, controlling the airbag to inflate from an uninflated state; in response to a user's pressure confirmation operation (or in response to a user's inflation stop operation), stopping the inflation of the airbag, and determining the pressure corresponding to the current inflation level of the airbag as the reference pressure (or determining the operating state or operating parameters corresponding to the current inflation level of the airbag as the operating state or operating parameters corresponding to the airbag at the reference pressure).

[0124] In some embodiments, the haptic feedback structure is a rigid cavity, such as a cavity that compresses the area in contact with the user through displacement. The reference pressure of the haptic feedback structure is confirmed by: responding to the user's pressure calibration operation, controlling the rigid cavity to move from a state of no displacement to compress the area in contact with the user; responding to the user's pressure confirmation operation, stopping the displacement of the rigid cavity, and determining the pressure corresponding to the current displacement distance of the rigid cavity as the reference pressure (or determining the current operating state or operating parameters of the rigid cavity as the operating state or operating parameters of the rigid cavity under the reference pressure).

[0125] Device connectivity: Establishing a connection between wearable devices and electronic devices corresponding to extended reality scenarios.

[0126] Positioning and feedback: Determine the relative orientation and relative distance between manipulable elements and wearable devices in an extended reality scenario.

[0127] In some embodiments, prior to positioning and feedback, the wearable device may also obtain information about operable elements in an extended reality scene from an electronic device and prompt the user about the operable element.

[0128] In some embodiments, the relative direction and relative distance can be determined by one or more of the following methods: measured received signal strength indicator (RSSI) of wireless signals such as Bluetooth, Wi-Fi triangulation, acoustic positioning, global positioning system (GPS) positioning, image recognition and computer vision, and multi-device collaboration.

[0129] For example, the wearable device uses a haptic feedback structure through which the line connecting the center points of the wearable device and the operable element passes as a haptic feedback structure to prompt the user about the orientation of the operable element relative to the wearable device. For instance, Figure 10 is a schematic diagram of a wearable device providing orientation prompts according to some embodiments of this disclosure. When the relative orientation between the wearable device and the operable element changes, the haptic feedback structure used to prompt the user about the orientation of the operable element relative to the wearable device changes synchronously.

[0130] Proximity and Function Activation: In response to the user's movement of the wearable device, control multiple haptic feedback structures in the wearable device to press against the area in contact with the user.

[0131] For example, Figure 11 is a schematic diagram of a wearable device providing distance prompts according to some embodiments of the present disclosure. As the wearable device approaches a directional prompt distance threshold for an operable element (here, the relative distance between the wearable device and the operable element is defined as the relative distance between the wearable device and the operable element), the directional haptic feedback structure begins to inflate; as the wearable device approaches a distance prompt range threshold for the operable element, the left and right sides of the directional haptic feedback structure also gradually inflate. Within the operable range of the wearable device reaching the operable element, all haptic feedback structures of the wearable device compress the user's contact area to indicate activation of manipulation of the operable element in the extended reality scene, allowing the user to control the operable element by inputting commands.

[0132] In some embodiments, the wearable device reaching the operable range of the operable element refers to a haptic feedback structure of the wearable device reaching the operable range of the operable element.

[0133] User operation: The user input module obtains the user's operation instructions and sends the operation instructions to the electronic device corresponding to the extended reality, so that the electronic device can control the operable elements based on the operation instructions.

[0134] For example, operation instructions may include adjusting the element itself or turning on or off specific functions indicated by the operable element.

[0135] Operation complete and exit: After control of the operable elements is completed, each haptic feedback structure of the wearable device returns to the reference pressure.

[0136] In some embodiments, the haptic feedback structure is a flexible cavity, such as an airbag. The haptic feedback structure of the wearable device returns to a reference pressure by deflating the airbag to the reference pressure.

[0137] In some embodiments, the haptic feedback structure is a rigid cavity, such as a cavity that is compressed against the user area by displacement. The haptic feedback structure of the wearable device returns to the reference pressure by controlling the rigid cavity to displace to the position corresponding to the reference pressure.

[0138] The following is an embodiment of the execution progress prompt of a task to be performed provided by the present disclosure, including: wearing and calibration, execution operation, process judgment, process feedback, process completion and operation, operation completion and exit.

[0139] Wearing and calibration: After the user wears the wearable device, a pressure calibration is performed first to confirm the reference pressure of the haptic feedback structure. The reference pressure serves as the reference value without prompts.

[0140] Perform the operation: Determine if a metering or timing task exists. For example, a 5-minute timing task is set or a download task is executed.

[0141] Determine progress: Calculate and obtain the progress of task completion.

[0142] Process feedback: Based on multiple haptic feedback structures of the ring, the area that comes into contact with the user is squeezed one by one to mark the current task progress.

[0143] For example, if the task progress reaches 50%, then half of the haptic feedback structures squeeze the area in contact with the user.

[0144] Process Completion and Operation: When the process reaches 100%, all haptic feedback structures press against the area in contact with the user, prompting the user and responding to user input. Based on the user input, the module issues the next instruction. For example, the next instruction could be: performing an operation on the countdown reminder notification or further editing the downloaded task object, etc.

[0145] Operation completion and exit: After the user's instruction is completed, the multiple haptic feedback structures return to the reference pressure. At this time, the user's operation and instruction will no longer be applied to the object.

[0146] For example, Figure 12 is a schematic diagram of an execution progress prompt of a wearable device according to an embodiment of the present disclosure. The wearable device includes multiple flexible cavities and provides execution progress prompts for a task to be executed. As the execution progress of the task to be executed increases, the deformation process of the multiple flexible cavities of the wearable device includes an initial state, the start of execution progress prompts (or the start of progress prompt operation), the intermediate process of execution progress prompts, the completion of execution progress prompts, and the end of execution progress prompts. When the execution progress indicator begins, a flexible cavity in the wearable device deforms to signal the start of the progress indication. During the progress indication process, multiple flexible cavities deform sequentially, starting from the cavity where the progress indication began, in either a clockwise or counterclockwise direction (clockwise is used as an example in Figure 12), based on the execution progress of the task to be executed. When the execution progress is completed, each flexible cavity in the wearable device has deformed. At this time, the wearable device can also activate the next operation corresponding to the completion of the task to be executed (or the next operation input by the user). When the execution progress indication ends, each flexible cavity in the wearable device returns to its initial state (returning to the state before deformation).

[0147] In some embodiments, within a virtual home renovation scenario, a controllable load-bearing wall in the building structure is used as an operable element. The user is prompted based on the relative direction and distance between the wearable device and the load-bearing wall. The wearable device can also display user-manageable operations on the controllable load-bearing wall, including demolition, adding windows, adjusting height, and changing decorations. When the relative distance between the wearable device and the load-bearing wall is less than the operable distance, each haptic feedback structure aggregated by the wearable device presses against the area in contact with the user, prompting the user to perform operations on the load-bearing wall. In response to the user confirming completion of the operation on the load-bearing wall, the wearable device returns to its default non-expanded state, where it does not issue control commands corresponding to user operations.

[0148] In some embodiments, in addition to the virtual home decoration scene, users can also use wearable devices to obtain (or discover) operable elements in other game or entertainment scenes and interact with those operable elements.

[0149] In some embodiments, in the context of motion-sensing games, in response to user actions, game tasks are acquired, and the wearable device provides prompts to the user based on the progress of the game tasks through tactile feedback of pressure.

[0150] For example, the game task involves completing 30 high knees to earn a reward. The wearable device can acquire the user's motion data using sensors within the device and use multiple haptic feedback structures to press on the contact area, indicating the progress of the game task. After the user completes the task, each haptic feedback structure presses on the contact area. The user can then choose to end the task and claim the reward or continue to the next set. If the user chooses to end the task and claim the reward, the wearable device returns to its initial state. If the user chooses to continue to the next set, the wearable device returns to a progress state of 0 (at this point, one haptic feedback structure may press on the contact area, indicating the start of the task's progress) and begins acquiring the user's motion data again.

[0151] In some embodiments, within a smart healthcare scenario, wearable devices can be used to time a user's moderate-intensity heart rate (moderate-intensity heart rate ≈ (220 - age - resting heart rate) * 50% + resting heart rate). For example, since a user's exercise is most effective when they reach a moderate-intensity heart rate, continuing exercise for 10 minutes can be considered a valid exercise session. Therefore, when the user's heart rate reaches a moderate-intensity heart rate, the wearable device starts timing and controls the haptic feedback structures to squeeze the contact area with the user to indicate the timing progress. After 10 minutes, each haptic feedback structure squeezes the contact area with the user to remind them that they have completed 10 minutes of exercise. At this point, in response to the user's predefined gesture, an instruction to obtain a reward or an instruction to perform the next set of operations is generated.

[0152] In some embodiments, when downloading a large virtual model from the cloud to a local extended reality scene, the wearable device begins to prompt the user about the task progress at the start of the download. Upon completion of the download, in response to the user's gesture, the wearable device sends control commands to the electronic device corresponding to the extended reality scene to adjust the position, size, or orientation of the large virtual model. Furthermore, in response to the user's edit exit operation (or model effect confirmation operation) and corresponding gesture (e.g., an OK gesture), the user exits editing of the large virtual model and returns the wearable device to its initial state; subsequent user gestures do not change the position, size, or orientation of the large virtual model.

[0153] The disclosed embodiments can divide the device control apparatus into functional modules according to the above method embodiments. For example, each function can be divided into a separate functional module, or two or more functions can be integrated into one functional module. The integrated module can be implemented in hardware or software. It should be noted that the module division in this disclosed embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation. The following description uses the example of dividing each function into a separate functional module.

[0154] Figure 13 is a schematic diagram of a device control apparatus provided in an embodiment of this disclosure. The device control apparatus can execute the device control method provided in the above-described method embodiment. As shown in Figure 13, the device control apparatus 130 includes: an acquisition module 1301 and a control module 1302.

[0155] The acquisition module 1301 is used to acquire the instructions from the electronic device to the wearable device after the wearable device is connected to the electronic device.

[0156] The control module 1302 is used to control the working state of multiple haptic feedback structures according to instructions.

[0157] Figure 14 is a schematic diagram of another device control device provided in an embodiment of this disclosure. The device control device can execute the device control method provided in the above method embodiment. As shown in Figure 14, the device control device 140 includes: an acquisition module 1401, an identification module 1402, and a sending module 1403.

[0158] The acquisition module 1401 is used to acquire pressure information of multiple haptic feedback structures corresponding to the user's gesture actions;

[0159] The recognition module 1402 is used to recognize the type information of a hand gesture based on pressure information from multiple tactile feedback structures.

[0160] The sending module 1403 is used to send gesture action type information to the electronic device. The gesture action type information is used to trigger the electronic device to execute the control command corresponding to the gesture action type information.

[0161] In the case of implementing the functions of the integrated modules described above in hardware, this disclosure provides another structure of the device control device involved in the above embodiments. As shown in FIG15, the device control device 150 includes: a memory 1501, a processor 1502, a communication interface 1503, and a bus 1504.

[0162] The memory 1501 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions; it may be a random access memory (RAM) or other type of dynamic storage device capable of storing dynamic information and instructions; it may also be an electrically erasable programmable read-only memory (EEPROM), a disk storage medium or other magnetic storage device; or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0163] Processor 1502 may be a logic block, module, or circuit that implements or performs the various exemplary methods described in connection with embodiments of this disclosure. Processor 1502 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. Processor 1502 may also implement or perform the various exemplary logic blocks, modules, and circuits described in connection with embodiments of this disclosure. Processor 1502 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP (digital signal processor) and a microprocessor, etc.

[0164] The communication interface 1503 is used to connect to other devices via a communication network. This communication network can be Ethernet, wireless access network, wireless local area network (WLAN), etc.

[0165] In some implementations, the memory 1501 may exist independently of the processor 1502. The memory 1501 can be connected to the processor 1502 via a bus 1504 and is used to store instructions or program code. When the processor 1502 calls and executes the instructions or program code stored in the memory 1501, it can implement the device control method provided in the embodiments of this disclosure.

[0166] In some implementations, memory 1501 may also be integrated with processor 1502.

[0167] Bus 1504 can be an extended industry standard architecture (EISA) bus, etc. Bus 1504 can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used to represent bus 1504 in Figure 15, but this does not mean that there is only one bus or one type of bus.

[0168] Some embodiments of this disclosure provide a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium) storing computer program instructions that, when executed on a computer, cause the computer to perform a device control method as described in any of the above embodiments.

[0169] Exemplary examples show that the aforementioned computer-readable storage media may include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks, or magnetic tapes), optical disks (e.g., compact disks (CDs), digital versatile disks (DVDs), etc.), smart cards, and flash memory devices (e.g., erasable programmable read-only memory (EPROMs), cards, sticks, or key drives, etc.). The various computer-readable storage media described in this disclosure may represent one or more devices for storing information and / or other machine-readable storage media. The term "machine-readable storage media" may include, but is not limited to, wireless channels and various other media capable of storing, containing, and / or carrying instructions and / or data.

[0170] This disclosure provides a computer program product containing instructions that, when run on a computer, cause the computer to execute the device control method described in any of the above embodiments.

[0171] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any changes or substitutions within the technical scope disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A device control method applied to a wearable device, wherein, The wearable device includes a plurality of haptic feedback structures for providing haptic feedback to a user wearing the wearable device, and the method includes: After the wearable device is connected to the electronic device, an instruction of the electronic device to the wearable device is acquired; According to the instruction, the working state of the plurality of haptic feedback structures is controlled.

2. The method of claim 1, wherein, The instruction includes at least one of the following: execution progress of a task to be executed, relative distance between the wearable device and an operable element in an extended reality scene, relative orientation between the wearable device and the operable element.

3. The method of claim 2, wherein, The working state of the plurality of haptic feedback structures is adjusted by at least one of the following parameters: number of target haptic feedback structures, identification of target haptic feedback structures, pressure parameter of target haptic feedback structures; The target haptic feedback structure is a haptic feedback structure that causes a squeezing feeling to a region contacted by the user; The pressure parameter is used to determine the pressure of the squeezing feeling.

4. The method of claim 3, wherein, The instruction includes the execution progress of the task to be executed or the relative distance between the wearable device and the operable element, and the working state of the plurality of haptic feedback structures satisfies at least one of the following relationships: The instruction and the number of target haptic feedback structures satisfy a first preset relationship; The instruction and the pressure parameter of the target haptic feedback structure satisfy a second preset relationship.

5. The method of claim 3, wherein, The instruction includes the relative orientation between the wearable device and the operable element, and a haptic feedback structure closest to the operable element in the plurality of haptic feedback structures is the target haptic feedback structure.

6. The method of claim 1, further comprising: acquiring a relative distance between the wearable device and an operable element in an extended reality scene; in response to the relative distance being less than a preset distance threshold, acquiring pressure information of the plurality of haptic feedback structures corresponding to a gesture action of a user; based on the pressure information of the plurality of haptic feedback structures, identifying type information of the gesture action; sending the type information of the gesture action to the electronic device, the type information of the gesture action being used to trigger the electronic device to execute a control instruction corresponding to the type information of the gesture action.

7. The method of claim 1, wherein, The haptic feedback structure is a first haptic feedback structure or a second haptic feedback structure; the first haptic feedback structure provides haptic feedback to the user by deforming itself; and the second haptic feedback structure provides haptic feedback to the user by displacing itself.

8. The method of claim 7, wherein, The first haptic feedback structure includes an air bag and an inflation device for inflating the air bag.

9. A device control method applied to a wearable device, wherein, The wearable device includes a plurality of haptic feedback structures for acquiring information acted on the wearable device by a user, and the method includes: acquiring pressure information of the plurality of haptic feedback structures corresponding to a gesture action of a user; based on the pressure information of the plurality of haptic feedback structures, identifying type information of the gesture action; sending the type information of the gesture action to an electronic device, the type information of the gesture action being used to trigger the electronic device to execute a control instruction corresponding to the type information of the gesture action.

10. The method of claim 9, wherein, The control instruction is used to control an operable element in an extended reality scene managed by the electronic device. Obtaining pressure information of the plurality of haptic feedback structures corresponding to the gesture action of the user, including: Obtaining the relative distance between the wearable device and the operable element; 11. An electronic device comprising: In a case where the relative distance is less than a preset distance threshold, obtaining the pressure information of the plurality of haptic feedback structures corresponding to the gesture action of the user.

12. A computer readable storage medium, wherein, A memory and a processor; wherein the memory and the processor are coupled; the memory is used to store instructions executable by the processor; the processor executes the instructions to execute the method according to any one of claims 1-8, or execute the method according to any one of claims 9-10.

13. A computer program product comprising a computer program, wherein, The computer readable storage medium has stored computer instructions, when the computer instructions run on the computer, make the computer execute the method according to any one of claims 1-8, or execute the method according to any one of claims 9-10. The computer program is executed by the processor to implement the method according to any one of claims 1-8, or the computer program is executed by the processor to implement the method according to any one of claims 9-10.

Images