Electronic device for controlling a host device using motion signals and mouse signals

Abstract

An electronic device for pairing with a host device to control content of the host device is disclosed. An electronic device can include a touch sensing module, a movement sensing module for obtaining first movement information of the electronic device based on an acceleration sensor, a gyro sensor, and a geomagnetic sensor, and a control signal output portion for obtaining second movement information through an operation of the first movement information, determining an action corresponding to a movement of the electronic device based on the second movement information in a gesture mode, outputting an action signal representing the determined action to the host device, determining a mouse operation based on at least one of touch information obtained from the touch sensing module and the second movement information in a mouse mode, and outputting a mouse signal representing the mouse operation.

Description

Technical Field

[0001] The present invention relates to an electronic device for controlling a host device, and more particularly to an electronic device for easily and intuitively controlling various contents of a host device based on touch operation or movement in three-dimensional space. Background Technology

[0002] In recent years, the content market based on virtual reality, such as Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR), has been developing. Furthermore, with the popularization of virtual reality, there is increasing attention being paid to interfaces capable of generating and controlling virtual content. Virtual content consists of virtual objects that are played and processed in a virtual three-dimensional space; current interface devices are insufficient to meet consumer needs and are unsuitable as universal interfaces.

[0003] Traditional VR controllers are not only large, heavy, and expensive, but also require learning to operate on various functions. Furthermore, VR controllers are specifically designed for head-mounted displays (HMDs), thus limiting their usability compared to a regular mouse. Summary of the Invention

[0004] Technical issues

[0005] By using motion tracking and touch sensing in three-dimensional space, an electronic device can be provided that can simultaneously perform general mouse operations and control content operations.

[0006] The technical tasks to be achieved in this embodiment are not limited to those described above; other technical tasks can be derived from the following embodiments.

[0007] Technical problems to be solved

[0008] By using motion tracking and touch sensing in three-dimensional space, an electronic device can be provided that can simultaneously perform general mouse operations and control content operations.

[0009] Invention Effects

[0010] It can provide an electronic device that is easy to wear, intuitive, and requires no learning of how to operate.

[0011] Because the electronic device is portable, it allows for easy and precise control of various content on smartphones, TVs, computers, tablet PCs, holograms, and head-mounted displays anytime, anywhere. Attached Figure Description

[0012] Figure 1AAn electronic device (off state) for controlling a host device is shown according to an exemplary embodiment.

[0013] Figure 1B An electronic device (in an open state) for controlling a host device is shown according to an exemplary embodiment.

[0014] Figure 2A A ring-shaped device in an electronic device according to an exemplary embodiment is shown.

[0015] Figure 2B This illustrates the operation of using an electronic device to control content according to an exemplary embodiment.

[0016] Figure 3 A system including an electronic device is shown according to an exemplary embodiment.

[0017] Figure 4A This illustrates mouse operations performed by an electronic device in mouse mode, according to an exemplary embodiment.

[0018] Figure 4B This illustrates, according to an exemplary embodiment, that the front portion of an electronic device is divided into three touch areas.

[0019] Figure 4C This illustrates mouse operations identified using three regions according to an exemplary embodiment;

[0020] Figure 5 This illustrates a determination model for determining actions corresponding to a user's movement information, according to an exemplary embodiment.

[0021] Figure 6 The illustration shows a joystick using an electronic device according to an exemplary embodiment.

[0022] Figure 7 This illustrates, according to an exemplary embodiment, that in gesture mode, the electronic device controls the operation of the host device.

[0023] Figure 8 A die with an electronic device built in according to an exemplary embodiment is shown.

[0024] Figure 9 A cane with an electronic device connected according to an exemplary embodiment is shown.

[0025] Figure 10 A flowchart illustrating a method for controlling content using an electronic device according to an exemplary embodiment.

[0026] Figure 11 A flowchart illustrating a method for obtaining distance data according to an exemplary embodiment is provided.

[0027] Figure 12A flowchart illustrating a method for determining the movement of an object as a mouse movement in mouse mode or a movement action in gesture mode for an electronic device, according to an exemplary embodiment, and obtaining movement distance data.

[0028] Figure 13 A flowchart illustrating the operation of a host device to zoom in or out on content based on a movement action, according to an exemplary embodiment.

[0029] Figure 14 A flowchart illustrating a method for determining the movement of an object as a tapping action in an electronic device according to an exemplary embodiment.

[0030] Figure 15 A flowchart illustrating a method for determining the movement of an object as a grasping action for an electronic device according to an exemplary embodiment.

[0031] Figure 16 A flowchart illustrating a method for determining the movement of an object as a scrolling motion in an electronic device according to an exemplary embodiment.

[0032] Figure 17 A flowchart illustrating a method for determining the movement of an object as a sliding motion in an electronic device according to an exemplary embodiment.

[0033] Figure 18 A flowchart illustrating a method for determining the movement of an object as a rotational motion using an electronic device, according to an exemplary embodiment. Detailed Implementation

[0034] An electronic device, configured to pair with a host device to control the contents of the host device, may include: a touch sensing module; a motion sensing module for obtaining first motion information of the electronic device based on an accelerometer, a gyroscope, and a magnetometer; and a control signal output unit for obtaining second motion information through calculation of the first motion information, determining an action corresponding to the movement of the electronic device based on the second motion information in a gesture mode, and outputting an action signal representing the determined action to the host device; and in mouse mode, based on touch information obtained from the touch sensing module and the second motion information... The system determines mouse operations using at least one of the following methods and outputs mouse signals representing the mouse operations to the host device. The first movement information may include at least one of acceleration data obtained by the accelerometer and angular velocity data obtained by the gyroscope. The movement sensing module can transmit the first movement information to the control signal output unit via an internal bus. The second movement information may include at least one of angle data, distance data, speed data, and direction data. The mouse signals may include mouse click, mouse scroll, mouse move, or mouse drag. Based on the touch operation sensed by the touch sensing module, the system can perform a switch between the mouse mode and the gesture mode.

[0035] The distance data may include distance data relative to the x-axis, y-axis, and z-axis directions, respectively. The velocity data includes velocity data relative to the x-axis, y-axis, and z-axis directions, respectively. The direction data includes information about whether there is an increase or decrease along the x-axis, y-axis, and z-axis directions.

[0036] The control signal output unit can be configured to determine, in the gesture mode, a predefined action corresponding to the second movement information among the predefined actions used to control the host device, wherein the predefined actions may include movement and rotation.

[0037] The predefined action may also include user-defined actions, which can be defined by the user repeatedly holding or wearing the electronic device in their hand, and can be defined by the user corresponding the specific movement to the user-defined action.

[0038] If the user of the electronic device is a first user, the control signal output unit inputs the second movement information of the first user into a first determination model to determine the corresponding action. If the user of the electronic device is a second user, the control signal output unit inputs the second movement information of the second user into a second determination model to determine the corresponding action. The first determination model is generated based on a machine learning method that uses the second movement information of the first user and the specific action as input and output, respectively, and applies them repeatedly. The second determination model is generated based on a machine learning method that uses the second movement information of the second user and the specific action as input and output, respectively, and applies them repeatedly.

[0039] The motion sensing module may further include a sensor fusion unit, which is used to compensate and fuse the data obtained from the accelerometer, the gyroscope, and the geomagnetic sensor respectively based on filters or algorithms, thereby obtaining optimized first motion information.

[0040] The control signal output unit may include: a processor, which is configured to obtain the second movement information by calculating the first movement information, and determine the action or mouse operation corresponding to the movement of the electronic device based on the second movement information; and a communication unit, which is configured to transmit the action signal or the mouse signal to the host device based on a wireless communication interface.

[0041] The processor can obtain the second movement information every reference time, which can be less than 30ms.

[0042] The processor can obtain the velocity data and the distance data by performing an integral operation on the linear acceleration data from which the gravitational acceleration component has been removed.

[0043] The processor can obtain the angle data by performing an integral operation on the angular velocity data, and can obtain the direction data by comparing the current distance data with the previous distance data.

[0044] The control signal output unit can distinguish between when the user of the electronic device is using the keyboard and when the user is using the mouse in the mouse mode, and if it is determined that the user is using the keyboard, the mouse signal can be not output.

[0045] It may also include a battery for supplying power required for the operation of at least one of the touch sensing module, the motion sensing module, and the control signal output unit.

[0046] It may also include a housing for housing the touch sensing module, the motion sensing module and the control signal output unit, and may also include a connecting part for wearing or fixing the housing on the user's finger.

[0047] It may also include a support device for housing the housing and the connecting part, through which the battery can be charged, and a center weight may be arranged on the lower side of the support device. If the housing and the connecting part are housed in the support device, one side of the housing is exposed.

[0048] The touch sensing module may include a first touch area, a second touch area, and a third touch area. In the mouse mode, if the touch operation sensed by the touch sensing module is only a touch of the first touch area or a simultaneous touch of the first touch area and the second touch area, the mouse operation can be determined as a mouse click. If the touch operation sensed by the touch sensing module is a sequential touch of the first touch area, the second touch area, and the third touch area, the mouse operation can be determined as mouse scrolling.

[0049] Modes used in this invention

[0050] Some embodiments will now be clearly and specifically described with reference to the accompanying drawings, enabling those skilled in the art to readily implement the invention.

[0051] Additionally, the terms "section" or "module" used in this specification may refer to hardware components or circuits such as Field Programmable Gate Array (FPGA) or Application Specific Integrated Circuit (ASIC).

[0052] Hereinafter, "content" can include, but is not limited to, media itself or objects played on media, such as games, music, movies, images, animations, characters, props, objects, etc. "Content" can include operating systems or software running on a host device. Software can include document programs such as Word and PowerPoint, image processing programs for performing professional tasks, CAD programs, and games. "Content" can include virtual content generated in virtual reality such as AR / VR / MR. "Content" can include objects played on a two-dimensional screen or three-dimensional objects represented in three-dimensional space, such as holograms. "Content" can be generated, executed, or played by the host device. In the case where "content" is virtual content represented in three-dimensional space (e.g., a hologram), the physical location of the "content" may differ from that of the host device.

[0053] In this context, "motion" refers to a meaningful movement made by a user in order to control content, which can be captured, extracted, identified, analyzed, or determined from the user's movements.

[0054] Hereinafter, a "control signal" is a signal that includes information about the action itself or the type of action, and an electronic device can generate a "control signal," which the host device can use to operate or control content based on the "control signal" received from the electronic device. For example, a "control signal" can be in the form of a bit sequence, and each action can be represented by a different bit sequence.

[0055] Figure 1A An electronic device (off state) for controlling a host device is shown according to an exemplary embodiment. Figure 1B An electronic device (in an open state) for controlling a host device is shown according to an exemplary embodiment.

[0056] Users can control the host device by touch or hand movement while holding or wielding the electronic device 1000. The host device can include various types of electronic devices. For example, the host device can be any of, but is not limited to, a game console, smartphone, tablet PC, TV, desktop PC, laptop PC, mobile medical device, camera, or wearable device (e.g., electronic glasses, electronic clothing, electronic bracelet, electronic necklace, electronic jewelry, electronic tattoo, or smartwatch). For example, the host device can include a head-mounted display for displaying virtual content and a game console (e.g., a console device) for executing or playing virtual reality games or virtual reality content. The host device can also include a computer for displaying PowerPoint presentations.

[0057] Reference Figure 1A and Figure 1BThe electronic device 1000 may include a ring-shaped device 1200 that can be worn on a user's finger and a support device 1400 for storing the ring-shaped device 1200. The user can store the ring-shaped device 1200 in the support device 1400 and close the cover.

[0058] With the annular device 1200 housed in the support device 1400 and the cover of the support device 1400 closed, the front portion (head portion, the front (or one side) of the main module 1240 described later) of the annular device 1200 is exposed on the front of the support device 1400. The user can hold and move the support device 1400 to generate an action signal, or can touch the exposed front portion of the annular device 1200 to perform a touch operation. According to an embodiment, the touch sensing module may be located on the front portion of the annular device 1200.

[0059] The user can open the cover of the support device 1400 to remove the ring device 1200. The user can control the main unit by touch or gesture while wearing the ring device 1200 on their finger. The support device 1400 can be shaped for easy hand-holding, and a center of gravity weight can be positioned on the underside of the support device 1400 to lower its center of gravity. The support device 1400 may include a charging terminal and a power supply for charging the ring device 1200.

[0060] The ring device 1200 may include a motion sensor for acquiring user movement information and a touch sensor for acquiring user touch information. The ring device 1200 can generate control signals based on the acquired movement and touch information, and can output the generated control signals to a host device. The host device can control content based on the control signals received from the ring device 1200.

[0061] Figure 2A A ring-shaped device according to an exemplary embodiment is shown. Figure 2B This illustrates the operation of a user controlling content while wearing a ring-shaped device, according to an exemplary embodiment.

[0062] Reference Figure 2A and Figure 2B The ring device 1200 can be a small wearable device that can be attached to or connected to the human body or an object, or it can be worn. The ring device 1200 is easy to wear, and users can intuitively operate the functions of the ring device 1200 without any additional learning. Furthermore, the ring device 1200 utilizes motion information and touch information, and can be used as a general-purpose device like a universal mouse.

[0063] The ring device 1200 may include a connector 1220 for wearing on a user's finger 2300 and a main module 1240 for acquiring movement and touch information using sensors. The connector 1220 may be made of materials such as silicon or metal. The main module 1240 can acquire the user's touch and movement information and can output control signals corresponding to the acquired information. The main module 1240 may refer to the components of the electronic device 3000 described later and the housing containing the components. According to an embodiment, the main module 1240 may be detached from the connector 1220, and the user may insert, attach, or embed the main module 1240 into various types of objects (canes, dice, pens, etc.) and use these objects to control the main device.

[0064] The main module 1240 can acquire, process, and handle touch information and motion information (e.g., angular velocity, acceleration, speed, distance, angle, direction, and position (three-dimensional spatial coordinates) information relative to the main module 1240), and output control signals for controlling the content 2500.

[0065] Figure 2B The ring device 1200 is shown being worn on a user's finger 2300, but the ring device 1200 can be connected to or attached to other forms of objects.

[0066] For example, the main module 1240 can be built into a die and can control content 2500 based on the movement of the die. Alternatively, the main module 1240 can be attached to a cane and can control content 2500 based on the movement of the cane. Alternatively, the main module 1240 can be built into a pen and can control content 2500 on a smartphone based on the movement of the pen. Hereinafter, "object" can refer to a human body part (e.g., a finger), an item that can be worn or held in the hand, or the electronic device 3000 itself described later.

[0067] In addition, Figure 2B In this context, content 2500 is illustrated as a holographic object in three-dimensional space, but content 2500 can include any form of content or software (such as Microsoft's MS Office, games, etc.) played by the host device.

[0068] Figure 3 A system including an electronic device is shown according to an exemplary embodiment.

[0069] Reference Figure 3 System 100 may include an electronic device 3000 and a host device (or target device). The electronic device 3000 can connect to the host device via wireless communication. For example, the electronic device 3000 can pair with the host device via Bluetooth. The electronic device 3000 may refer to a device that houses... Figure 1AThe ring device 1200 has a support device 1400, and the ring device 1200 is separate from the support device 1400 or Figure 2A The main module 1240 of the ring device 1200.

[0070] Users can control various contents of the host device using the electronic device 3000. According to an embodiment, users can control the contents of the host device based on movement of the electronic device 3000 itself or objects connected to the electronic device 3000, and / or user touch operations input to the electronic device 3000. For example, a user can control various contents of the host device by wearing the electronic device 3000 on their finger and by moving their finger or touching the electronic device 3000 with their finger.

[0071] Reference Figure 3 The electronic device 3000 may include a motion sensing module 3200, a touch sensing module 3300, a communication channel 3400, and a control signal output unit 3600.

[0072] The electronic device 3000 can be operated in mouse mode or gesture mode. In mouse mode, the electronic device 3000 can be operated like a general mouse; in gesture mode, the electronic device 3000 can be operated as a motion recognition device.

[0073] In mouse mode, the electronic device 3000 can determine mouse operation based on at least one of touch operation sensed by touch sensing module 3300 and motion information sensed by motion sensing module 3200, and can output mouse signal representing mouse operation.

[0074] For example, a mouse click can be performed by touching one side of the electronic device 3000, a mouse double click can be performed by double-touching one side of the electronic device 3000 within a reference time, mouse movement can be determined by movement information of the electronic device 3000 (e.g., the second movement information described below), mouse scrolling can be a continuous touch change (e.g., scrolling up is a movement of sweeping the finger across one side of the electronic device 3000 from left to right, scrolling down is a movement of sweeping the finger across one side of the electronic device 3000 from right to left), and mouse dragging can be determined by the touch time (e.g., a long touch) and the movement information of the electronic device 3000. According to an embodiment, if the touch time is less than 200 ms, it can be determined as a short touch, and if the touch time is more than 500 ms, it can be determined as a long touch. One side of the electronic device 3000 performing the touch can serve as a reference for the touch sensing module 3300. Figure 2A The beginning of the description.

[0075] In gesture mode, the electronic device 3000 can acquire movement information using sensors and determine an action corresponding to the movement of the electronic device 3000 based on the acquired movement information. The electronic device 3000 can output an action signal representing the determined action to the host device. According to an embodiment, the movement information may include characteristics of the object's movement (e.g., at least one of angular velocity, acceleration, speed, distance, angle, direction, and position).

[0076] The operation and function of the components of the electronic device 3000 will be described below.

[0077] Reference Figure 3 The touch sensing module 3300 can sense the user's touch. For example, when the user wears the ring device 1200 on their index finger and touches the front part of the ring device 1200 with their thumb, the touch sensing module 3300 can sense the touch operation. The touch operation sensed by the touch sensing module 3300 can be transmitted to the control signal output unit 3600 through the communication channel 3400.

[0078] The touch operations sensed by the touch sensing module 3300 can be used to determine mouse operations in the aforementioned mouse mode. Alternatively, the touch operations sensed by the touch sensing module 3300 can be used to determine actions corresponding to object movement in gesture mode. Furthermore, the touch operations sensed by the touch sensing module 3300 can be used for switching between mouse mode and gesture mode.

[0079] Switching between mouse mode and gesture mode can be performed via touch operation. For example, when the touch sensing module 3300 continuously senses short touches, short touches, and long touches from the user, it can switch from gesture mode to mouse mode or vice versa. Alternatively, when the touch sensing module 3300 touches the middle portion of the front part of the ring device 1200 for a reference time or longer, it can switch from gesture mode to mouse mode or vice versa.

[0080] The motion sensing module 3200 can acquire first motion information from the electronic device 3000. This first motion information can be used in mouse mode and gesture mode. The first motion information may include at least one of acceleration data obtained by an accelerometer and angular velocity data obtained by a gyroscope sensor.

[0081] The motion sensing module 3200 may include an accelerometer 3220, a gyroscope 3240, a geomagnetic sensor 3260, and a sensor fusion unit 3280. According to an embodiment, the gyroscope 3240 is a sensor for measuring angular velocity. According to an embodiment, the accelerometer 3220 is a sensor for measuring acceleration and dynamic forces such as vibration and impact. According to an embodiment, the geomagnetic sensor 3260 is a sensor for measuring the Earth's magnetic field and detecting its magnitude.

[0082] The values ​​measured by the gyroscope sensor 3240 are affected by temperature, resulting in errors. These errors accumulate during integration, potentially causing the final value to drift. Therefore, it is necessary to compensate for the errors of the gyroscope sensor 3240 by using a temperature sensor in conjunction with it.

[0083] From a long-term viewpoint of a stationary state, the tilt angle calculated by the accelerometer 3220 represents the correct value, but due to drift accumulated over time, the gyroscope sensor 3240 may represent an incorrect value. Conversely, from a short-term viewpoint of movement, the gyroscope sensor 3240 represents the correct angular velocity, but the accelerometer 3220 may derive a calculated value different from the tilt angle. Furthermore, when the subject moves in a straight line while stationary, the tilt angle cannot be measured.

[0084] Therefore, to compensate for each drawback by using the accelerometer 3220 and gyroscope 3240, filters such as Kalman filters or compensation and fusion algorithms can be applied. However, even with such compensation and fusion operations, if only the gyroscope 3240 and accelerometer 3220 are used, the error rate increases when calculating coordinates in three-dimensional space, making it unsuitable as an interface for controlling host devices such as VR devices. Furthermore, if only the accelerometer 3220 and gyroscope 3240 are used, it is difficult to determine the absolute position of the moving subject because relative azimuth angles are used instead of absolute azimuth angles.

[0085] Therefore, since the motion sensing module 3200 also includes a geomagnetic sensor 3260, the change in absolute azimuth measured by the geomagnetic sensor 3260 is processed together with the data measured by the accelerometer 3220 and the gyroscope sensor 3240, thereby generating data with a low error rate. Because the motion sensing module 3200 includes the geomagnetic sensor 3260, it can more perfectly compensate for the cumulative drift generated in the gyroscope sensor 3240, and because the gyroscope sensor 3240 can resolve the instantaneous magnetic field fluctuation phenomenon (sudden and large changes in the magnetic field) caused by the magnetization of the geomagnetic sensor 3260, it plays a role in compensating for and mitigating the shortcomings of each other.

[0086] According to an embodiment, the motion sensing module 3200 may include a nine-axis sensor capable of accurately acquiring position data in three-dimensional space. The nine-axis sensor is a sensor composed of three axes of acceleration, two axes of gyroscope, three axes of magnetism, and one axis of temperature; it is a sensor capable of acquiring three-dimensional position and the degree of rotation along the three axes.

[0087] Reference Figure 3 The motion sensing module 3200 may include a sensor fusion unit 3280 for performing a sensor fusion operation to generate optimized position data by compensating and fusing the outputs of sensors 3220, 3240, and 3260. The sensor fusion unit 3280 can generate first motion information by optimizing data obtained from the accelerometer 3220, gyroscope 3240, and magnetometer 3260, respectively, through noise cancellation, compensation, and fusion. Accurate position data cannot be obtained by directly using the raw data obtained from sensors 3220, 3240, and 3260; therefore, optimized position data can be generated by estimating the accurate position through filters. For example, the sensor fusion operation may be performed based on filters such as Kalman filters or data compensation and fusion algorithms.

[0088] The first motion information obtained by the motion sensing module 3200 can be transmitted to the control signal output unit 3600 via the communication channel 3400. According to an embodiment, the communication channel 3400 can be an internal bus of an electronic device 3000 for transmitting the first motion information to the processor 3620. The motion sensing module 3200 and the control signal output unit 3600 can exchange data based on the bus format of the communication channel 3400. For example, the bus format can include at least one of various interface protocols such as Universal Serial Bus (USB), Serial Peripheral Interface (SPI), and Inter-Integrated Circuit (I2C).

[0089] The control signal output unit 3600 can output control signals for controlling the host device. The control signals may include motion signals and mouse signals. The control signal output unit 3600 can obtain second motion information by processing first motion information. In gesture mode, the control signal output unit 3600 can determine an action corresponding to the movement of the electronic device 3000 based on the second motion information, and can output an motion signal representing the determined action. In mouse mode, the control signal output unit 3600 can determine a mouse operation based on at least one of touch information obtained from the touch sensing module 3300 and second motion information, and can output a mouse signal representing the mouse operation. The control signal may be an interrupt signal for controlling the contents of the host device. For example, the control signal may include a bit array representing a specific mouse signal or a specific motion signal.

[0090] The control signal output unit 3600 can generate second movement information by processing the first movement information received through the communication channel 3400. The second movement information may include at least one of angle data, distance data, speed data, and direction data from the electronic device 3000. The second movement information of the electronic device 3000 can be used in mouse mode and gesture mode. For example, in mouse mode, the second movement information can be used to determine mouse movement operations of the electronic device 3000. In gesture mode, the second movement information can be used to determine various action signals output from the electronic device 3000.

[0091] According to an embodiment, the control signal output unit 3600 may include a processor 3620 and a communication unit 3640.

[0092] The processor 3620 can generate second motion information by processing first motion information received from the motion sensing module 3200 via the communication channel 3400. The second motion information may include at least one of angle data, distance data, speed data, and direction data relative to the motion. The processor 3620 can obtain the second motion information by performing calculations on the first motion information at each reference time (e.g., 5 ms). The reference time may be less than 30 ms, but is not limited thereto.

[0093] The angle data may include angle data relative to the x-axis, y-axis, and z-axis directions, respectively. According to an embodiment, the processor 3620 can obtain the angle data by performing an integral operation on the angular velocity data.

[0094] Velocity data may include velocity data relative to the x-axis, y-axis, and z-axis directions respectively. Distance data may include distance data relative to the x-axis, y-axis, and z-axis directions respectively. According to an embodiment, processor 3620 can obtain velocity data and distance data by performing an integral operation on acceleration data. Processor 3620 can obtain linear acceleration data by removing the gravitational acceleration component from the acceleration data. Processor 3620 can obtain velocity data by performing an integral operation on the linear acceleration data, and can obtain distance data by performing an integral operation on the velocity data again.

[0095] The direction data pertains to the instantaneous movement direction of an object and may include whether it increases or decreases along the x-axis, y-axis, and z-axis. According to an embodiment, the processor 3620 may include direction data based on a comparison of current distance data with previous distance data. For example, if the current distance data has an x-axis value of +50, a y-axis value of +10, and a z-axis value of -5, and the previous distance data had an x-axis value of +60, a y-axis value of +15, and a z-axis value of -10, then the processor 3620 can determine the current movement direction as increasing along the x-axis, increasing along the y-axis, and decreasing along the z-axis.

[0096] In mouse mode, the processor 3620 can determine the corresponding mouse operation based on the touch information and second movement information obtained from the touch sensing module 3300. Figure 4A This illustration shows mouse operations performed by the electronic device 3000 in mouse mode, according to an exemplary embodiment. Mouse operations may include mouse clicking, zooming in / out (or scrolling up / down), mouse movement, and mouse dragging. Mouse clicking may include single click, double click, and long click. Mouse movement may move the mouse pointer of the host device.

[0097] Reference Figure 4B In order to recognize mouse operations in mouse mode, the surface of the touch sensing module 3300 of the electronic device 3000 ( Figure 2A The front of the main module 1240 can be divided into a touch area R1 on the left, a touch area R2 in the middle, and a touch area R3 on the right. A mouse signal can be determined by sensing touch operations of at least one of the touch areas R1, R2, and R3 through the touch sensing module 3300.

[0098] Reference Figure 4CIf the user touches only touch touch area R1 or simultaneously touches touch areas R1 and R2, the corresponding action can be determined as a left mouse click. If the user touches only touch touch area R3 or simultaneously touches touch areas R3 and R2, the corresponding action can be determined as a right mouse click. If the user touches only touch touch area R2 or simultaneously touches touch areas R2, R1, and R3, the corresponding action can be determined as a switch between mouse mode and gesture mode. If the user touches touch areas R1, R2, and R3 sequentially and continuously, the corresponding action can be determined as scrolling the mouse up. If the user touches touch areas R3, R2, and R1 sequentially and continuously, the corresponding action can be determined as scrolling the mouse down.

[0099] The processor 3620 can distinguish between the user using the keyboard and the user using the mouse in the mouse mode, and if it is determined that the user is using the keyboard, it can choose not to output a mouse signal.

[0100] In gesture mode, processor 3620 can determine an action corresponding to the movement of electronic device 3000 based on second movement information. For example, processor 3620 can determine one of the predefined actions corresponding to the movement of electronic device 3000 based on the second movement information. Processor 3620 can generate an action signal representing the determined action and transmit the action signal to host device via communication unit 3640. If the distance between electronic device 3000 and host device is greater than a reference distance, or if the movement of electronic device 3000 is determined to be neither a predefined action nor a meaningful movement, processor 3620 can perform exception handling.

[0101] Predefined actions can include moving, tapping, gripping, scrolling, swiping, gestures, and rotating. A moving action is the operation of moving the electronic device 3000 in any direction; it can be used to move virtual content or perform page turning operations. For example, a moving action can include movement along three axes (x, y, z axes). A tapping action is the action of striking something; it can be used to select or click virtual content. A user can perform two consecutive taps within a reference time to double-click virtual content. A tapping action is a different operation from a click operation in mouse mode. A gripping action is the action of connecting two separate objects; it can be used to grasp virtual content. Gestures can refer to movements used to express text, symbols, or shapes (e.g., "?" or "X").

[0102] Predefined actions can be added through user-defined actions. User-defined actions are actions defined by the user, not by the manufacturer of the electronic device 3000, and the user can add specific movements input by themselves as user-defined actions. For example, the user can repeatedly make specific movements while holding or wearing the electronic device 3000, matching them with a specific function or action. The corresponding user-repeated movement information and the corresponding function or action can be stored in the electronic device 3000. For example, a user wearing the electronic device 3000 on their finger can perform 10 pricking operations, designating and storing them as pricking actions. Subsequently, if the user wears the electronic device 3000 and performs pricking operations, the processor 3620 can transmit the action signal representing the pricking action to the host device via the communication unit 3640.

[0103] The movement information of the electronic device 3000 can be matched with specific actions or predefined actions based on machine learning. That is, if specific movement information is input into a deterministic model learned through machine learning, the deterministic model can output an action signal corresponding to the input specific movement information.

[0104] According to an embodiment, the electronic device 3000 can use an independent, defined model for each user. This is because even if a user performs the same movement, the generated motion information will vary from user to user. For example, assuming that rotating an arm to draw a circle corresponds to a specific motion signal output from the electronic device 3000 or a specific function of the host device, the motion information generated when each user performs the aforementioned action can be completely different, and each user can have a unique pattern. Alternatively, the motion information generated when each user performs a leftward movement operation can be completely different, and each user can have a unique pattern.

[0105] Reference Figure 5 The first determination model DEC#1, the second determination model DEC#2, and the third determination model DEC#3 can be used to output corresponding action signals from the movement information of the first user, the second user, and the third user, respectively. If the user currently using the electronic device 3000 is the first user, the control signal output unit 3600 or the processor 3620 can input the obtained movement information of the first user (e.g., second movement information) into the first determination model DEC#1 and determine the corresponding action signal. If the user currently using the electronic device 3000 is the third user, the control signal output unit 3600 or the processor 3620 can input the obtained movement information of the third user (e.g., second movement information) into the third determination model DEC#3 and determine the corresponding action.

[0106] The deterministic model used to determine the action signal can be generated based on machine learning. For example, a first deterministic model DEC#1 can be generated by performing machine learning, where machine learning takes the first user's movement information (e.g., second movement information) and a specific action as input and output and applies it repeatedly. The first deterministic model DEC#1 can be inputted with the first user's second movement information generated by rotating their arm to draw a circle more than 10 times, and learning can be performed to make the input movement information correspond to a specific action signal. Similarly, a second deterministic model DEC#2 can be generated by performing machine learning, where machine learning takes the second user's movement information (e.g., second movement information) and a specific action as input and output and applies it repeatedly. The second deterministic model DEC#2 can be inputted with the second user's second movement information generated by rotating their arm to draw a circle more than 10 times, and learning can be performed to make the input movement information correspond to a specific action signal.

[0107] Machine learning techniques can include Support Vector Machine (SVM), Random Forest, NaiveBayes, Adaptive Boosting (AdaBoost), Gradient Boosting, K-means clustering, and Artificial Neural Network, among others.

[0108] The machine learning-based determination model used to determine the action signal can be stored in the memory (not shown) of the electronic device 3000 or in the host device. Alternatively, learning for generating the determination model can be performed in the electronic device 3000 or the host device. According to an embodiment, learning for generating the determination model can be performed in the host device, and the generated determination model can be stored in the memory (not shown) of the electronic device 3000. Alternatively, learning relative to the determination model can be performed in the electronic device 3000, and the determination model can be stored in the memory (not shown) of the electronic device 3000.

[0109] Figure 6 This illustration shows that, according to an exemplary embodiment, the electronic device 1000 is used as a joystick. (See reference...) Figure 6The electronic device 1000 is placed on a base and can be operated like a joystick. The electronic device 1000 rotates on the base, thereby enabling mouse movement operations and control of the mouse pointer. Users can perform various mouse operations by touching the front of the main module 1240 of the electronic device 1000. Mouse operation and reference. Figures 4A to 4C The descriptions are the same.

[0110] Figure 7 This illustration shows, according to an exemplary embodiment, gesture mode for controlling leftward, rightward, upward, downward, rotational, and forward / backward movements of the host device via an electronic device. Leftward and rightward movements can be determined by movement along the x-axis (+ / -). Upward and downward movements can be determined by movement along the z-axis (+ / -). Forward and backward movements can be determined by movement along the y-axis (+ / -). According to the embodiment, the user can use the gesture mode with the ring device 1200 worn on their finger, but is not limited to this.

[0111] In this embodiment, the user can perform left or right movement actions in three-dimensional space while wearing the electronic device 3000, thereby turning the pages of a document running on the host device. In gesture mode, the actions used by the electronic device 3000 to control the host device are not limited to the above embodiment. For example, actions supported by the electronic device 3000 may also include tapping, gripping, scrolling, and swiping. Furthermore, actions in gesture mode (including user-defined actions) can be added and used by the user through the aforementioned machine learning-based determination model.

[0112] Refer again Figure 3 The processor 3620 can generate control signals representing action signals or mouse signals. For example, if the user's movement is determined to be a left movement, the processor 3620 can generate a first bit sequence representing the left movement as a control signal. If the user's movement is determined to be a clockwise rotation movement, the processor 3620 can generate a second bit sequence representing the clockwise rotation movement as a control signal. Alternatively, if a protocol agreed upon with the electronic device 2000 and the host device is used, numbers assigned to each action can be generated as control signals. If the user's movement is determined to be mouse movement in mouse mode, the processor 3620 can generate a third bit sequence representing the mouse movement as a control signal.

[0113] Processor 3620 may include one processor core, or may include multiple processor cores. For example, processor 3620 may include multi-core processors such as dual-core, quad-core, or hexa-core processors. Additionally, processor 3620 may include internal or external cache memory.

[0114] The communicator 3640 can transmit control signals to the host device via a wireless communication interface. The communication unit 3640 may include a modem communication interface capable of accessing wireless local area networks (WLANs) such as Wi-Fi (Wireless Fidelity), wireless personal area networks (WPANs) such as Bluetooth, and mobile cellular networks such as wireless USB (Wireless Universal Serial Bus), Zigbee, NFC (Near Field Communication), RFID (Radio-frequency identification), or mobile cellular networks such as 3G (3rd Generation), 4G (4th Generation), and LTE (Long Term Evolution). The Bluetooth interface may support BLE (Bluetooth Low Energy).

[0115] The electronic device 3000 may include memory (not shown) required for operations performed by the electronic device 3000. For example, the electronic device 3000 may include memory (not shown) required for performing sensor fusion operations in the sensor fusion unit 3280. Additionally, the electronic device 3000 may include memory (not shown) required for storing predefined actions and / or user-defined actions, or for operations performed by the processor 3620. The memory (not shown) may store a deterministic model generated based on machine learning to determine motion signals corresponding to user movement. The memory (not shown) may include volatile storage devices such as Dynamic Random Access Memory (DRAM) and Static Random Access Memory (SRAM), and non-volatile storage devices such as flash memory devices and solid state drives (SSDs).

[0116] The electronic device 3000 may include a battery (not shown) for providing power required for operations performed in the electronic device 3000. The battery (not shown) may include, but is not limited to, a lithium-ion battery or a lithium polymer battery. For example, the battery (not shown) may be included in a control signal output unit 3600, and a portion of the power output from the battery (not shown) may be bypassed to the motion sensing module 3200.

[0117] The electronic device 3000 may include a charging terminal for charging a battery (not shown). The electronic device 3000 may include a USB-type charging terminal. Current flowing through the charging terminal can be used to charge the battery. According to an embodiment, the charging terminal may be present in… Figure 1A and Figure 1B The ring device 1200 is used, and the charging terminal may also be located in the bracket device 1400. For example, the charging terminal may be located in the main module 1240, and the ring device 1200 may be housed in the bracket device 1400, thereby enabling the main module 1240 to be charged. A power supply device for charging the main module 1240 may be mounted on the bracket device 1400.

[0118] Figure 8 A die with an electronic device built in according to an exemplary embodiment is shown.

[0119] Refer to Figure 2 to Figure 3 The electronic device 3000 is built into the dice, and users can use the dice to enjoy AR board games and other activities.

[0120] The dice may include: a sensing module that obtains first movement information of the dice based on an accelerometer, a gyroscope, and a magnetometer; a control signal output unit that generates second movement information by calculating the first movement information of the dice and outputs a control signal corresponding to the movement of the dice to a host device based on the second movement information; and a battery that provides the power required for the operation of at least one of the sensing module and the control signal output unit.

[0121] According to an embodiment, the control signal output unit may include: a processor configured to obtain second movement information by calculating first movement information, and to generate a control signal corresponding to the movement of the dice based on at least one of the second movement information; and a communication unit configured to transmit the control signal to a host device via Bluetooth communication. The control signal output unit may determine the number indicated by the dice based on at least one of acceleration data, angular velocity data, velocity data, distance data, and direction data, and may output a control signal including digital information. The control signal output from the dice may include rotational motion based on the movement of the dice and / or digital information (or digital change information) of the dice determined based on the rotational motion. The digital change of the dice may be represented by the host device in three-dimensional space.

[0122] The dice may include a hollow portion for housing the electronic device 3000. The dice may also include a sensing module, a control signal output section, and a hollow portion for housing a battery, thereby enabling the housing of the electronic device 3000.

[0123] The dice may also include charging terminals for charging the battery. For example, charging terminals for receiving charging current may be located on the surface of the dice. Therefore, the electronic device 3000 built into the dice can be charged simply by connecting a charging cable to the dice, without removing the electronic device 3000 from the dice.

[0124] Figure 9 A cane with an electronic device connected according to an exemplary embodiment is shown.

[0125] Reference Figure 3 The cane is connected to the aforementioned electronic device 3000, allowing the user to enjoy various games using the cane, such as fencing and knife fighting. According to another embodiment, see... Figure 3 The electronic device 3000 can also be built into the cane.

[0126] In addition, the electronic device 3000 can also be built into a content controller such as a joystick.

[0127] The following is for reference Figures 10 to 18 This will describe methods for controlling content using electronic devices. (See reference...) Figures 10 to 18 , can Figure 3 At least one of the electronic device 3000 and the host device performs the described method. Therefore, even if omitted below, regarding Figure 3 The content described for the electronic device 3000 or the host device may also be applied to Figures 10 to 18 Additionally, regarding Figures 10 to 18 The content of the method can also be applied to Figure 3 Electronic device 3000 or host device.

[0128] Figure 10 A flowchart illustrating a method for controlling content using an electronic device according to an exemplary embodiment.

[0129] In step S200, the electronic device can obtain first motion information of an object based on the sensing module. The object may refer to the electronic device itself. The sensing module may include an accelerometer, a gyroscope, and a geomagnetic sensor. The first motion information may include acceleration data and angular velocity data regarding the object's motion. For example, the first motion information may be data optimized by the sensor fusion unit, consisting of acceleration data obtained from the accelerometer and angular velocity data obtained from the gyroscope.

[0130] In step S400, the electronic device can generate second movement information by calculating the first movement information obtained in step S200. The second movement information may include at least one of angle data, velocity data, distance data, and direction data. When the object moves, the electronic device can calculate and obtain the second movement information in real time. For example, the electronic device can obtain the second movement information by performing calculations on the first movement information at each reference time interval (e.g., 5 ms). The reference time can be less than 30 ms, but is not limited to this. The electronic device can be worn on the middle phalanx of the index finger to determine the angle and velocity of the movement of the middle phalanx of the index finger, with the joint between the distal and middle phalanxes of the index finger as the axis. Alternatively, the electronic device can be worn on the proximal phalanx of the index finger to determine the angle and velocity of the movement of the proximal interphalanx of the index finger, with the joint between the distal and middle phalanxes of the index finger as the axis.

[0131] In step S500, the electronic device can determine whether the current mode is mouse mode or gesture mode. If the current mode is mouse mode (Yes), the electronic device can obtain touch information in step S520 and can determine a mouse signal based on at least one of the second movement information and touch information in step S540.

[0132] If the current mode is gesture mode (No), the electronic device can determine an action signal corresponding to the movement of the object in step S600 based on the obtained second movement information. According to an embodiment, the electronic device can determine the action signal corresponding to the movement of the object. According to an embodiment, the electronic device can determine the action based on the speed, angle, distance, etc., of the index finger movement. Actions can include movement (including x, y, and z axis directions), tapping, grasping, rolling, sliding, gestures, rotation, etc., but are not limited to these. Actions can include user-defined actions. If the movement of the object is determined to be a movement that is not corresponding to or meaningless in any of the actions predefined by the manufacturer or user-defined actions added by the user, the electronic device can perform exception handling without generating an action signal.

[0133] In step S800, the electronic device can transmit a control signal, representing the determined motion signal or mouse signal, to the host device via a wireless communication interface. The control signal may be an interrupt signal used to control the host device. According to an embodiment, the electronic device can determine whether the object's position is within a reference distance from the content playback location, and can only transmit the control signal to the host device if the object's position is determined to be within the reference distance. This is because if the user is far from the content, the user's movement is difficult to consider as movement for controlling the content.

[0134] In step S900, the host device can control the content based on the received control signals. For example, if the received control signal is a movement action, the host device can move the baseball in the game in a direction, speed, and distance proportional to the object's movement. If the received control signal is a tap action, the host device can select a game item. If the received control signal is a rotate action, the host device can rotate the game disc. If the received control signal is a movement action, the host device can zoom in or out of the content based on the distance between the object and the content. If the received control signal is a left movement action, the host device can scroll forward on the currently running Word or PPT document. If the received control signal is a right movement action, the host device can scroll backward on the currently running Word or PPT document. If the received control signal is a mouse movement, the host device can move the mouse pointer. If the received control signal is a mouse click, the host device can perform a click operation at the current mouse position. If the received control signal is a mouse scroll up, the host device can perform the operation corresponding to the mouse scroll up.

[0135] Figure 11 A flowchart illustrating a method for an electronic device to obtain distance data regarding the movement of an object, according to an exemplary embodiment.

[0136] In step S420, the electronic device can generate linear acceleration data by removing the gravitational acceleration component from the acceleration data. By removing the influence of gravitational acceleration from the acceleration data, acceleration data regarding the movement of the object can be obtained.

[0137] In step S440, the electronic device can perform an integral operation on the linear acceleration data to obtain velocity data.

[0138] In step S460, the electronic device can perform an integral operation on the speed data to obtain the distance data.

[0139] Figure 12A flowchart illustrating a method for determining the movement of an object as a mouse movement in mouse mode or a movement action in gesture mode, according to an exemplary embodiment, and obtaining movement distance data. Figure 12 It can represent Figure 10 The sub-steps of step S540 or S600.

[0140] In step S612, the electronic device can determine the angle and velocity of the initial movement of the object. The electronic device can obtain angle data and velocity data regarding the initial movement of the object after the movement has started (e.g., within a reference time after the movement has started).

[0141] In step S614, the electronic device can determine whether the angle data and velocity data obtained in step S612 meet the reference conditions. For example, if the velocity is equal to or greater than the reference value and the change in angle is within 20 degrees, the electronic device can determine that the object's movement is a movement along a straight line. If the angle data and velocity data do not meet the reference conditions (No), the electronic device can determine whether the object's movement corresponds to another action, or if it is determined that it does not belong to any action, it can perform exception handling (S618).

[0142] If the angle and speed data meet the reference condition (Yes), the electronic device can determine the movement as a movement action or mouse movement, and can obtain distance data about the object's movement (S616). For example, the electronic device can determine the time point at which it moves at a preset speed or greater in the desired direction of movement as the initial point of the movement. (Refer to...) Figure 11 The distance data about the object's movement can be determined using the methods described above.

[0143] According to an embodiment, in addition to distance data, the electronic device can also obtain position data (e.g., the three-dimensional spatial coordinates of an object) and orientation data.

[0144] When the position of an object at any point in time or in any space is used as a reference point, the electronic device can obtain the current position data of the object based on the distance data of the object's movement. The electronic device can determine the distance traveled for each unit of movement of the object and can store the determined distance data in memory. The electronic device can read the distance data about the movement from memory and can determine the current position data of the object by adding the read distance data together.

[0145] For example, if the object's position data at any previous time point is (0, 0, 0) and three consecutive movement actions occur, and the first distance data for the first movement action is (10, -20, 30), the second distance data for the second movement action is (-10, -30, -10), and the third distance data for the third movement action is (20, 100, 100), the electronic device can determine the current object's position data as (20, 50, 120). For example, if a first movement action occurs, the object simply moves with the person's position, and then another movement action occurs, and the first distance data for the first movement action is (5, 30, 20), the second distance data for the simple movement is (500, 500, 0), and the second distance data for the second movement action is (10, 30, 30), the current object's position data can be determined as (515, 560, 50).

[0146] The electronic device can obtain the direction data of an object's movement based on a comparison between current distance data and previous distance data. Previous distance data can refer to previously acquired distance data. For example, when calculating distance data every 5ms, the device can determine whether there is an increase or decrease along the x-axis, y-axis, and z-axis based on the distance data at time point t and the distance data acquired at time point t-5ms. For instance, if the current distance data has an x-axis value of +50, a y-axis value of +10, and a z-axis value of -5, and the previous distance data had an x-axis value of +60, a y-axis value of +15, and a z-axis value of -10, then the electronic device 3000 can determine the current direction of movement as increasing along the x-axis, increasing along the y-axis, and decreasing along the z-axis.

[0147] Figure 13 A flowchart illustrating the operation of a host device to zoom in or out on content based on a movement action, according to an exemplary embodiment. Figure 14 A flowchart can represent Figure 10 The sub-step of step S900.

[0148] In step S920, the host device can determine whether the control signal received from the electronic device is a movement action. (See reference...) Figure 7 The movement action can be either a forward or reverse movement action as described above. If the received control signal is not a movement signal (No), the host device can perform another operation corresponding to the received control signal (S930).

[0149] If the received control signal is a movement signal (Yes), then in step S940, it can be determined whether the distance between the content and the object has decreased. If the content is being played on the host device's display, the position of the content can be the same as the position of the host device itself; if the content is virtual reality content played by the host device, the position of the content can be different from the position of the host device.

[0150] If it is determined that the distance between the object's position, determined by the object's movement distance, and the content's position has decreased (Yes), the host device can zoom in on the content (S950). If it is determined that the distance between the object's position and the host device's position has increased (No), the host device can zoom out on the content (S960). However, Figure 14 The flowchart above is merely one embodiment of a host device performing zoom-in / zoom-out operations based on movement actions, and the zoom-in / zoom-out operation can be defined as another movement action. For example, zooming out can be performed if the content is close to the objects, and zooming in can be performed as the distance increases. Additionally, zooming in can be performed if the objects move to the right (or left), and zooming out can be performed if the objects move to the left (or right). Furthermore, the electronic device can determine the distance between the content and the objects and, based on the determination result, output a control signal to the host device as a zoom-in / zoom-out signal instead of a movement signal. In this case, it is possible to... Figure 10 Steps S600 and S800 perform operations to determine the distance between the content and the object and to transmit the zoom-in / zoom-out signal.

[0151] the following, Figures 14 to 18 A flowchart illustrating a method for determining a specific motion signal of a control host device from movement information of an object in gesture mode, according to an exemplary embodiment.

[0152] Figure 14 A flowchart illustrating a method for determining the movement of an object as a tapping (or clicking) action for an electronic device according to an exemplary embodiment. Figure 14 A flowchart can represent Figure 10 The sub-step of step S600.

[0153] In step S624, the electronic device can determine whether the angle data and speed data regarding the movement meet the reference conditions. The electronic device can determine whether the angle is within a first reference range and whether the speed is within a second reference range.

[0154] If the angle and speed meet the reference conditions (Yes), then in step S626, the electronic device can determine that the movement of the object is a tapping action. Otherwise (No), in step S628, the electronic device can determine whether the movement of the object corresponds to another action, or perform exception handling if it is determined that it does not belong to any action.

[0155] Figure 15 A flowchart illustrating a method for determining the movement of an object as a grasping action for an electronic device according to an exemplary embodiment. Figure 16 A flowchart can represent Figure 10 The sub-step of step S600.

[0156] In step S634, the electronic device can determine whether the angle and speed meet the reference conditions and whether the movement is echoless. For example, the electronic device can determine whether the angle is within a first reference range and the speed is within a second reference range. Simultaneously, the electronic device can determine whether there is an echo related to the movement of the object. An echo refers to the movement (or jerking) of an object (e.g., a finger) that occurs independently of the user's intention, based on the object's characteristics or movement inertia, even after the user has finished moving. For example, an echo can refer to the residual signal caused by the inertia of an accelerometer. Grasping actions differ from tapping actions; because the object (e.g., the index finger) contacts or engages with other objects (e.g., the thumb), the movement ends without an echo. Therefore, the presence or absence of an echo can serve as a reference for distinguishing between tapping and grasping actions.

[0157] If the angle and speed meet the reference conditions and there is no response (Yes), then in step S636, the electronic device can determine the movement of the object as a grasping action. Otherwise (No), in step S638, the electronic device can determine whether the movement of the object corresponds to another action, or if it is determined that it does not belong to any action, it can perform exception handling.

[0158] Figure 16 A flowchart illustrating a method for determining the movement of an object as a scrolling motion in an electronic device according to an exemplary embodiment. Figure 16 A flowchart can represent Figure 10 The sub-step of step S600.

[0159] In step S644, the electronic device can determine whether the angle, velocity, and distance meet reference conditions. For example, the electronic device can determine whether the angle is within a first reference range, the velocity is within a second reference range, and the distance is within a third reference range. For example, the electronic device can determine whether the angular velocity relative to any one of the x, y, and z axes is equal to or greater than a reference value, and whether the angle relative to at least one of the x, y, and z axes is within a reference range. Simultaneously, the electronic device can determine whether the distance (e.g., the distance moved by the middle or proximal phalanx of the index finger) is greater than a threshold. Considering that the finger movement in a rolling motion is greater than that in a tapping motion, distance can serve as a reference for distinguishing between tapping and rolling motions.

[0160] If the angle, speed, and distance meet the baseline conditions (Yes), then in step S646, the electronic device can determine that the object's movement is a rolling motion. Otherwise (No), in step S648, the electronic device can determine whether the object's movement corresponds to another motion, or, if determined not to belong to any motion, can perform exception handling.

[0161] Figure 17 A flowchart illustrating a method for determining the movement of an object as a sliding motion in an electronic device according to an exemplary embodiment. Figure 17 A flowchart can represent Figure 10 The sub-step of step S600.

[0162] In step S654, the electronic device can determine whether the speed and distance meet the reference conditions. For example, the electronic device can determine whether the speed is within a first reference range and whether the distance is within a second reference range. Considering that the movement of a sliding motion is greater than that of a moving motion, whether both the speed and distance are greater than a threshold can serve as a reference for distinguishing between sliding and moving motions.

[0163] If the speed and distance of the movement meet the baseline condition (Yes), then in step S656, the electronic device can determine that the movement of the object is a sliding movement. Otherwise (No), in step S658, the electronic device can determine whether the movement of the object corresponds to another movement, or if it is determined that it does not belong to any movement, it can perform exception handling.

[0164] Figure 18 A flowchart illustrating a method for determining the movement of an object as a rotational motion using an electronic device, according to an exemplary embodiment. The flowchart in Figure 19 can represent... Figure 10 The sub-step of step S600.

[0165] In step S664, the electronic device can determine whether the angles meet the reference conditions. For example, if the movement angles relative to the x-axis, y-axis, and z-axis respectively meet all the reference conditions (Yes), then in step S666, the electronic device can determine the movement of the object as a rotational action. Otherwise (No), in step S668, the electronic device can determine whether the movement of the object corresponds to another action, or if it is determined that it does not belong to any action, it can perform exception handling.

[0166] On the one hand, the aforementioned content control method can be implemented as computer-readable code on a computer-readable recording medium. Computer-readable recording media include all types of recording devices storing data that can be read by a computer system. Examples of computer-readable recording media include read-only memory, random access memory, CD-ROM, magnetic tape, floppy disk, optical data storage devices, etc. Furthermore, computer-readable recording media can be distributed across computer systems connected via a network, allowing processor-readable code to be stored and executed in a distributed manner.

[0167] The foregoing description is intended to provide exemplary configurations and operations for implementing the present invention. The spirit of the present invention includes not only the embodiments described above, but also implementations obtained by simply changing or modifying the embodiments described above. Furthermore, the spirit of the present invention will include implementations that can be easily achieved in the future by changing or modifying the embodiments described above.

Claims

1. An electronic device, characterized in that, The electronic device is used to pair with a host device to control the contents of the host device, including: A touch sensing module, the touch sensing module including a surface divided into a first touch area, a second touch area and a third touch area; A motion sensing module, which is used to obtain first motion information of the electronic device in three-dimensional space based on an accelerometer, a gyroscope, and a geomagnetic sensor; A control signal output unit is configured to obtain second movement information through the calculation of the first movement information. In gesture mode, the control signal output unit determines an action corresponding to the movement of the electronic device based on the second movement information and outputs an action signal representing the determined action to the host device. In mouse mode, the control signal output unit determines a mouse operation based on at least one of touch information obtained from the touch sensing module and the second movement information and outputs a mouse signal representing the mouse operation to the host device. Housing, the housing being used to house the touch sensing module, the motion sensing module, and the control signal output unit; and A support device for receiving the housing and closing or opening the lid to remove the housing; The first movement information includes at least one of acceleration data obtained by the accelerometer and angular velocity data obtained by the gyroscope sensor. The motion sensing module transmits the first motion information to the control signal output unit via an internal bus. The second movement information includes at least one of angle data, distance data, speed data, and direction data. The mouse signals include mouse click, mouse scroll, mouse move, or mouse drag. Based on the touch operation sensed by the touch sensing module, the system performs a switch between the mouse mode and the gesture mode. A center-of-gravity hammer is disposed on the lower side of the support device. The housing is housed within the support device. When the cover is closed, the first touch area, the second touch area, and the third touch area for touch operation are exposed. The user holds the support device in their hand and moves or touches the exposed first touch area, second touch area, or third touch area, thereby outputting the action signal or the mouse signal. The user moves the housing or touches the first touch area, second touch area, or third touch area while wearing the housing detached from the support device on their finger, thereby outputting the action signal or the mouse signal. In the mouse mode, the control signal output unit determines the mouse movement from the second movement information. If the touch operation sensed by the touch sensing module is only a touch of the first touch area or a simultaneous touch of the first touch area and the second touch area, the mouse operation is determined as a mouse click. If the touch operation sensed by the touch sensing module is a sequential touch of the first touch area, the second touch area, and the third touch area, the mouse operation is determined as mouse scrolling.

2. The electronic device according to claim 1, characterized in that, The distance data includes distance data relative to the x-axis, y-axis, and z-axis directions, respectively. The speed data includes speed data relative to the x-axis, y-axis, and z-axis directions, respectively. The direction data includes information about whether there is an increase or decrease along the x-axis, y-axis, and z-axis directions. If the housing is moved while the touch sensing module is touched for a longer time than the reference time, the control signal output unit determines the mouse operation as mouse dragging.

3. The electronic device according to claim 1, characterized in that, The control signal output unit is configured to determine, in the gesture mode, a predefined action corresponding to the second movement information among the predefined actions for controlling the host device. The predefined actions include movements for turning pages in the host device and user-defined actions. The user-defined actions are defined by the user repeatedly holding or wearing the electronic device in their hand, and the user makes the specific movements correspond to the user-defined actions.

4. The electronic device according to claim 3, characterized in that, If the user of the electronic device is a first user, the control signal output unit inputs the second movement information of the first user into a first determination model to determine the corresponding action; if the user of the electronic device is a second user, the control signal output unit inputs the second movement information of the second user into a second determination model to determine the corresponding action. The first determining model is generated based on a machine learning method that repeatedly applies the second movement information and specific actions of the first user as input and output, respectively. The second determining model is generated based on a machine learning method that repeatedly applies the second movement information and specific actions of the second user as input and output, respectively.

5. The electronic device according to claim 1, characterized in that, The motion sensing module further includes a sensor fusion unit, which is used to compensate and fuse data obtained from the accelerometer, the gyroscope, and the geomagnetic sensor respectively based on a filter, thereby obtaining optimized first motion information.

6. The electronic device according to claim 1, characterized in that, The control signal output unit includes: a processor, which is configured to obtain the second movement information by calculating the first movement information, and determine the action or mouse operation corresponding to the movement of the electronic device based on the second movement information; and a communication unit, which is configured to transmit the action signal or the mouse signal to the host device based on a wireless communication interface.

7. The electronic device according to claim 6, characterized in that, The processor obtains the second movement information every reference time interval, wherein the reference time interval is less than 30ms.

8. The electronic device according to claim 6, characterized in that, The processor obtains the velocity data and the distance data by performing an integral operation on the linear acceleration data from which the gravitational acceleration component has been removed.

9. The electronic device according to claim 6, characterized in that, The processor obtains the angle data by performing an integral operation on the angular velocity data, and obtains the direction data by comparing the current distance data with the previous distance data.

Images