A method and device for movement recognition
The use of a bone conduction sensor to process bone voiceprint signals addresses the inefficiencies of image recognition technologies by providing a cost-effective method for recognizing human body movements, enabling accurate detection and notification of user conditions.
Patent Information
- Application Number
- PCT/EP2024/061731
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-15
- Filing Date
- 2024-04-29
- Publication Date
- 2024-11-21
AI Technical Summary
Existing methods for recognizing human body movements, such as image recognition technology, require costly hardware and software infrastructure, making them inefficient and expensive for accurate movement detection.
A method and device utilizing a bone conduction sensor to obtain and process bone voiceprint signals, which are associated with user movements, allowing for efficient recognition and generation of notification information based on threshold conditions without the need for high-processing power devices.
Enables accurate and cost-effective recognition of user movements, reducing hardware and software costs while providing rich information about the user's physical condition through the analysis of bone voiceprint signals.
Smart Images

Figure EP2024061731_21112024_PF_FP_ABST
Abstract
Description
[0001] A Method and Device for Movement Recognition
[0002] Technical Field
[0003] The present disclosure generally relates to movement recognition and more specifically to a method and a device for recognizing a user’s movements using a bone conduction sensor.
[0004] Background Art
[0005] Since human body movements (e.g., instinctive movements such as yawning, blinking, burping, coughing, sneezing, and chewing) can reflect to some extent the health and mental condition of an observed subject, several solutions have been developed to recognize human body movements, and thus provide the user with prompts about their physical condition. For example, based on image recognition technology, by collecting and analyzing the driver’s facial images in vehicles, it can be determined if the driver is yawning, thereby alerting them to the possibility of being in a fatigued driving state.
[0006] However, the adoption of image recognition technology-based solutions often requires hardware facilities such as computing devices with high processing power and high-definition cameras. Additionally, to achieve accurate judgment, a large amount of sample data based on multiple facial expression details is needed to design complex software products, further increasing the cost of implementation.
[0007] Accordingly, it is hoped that an efficient method and device for recognizing human body movements can be provided to quickly and accurately identify user movements at a lower cost.
[0008] Summary of the Invention
[0009] According to one aspect of the present disclosure, a method for movement recognition is provided, comprising: obtaining bone voiceprint signals from a bone conduction sensor, wherein the bone voiceprint signals are associated with a user’s movements; processing the bone voiceprint signals to generate processed signals; recognizing the user’s movements based on the processed signals; and generating notification information in response to the user’s movements meeting threshold conditions.
[0010] According to another aspect of the present disclosure, a device for movement recognition is provided, comprising: a memory; a processor coupled to the memory, with the processor configured to execute the method described herein.
[0011] According to yet another aspect of the present disclosure, there is provided a computer-readable medium storing a computer program comprising instructions, that, when executed by the processor, cause the processor to be configured to execute the method described herein.
[0012] According to yet another aspect of the present disclosure, there is provided a device for movement recognition, comprising a unit configured to execute the method described herein.
[0013] Description of Drawings
[0014] The various examples of the subject matter to be protected are described by way of typical examples with reference to the accompanying drawings. The same reference numbers are used in different accompanying drawings to denote the same or similar components.
[0015] FIG. 1 shows a schematic diagram of a device 100 using a bone conduction sensor according to one example of the present disclosure.
[0016] FIG. 2 shows a schematic diagram of a system 200 for user movement recognition using a bone conduction sensor according to one example of the present disclosure.
[0017] FIG. 3 shows a flowchart of a method 300 for user movement recognition a using bone conduction sensor according to one example of the present disclosure.
[0018] FIG. 4 shows a block diagram of a device 400 for recognizing a user’s movement using a bone conduction sensor according to an example of the present disclosure.
[0019] Specific Embodiments
[0020] In the following description, numerous specific details are set forth to provide a thorough understanding of the examples of the present disclosure. However, those skilled in the art will recognize that the present disclosure can be implemented without one or more specific details, or by using alternative methods, components, etc. In some instances, well-known structures and operations are not shown or described in detail to avoid unnecessarily complicating the understanding of the present disclosure.
[0021] FIG. 1 shows a schematic diagram of a device 100 using a bone conduction sensor according to one example of the present disclosure. In an example, the device 100 may be a left headset or a right headset of an in-ear headset.
[0022] In an example, the device 100 comprises a microphone 110, a speaker 120, a processor 130, a memory 140, a communication unit 150, and a bone conduction sensor 160, wherein the processor 130 is a device that processes the input and output signals of the device 100. The memory 140 is communicatively coupled to the processor 130 for and is used to store instructions associated with operation of the processor 130. The communication unit 150 is used to establish a communication connection via wired or wireless means (e.g., Bluetooth, Wi-Fi, etc.) with other computing devices such as cell phones, computers, etc. to transmit signals.
[0023] The bone conduction sensor 160 in the device 100 is a sensor that converts physical vibrations into electrical signals. Specifically, when the device 100 is in contact with the skin of the head of the human body, the bone conduction sensor 160 disposed in the device 100 can detect vibrational information transmitted through the bones, skin, and other tissues of the head of the human body, and convert it into corresponding electrical signals to be sent to the processor 130 for processing. These electrical signals are referred to as bone voiceprint signals. In one example, the bone conduction sensor 160 is capable of generating bone voiceprint signals corresponding to the user’s voice, which can be processed alone or in conjunction with acoustic signals generated by the microphone 110 for purposes such as voice noise reduction and user identification.
[0024] According to one example of the present disclosure, bone voiceprint signals associated with the user’s movements may also be generated via the bone conduction sensor 160. Analysis of the bone voiceprint signals enables the identification of the user’s movements.
[0025] It should be understood that the device 100 shown in FIG. 1 is merely illustrative of one example of the device comprising the bone conduction sensor 160 according to the present disclosure, and is not intended to limit the structure of the device 100. The device 100 in FIG. 1 may comprise additional components (e.g., a bus), or one or more components of the device 100 (e.g., the microphone 110) may be omitted depending on the needs of the user. The device 100 may be any form of wired or wireless headset, such as a true wireless stereo (TWS) headset, or other suitable wearable device, such as a hearing aid, smart glasses, etc. A system for user movement recognition according to examples of the present disclosure will be further described below in conjunction with FIG. 2.
[0026] FIG. 2 shows a schematic diagram of a system 200 for user movement recognition using a bone conduction sensor according to one example of the present disclosure. The system 200 consists of a status detection unit 210, a data collection unit 220, a signal processor 230, a movement recognition unit 240, and a notification generation unit 250.
[0027] The status detection unit 210 is used to detect whether the bone conduction sensor is in a state suitable for generating bone voiceprint signals. In one example, the status detection unit 210 can be used to detect whether in-ear headset comprising a bone conduction sensor is worn in the user’s ear. In this example, if the status detection unit 210 determines that the in-ear headset is worn correctly, the bone conduction sensor may be activated to detect vibrations and generate bone voiceprint signals; conversely, if the status detection unit 210 determines that the in-ear headset is not worn properly or is away from the user's skin, the bone conduction sensor may be deactivated to further save power and computational costs.
[0028] The data collection unit 220 is used to collect bone voiceprint signals generated by the bone conduction sensor. The bone voiceprint signals correspond to skin vibrations generated when the user performs one or more actions (e.g., yawning, blinking, burping, coughing, sneezing, and chewing, etc.).
[0029] The signal processor 230 is used to process the bone voiceprint signals collected by the data collection unit 220 to generate processed signals. In an example, the bone voiceprint signals collected by the data collection unit 220 may also reflect vibrational information caused by other factors such as environmental noise, user speech, etc. The signal processor 230 can use a filter (e.g. a high-pass filter) to filter the bone voiceprint signals, thereby obtaining processed signals that better reflect the bone voiceprint characteristics of user movements. The processed signals can contribute to the accurate recognition of user movements. The movement recognition unit 240 is used for user movement recognition based on the bone voiceprint signals processed by the signal processor 230. In one example, the movement recognition unit 240 compares the processed signals with signal characteristics of predetermined movements to determine which movement the user has completed. For example, the movement recognition unit 240 may access a database containing predetermined bone voiceprint signals for each of the various user movements of interest. The movement recognition unit 240 may identify that the user has completed a predetermined movement by comparing processed signals with the bone voiceprint signals of each of the predetermined user movements on the time domain and / or frequency domain to determine the signals of the predetermined movement matching the processed signals. For example, the movement recognition unit 240 identifies that the user has completed a yawning movement in response to the frequency characteristics of the processed signals matching the frequency characteristics of the bone voiceprint signals for a yawning movement recorded in the database.
[0030] In one example, the movement recognition unit 240 may identify that the user has completed a set of movements over a period of time based on the processed signals. The set of movements may be simultaneous (e.g., blinking while yawning), or a series of sequential movements (e.g., yawning three times in a row or yawning once followed by three consecutive blinks). As the temporal sequence of user movements can reflect the user’s physical condition to some extent (e.g., continuous yawning within a short period may indicate that the user is fatigued), recognizing user movements over a period of time can help assess the user’s physical condition.
[0031] In one example, the movement recognition unit 240 may identify the magnitude of one or more user movements based on the processed signals. For example, compared to a brief yawning movement, a yawning movement with a longer duration may indicate that the user is more fatigued. Therefore, by identifying the magnitude of user movements based on processed signals, can also help assess the user’s physical condition.
[0032] The notification generation unit 250 is used to generate notification information in response to identified user movements satisfying a threshold condition. For example, a threshold condition could be set as the detection of yawning movements three times or more within one minute. If the user’s movements identified by the movement recognition unit 240 meet this threshold condition, the user is determined to be in a fatigued state, and a notification message corresponding to this state is generated. In one example, this notification information can be converted into alert messages and sent to the user via devices such as a display screen or speaker.
[0033] It should be understood that the constituent units of the system 200 in FIG. 2 are only exemplary. Embodiments based on the disclosed concept may be practiced even in the absence of some constituent units. For example, a system using a bone conduction sensor for user movement recognition can be implemented even without a status detection unit 210. Additionally, the various constituent units of the system 200 may be located in the same or different computing devices. For example, various constituent units of the system 200 may be disposed in an in-ear headset, or all or some of the constituent units of the system 200 may be disposed in a computing device communicatively connected to the in-ear headset to further save power and hardware costs of the in-ear headset.
[0034] Compared with solutions that use image recognition technology to identify human body movements, the bone conduction sensor used in this disclosure saves a lot of software and hardware costs. At the same time, examples of the present disclosure are able to further provide accurate information about the physical condition of the user due to the rich information contained in bone voiceprint signals.
[0035] FIG. 3 shows a flowchart of a method 300 for user movement recognition a using bone conduction sensor according to one example of the present disclosure.
[0036] At step S310, the method 300 comprises obtaining bone voiceprint signals from a bone conduction sensor, wherein the bone voiceprint signals are associated with the user’s movements. In one example, the bone conduction sensor is disposed in an in- ear headset, and the method 300 further comprises activating the bone conduction sensor in response to determining that the in-ear headset has been worn in the ear of the user. In an example, the in-ear headset is a TWS headset. In one example, the movements of the user comprise one or more of the following: yawning, blinking, burping, coughing, sneezing, and chewing.
[0037] At step S320, method 300 comprises processing bone voiceprint signals to generate processed signals. In one example, the processed signals comprise signals obtained by filtering bone voiceprint signals.
[0038] At step S330, the method 300 comprises identifying the user's movements based on the processed signals. In one example, the method 300 further comprises: in response to matching a feature of the processed signals with a feature of the signal of a predetermined movement, recognizing that the user has completed the predetermined movement. In one example, the method 300 further comprises: recognizing, based on the processed signals, that the user has completed a set of movements within a threshold period of time. In one example, the method 300 further comprises: recognizing, based on the processed signals, the magnitude of the user’s movements.
[0039] At step S340, the method 300 comprises generating a notification information in response to identified user movements satisfying a threshold condition. In one example, the method 300 further comprises: sending an alert message based on the notification information.
[0040] FIG. 4 shows a block diagram of a device 400 for recognizing a user’s movement using a bone conduction sensor according to an example of the present disclosure. In one example, the device 400 may comprise computing devices such as headsets, hearing aids, smart glasses, computers, mobile phones, and so on, as mentioned in this disclosure.
[0041] The device 400 comprises a processor 402 connected to an internal communication bus 408, with the processor 402 being used to execute instructions in the memory 404 to implement the movement recognition method described in detail above. Examples of the processor 402 may comprise a central processing unit (CPU), a microcontroller, etc. The memory 404 comprises various forms of memory, such as DRAM. The device 400 may also comprise an input interface 410 and an output interface 412. The input interface 410 is used to receive input signals and data from an input device (e.g., a sensor coupled to a computing device, etc.). The output interface 412 is used to send output signals and data to an output device (e.g., a speaker). Additionally, the device 400 may also comprise a non-volatile storage device 406 for tangibly storing computer program instructions and data.
[0042] The various methods, steps, operations, units, modules, components, models, and networks described in conjunction with the present disclosure may be implemented as hardware, software executed by a processor, firmware, or any combination thereof. According to one or more aspects of the present disclosure, a computer program product for movement recognition may comprise processorexecutable computer instructions for implementing one or a plurality of the methods or steps described above with reference to FIG. 3. According to other aspects of the present disclosure, the computer-readable medium can store computer instructions for movement recognition, which, when executed by the processor, cause the processor to perform one or a plurality of the methods or steps described above with reference to FIG. 3. Computer-readable media comprises both non-transitory computer storage media and communication media, with communication media encompassing any media that facilitate the transfer of computer programs from one location to another. Any connection may be appropriately referred to as a computer- readable medium. In addition to the content described herein, various modifications can be made to the examples and implementations of the present disclosure without departing from the scope of the examples and implementations of the present disclosure. Therefore, the description and examples herein should be interpreted as illustrative and not restrictive. The scope of the present disclosure should only be determined by reference to the Claims.
Claims
Claims1 . A method for movement recognition, comprising: obtaining bone voiceprint signals from a bone conduction sensor, wherein the bone voiceprint signals are associated with a user’s movements; processing the bone voiceprint signals to generate processed signals; recognizing the user’s movements based on the processed signals; and generating notification information in response to the user’s movements meeting threshold conditions.
2. The method according to Claim 1 , further comprising: in response to matching a feature of the processed signals with a feature of the signal of a predetermined movement, recognizing that the user has completed the predetermined movement.
3. The method according to Claim 1 , further comprising: recognizing, based on the processed signals, that the user has completed a set of movements within a threshold period of time.
4. The method according to Claim 1 , further comprising: recognizing, based on the processed signals, the magnitude of the user’s movements.
5. The method according to Claim 1 , wherein the bone conduction sensor is disposed in an in-ear headset, and wherein the method further comprises activating the bone conduction sensor in response to determining that the in-ear headset has been worn in the ear of the user.
6. The method according to Claim 1 , further comprising: sending an alert message based on the notification information.
7. The method according to any one of Claims 1 - 6, wherein the user’s movements comprise one or more of the following: yawning, blinking, burping, coughing, sneezing, and chewing.
8. A device for movement recognition, comprising:a memory; a processor which is coupled to the memory, with the processor configured for performing the method according to any one of Claims 1 - 7.
9. A computer-readable medium storing a computer program comprising instructions, that, when executed by the processor, cause the processor to be configured to execute the method according to any one of Claims 1 - 7.
10. A device for movement recognition, comprising a unit configured to execute the method according to any one of Claims 1 - 7.
