Method and device for action recognition

The bone voiceprint signal is obtained and analyzed through bone conduction sensors to identify user actions, solving the high cost and complexity problems based on image recognition in the prior art, and achieving fast and accurate motion recognition and accurate physical condition judgment.

CN118964830APending Publication Date: 2024-11-15ROBERT BOSCH GMBH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310546691.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-15
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

The existing action recognition scheme based on image recognition technology requires high processing capabilities computing devices and high-definition cameras, which leads to high cost and complex implementation, making it difficult to achieve fast and accurate action recognition.

Method used

The bone conduction sensor is used to obtain bone voiceprint signals, identify user actions by processing and analyzing signals, and generate notification information, reducing hardware cost and complexity.

Benefits of technology

It realizes the rapid and accurate identification of user actions at low cost, saves software and hardware costs, and provides accurate information about the user's physical condition.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118964830A_ABST
    Figure CN118964830A_ABST
Patent Text Reader

Abstract

The present disclosure provides a method for action recognition, comprising: obtaining a bone voiceprint signal from a bone conduction sensor, the bone voiceprint signal being associated with an action of a user; processing the bone voiceprint signal to generate a processed signal; identifying an action of the user based on the processed signal; and generating notification information in response to the action of the user satisfying a threshold condition.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates generally to motion recognition, and more particularly to a method and apparatus for recognizing a user's motion using a bone conduction sensor. Background Art

[0002] Since human actions (e.g., some instinctive actions, such as yawning, blinking, burping, coughing, sneezing, and chewing) can reflect the health and mental state of the observed object to a certain extent, some solutions have been developed to identify human actions and provide users with prompts about their physical conditions. For example, based on image recognition technology, by collecting and analyzing the driver's facial image in a vehicle, it can be determined that the driver is yawning, and then prompt that he may be in a state of fatigue driving.

[0003] However, the use of solutions based on image recognition technology often requires the provision of hardware facilities such as computing devices with high processing power and high-definition cameras. In addition, in order to achieve accurate judgment, a large amount of sample data based on various facial expression details is required, and complex software products need to be designed, which further increases the implementation cost.

[0004] Therefore, it is desirable to provide an efficient method and apparatus for recognizing human body motions, which can quickly and accurately recognize user motions at a relatively low cost. Summary of the invention

[0005] According to one aspect of the present disclosure, a method for action recognition is provided, comprising: obtaining a bone voiceprint signal from a bone conduction sensor, wherein the bone voiceprint signal is associated with a user's action; processing the bone voiceprint signal to generate a processed signal; identifying the user's action based on the processed signal; and generating notification information in response to the user's action satisfying a threshold condition.

[0006] According to another aspect of the present disclosure, a device for action recognition is provided, including: a memory; and a processor coupled to the memory, wherein the processor is configured to execute the method according to the present disclosure.

[0007] According to yet another aspect of the present disclosure, a computer-readable medium is provided, which stores a computer program including instructions, which, when executed by a processor, cause the processor to be configured to perform the method according to the present disclosure.

[0008] According to yet another aspect of the present disclosure, a device for action recognition is provided, comprising a unit for executing the method according to the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Various embodiments of the claimed subject matter will now be described by way of example with reference to the accompanying drawings. In the different drawings, the same reference numerals are used to denote the same or similar components.

[0010] Figure 1 A schematic diagram of a device 100 using a bone conduction sensor according to one embodiment of the present disclosure is shown.

[0011] Figure 2 A schematic diagram of a system 200 for identifying a user's motion using a bone conduction sensor according to an embodiment of the present disclosure is shown.

[0012] Figure 3 A flow chart of a method 300 for identifying a user's motion using a bone conduction sensor according to an embodiment of the present disclosure is shown.

[0013] Figure 4 A block diagram of a device 400 for recognizing a user's motion using a bone conduction sensor according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0014] In the following description, many specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure. However, those skilled in the relevant art will recognize that the present disclosure can be implemented without one or more specific details, or alternative methods, components, etc. can be used to implement the present disclosure. In some instances, well-known structures and operations are not shown or described in detail to avoid unnecessarily obscuring the present disclosure.

[0015] Figure 1 A schematic diagram of a device 100 using a bone conduction sensor according to an embodiment of the present disclosure is shown. In one embodiment, the device 100 may be a left earphone or a right earphone in an earbud-type earphone.

[0016] In one embodiment, the device 100 includes a microphone 110, a speaker 120, a processing unit 130, a memory 140, a communication unit 150, and a bone conduction sensor 160. The processing unit 130 is a device for processing input and output signals of the device 100. The memory 140 is communicatively coupled to the processing unit 130 and is used to store instructions associated with the operation of the processing unit 130. The communication unit 150 is used to establish a communication connection with other computing devices such as mobile phones, computers, etc. via a wired or wireless (e.g., Bluetooth, Wi-Fi, etc.) mode to transmit signals.

[0017] The bone conduction sensor 160 in the device 100 is a sensor that converts physical vibrations into electrical signals. Specifically, when the device 100 is attached to the skin of the human head, the bone conduction sensor 160 provided in the device 100 can detect the vibration information transmitted through the bones, skin and other tissues of the human head, and convert it into a corresponding electrical signal to be sent to the processing unit 130 for processing. The electrical signal is called a bone voiceprint signal. In one embodiment, the bone conduction sensor 160 can generate a bone voiceprint signal corresponding to the user's voice, and the bone voiceprint signal can be processed alone or together with the sound signal generated by the microphone 110 for purposes such as voice noise reduction and user identification.

[0018] According to an embodiment of the present disclosure, a bone voiceprint signal associated with the user's action may also be generated by the bone conduction sensor 160. By analyzing the bone voiceprint signal, the action performed by the user can be identified.

[0019] It should be understood that Figure 1 The device 100 shown in FIG. 1 is merely an example of a device including a bone conduction sensor 160 according to one embodiment of the present disclosure, and is not intended to limit the structure of the device 100 . Figure 1 The device 100 may include additional components (e.g., a bus), or one or more components of the device 100 (e.g., the microphone 110) may be omitted according to the needs of the user. The device 100 may be any form of wired or wireless headset, such as a True Wireless Stereo (TWS) headset, or other suitable wearable devices, such as hearing aids, smart glasses, etc. Figure 2 The system for identifying the user's action according to the embodiment of the present disclosure is further described.

[0020] Figure 2 A schematic diagram of a system 200 for identifying a user's motion using a bone conduction sensor according to an embodiment of the present disclosure is shown. The system 200 is composed of a state detection unit 210, a data acquisition unit 220, a signal processing unit 230, a motion recognition unit 240, and a notification generation unit 250.

[0021] The state detection unit 210 is used to detect whether the bone conduction sensor is in a state suitable for generating a bone voiceprint signal. In one embodiment, the state detection unit 210 can be used to detect whether an earbud-type headset including a bone conduction sensor is worn on the user's ear. In this embodiment, if the state detection unit 210 determines that the earbud-type headset has been worn correctly, the bone conduction sensor can be enabled to detect vibration and generate a bone voiceprint signal; on the contrary, if the state detection unit 210 determines that the earbud-type headset is not worn correctly or is far away from the user's skin, the bone conduction sensor can be disabled to further save power and computing costs.

[0022] The data acquisition unit 220 is used to acquire bone voiceprint signals generated by the bone conduction sensor, which correspond to skin vibrations generated when the user performs one or more actions (eg, yawning, blinking, burping, coughing, sneezing, and chewing, etc.).

[0023] The signal processing unit 230 is used to process the bone voiceprint signal collected by the data acquisition unit 220 to generate a processed signal. In one embodiment, the bone voiceprint signal collected by the data acquisition unit 220 may also reflect vibration information caused by other factors such as environmental noise and user voice. The signal processing unit 230 can use a filter (such as a high-pass filter) to filter the bone voiceprint signal to obtain a processed signal that can better reflect the bone voiceprint characteristics of the user's action. The processed signal can help to accurately identify the user's action.

[0024] The action recognition unit 240 is used to recognize the user's action based on the bone voiceprint signal processed by the signal processing unit 230. In one embodiment, the action recognition unit 240 compares the processed signal with the signal characteristics of the predetermined action to determine what action the user has completed. For example, the action recognition unit 240 can access a database that stores predetermined bone voiceprint signals for each of various user actions of interest. The action recognition unit 240 can determine the signal of the predetermined action that matches the processed signal by comparing the processed signal with the bone voiceprint signal of each predetermined user action in the time domain and / or frequency domain, thereby recognizing that the user has completed the predetermined action. For example, the action recognition unit 240 recognizes that the user has completed the yawning action in response to the frequency characteristics of the processed signal matching the frequency characteristics of the bone voiceprint signal of the yawning action recorded in the database.

[0025] In one embodiment, the action recognition unit 240 can recognize that the user has completed a set of actions within a period of time based on the processed signal. The set of actions can be simultaneous actions (such as yawning while blinking) or a series of actions that occur in sequence (such as yawning three times in a row, or yawning and then blinking three times in a row). Since the temporal relationship of the user's actions can reflect the physical condition to a certain extent (for example, continuous yawning for a short period of time may indicate that the user is tired), identifying the user's actions within a period of time can help to judge the user's physical condition.

[0026] In one embodiment, the action recognition unit 240 can recognize the amplitude of one or more actions of the user based on the processed signal. For example, compared with a short yawning action, a long yawning action may indicate that the user is more tired. Therefore, by recognizing the amplitude of the user's action based on the processed signal, it can also help to judge the user's physical condition.

[0027] The notification generation unit 250 is used to generate notification information in response to the identified user's action meeting the threshold condition. For example, the threshold condition can be set to detect more than three yawning actions within one minute. If the user's action identified by the action recognition unit 240 meets this threshold condition, it is determined that the user is in a fatigue state, and notification information corresponding to the state is generated. In one embodiment, the notification information can be converted into an alarm message to be sent to the user via, for example, a display or a speaker.

[0028] It should be understood that Figure 2 The components of the system 200 are merely exemplary. The embodiments according to the concepts of the present disclosure may be practiced in the absence of some of the components. For example, a system that uses a bone conduction sensor to identify the user's actions may be implemented without the state detection unit 210. In addition, the components of the system 200 may be located in the same or different computing devices. For example, the components of the system 200 may be arranged in an earbud headset, or all or part of the components of the system 200 may be arranged in a computing device that is communicatively connected to the earbud headset to further save the power and hardware costs of the earbud headset.

[0029] Compared with the solution of using image recognition technology to identify human body movements, the bone conduction sensor used in the present disclosure saves a lot of software and hardware costs. At the same time, due to the rich information contained in the bone voiceprint signal, the embodiment of the present disclosure can further provide accurate information about the user's physical condition.

[0030] Figure 3 A flow chart of a method 300 for identifying a user's motion using a bone conduction sensor according to an embodiment of the present disclosure is shown.

[0031] At step S310, method 300 includes obtaining a bone voiceprint signal from a bone conduction sensor, wherein the bone voiceprint signal is associated with a user's action. In one embodiment, the bone conduction sensor is arranged in an earbud headset, and wherein method 300 also includes enabling the bone conduction sensor in response to determining that the earbud headset has been worn on the user's ear. In one embodiment, the earbud headset is a true wireless stereo headset. In one embodiment, the user's action includes one or more of the following: yawning, blinking, hiccuping, coughing, sneezing, and chewing.

[0032] At step S320, the method 300 includes processing the bone voiceprint signal to generate a processed signal. In one embodiment, the processed signal includes a signal obtained after filtering the bone voiceprint signal.

[0033] At step S330, the method 300 includes identifying the user's action based on the processed signal. In one embodiment, the method 300 further includes: identifying that the user has completed the predetermined action in response to the characteristics of the processed signal matching the signal characteristics of the predetermined action. In one embodiment, the method 300 further includes: identifying that the user has completed a set of actions within a threshold time period based on the processed signal. In one embodiment, the method 300 further includes: identifying the magnitude of the user's action based on the processed signal.

[0034] At step S340, the method 300 includes generating notification information in response to the user's action satisfying a threshold condition. In one embodiment, the method 300 further includes: sending an alarm message based on the notification information.

[0035] Figure 4 A block diagram of a device 400 for identifying a user's motion using a bone conduction sensor according to an embodiment of the present disclosure is shown. In one embodiment, the device 400 may include a computing device, such as the headset, hearing aid, smart glasses, computer, mobile phone, etc. mentioned in the present disclosure.

[0036] The device 400 includes a processor 402 connected to an internal communication bus 408, and the processor 402 is used to execute instructions in a memory 404 to implement the method for action recognition described in detail above. Examples of processor 402 may include a central processing unit (CPU), a microcontroller, and the like. The memory 404 includes various forms of memory, such as DRAM. The device 400 may also include an input interface 410 and an output interface 412. The input interface 410 is used to receive input signals and data from an input device (such as a sensor coupled to a computing device, etc.). The output interface 412 is used to send output signals and data to an output device (such as a speaker). In addition, the device 400 may also include a non-volatile storage device 406 for tangibly storing computer program instructions and data.

[0037] The various methods, steps, operations, units, modules, components, models, and networks described in conjunction with the present disclosure may be implemented as hardware, software executed by a processor, firmware, or any combination thereof. According to one or more aspects of the present disclosure, a computer program product for action recognition may include a computer program for implementing the above-mentioned reference Figure 3 The processor may execute computer instructions for one or more of the methods or steps described. According to other aspects of the present disclosure, the computer readable medium may store computer instructions for action recognition, which, when executed by the processor, may cause the processor to perform the above-mentioned reference Figure 3 One or more of the methods or steps described. Computer-readable media includes both non-transitory computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. Any connection can be properly referred to as a computer-readable medium.

[0038] In addition to what is described herein, various modifications may be made to the embodiments and implementations of the present disclosure without departing from the scope of the embodiments and implementations of the present disclosure. Therefore, the descriptions and examples herein should be interpreted as illustrative rather than limiting. The scope of the present disclosure should be measured only by reference to the claims.

Claims

1. A method for action recognition, comprising: Obtaining a bone voiceprint signal from a bone conduction sensor, wherein the bone voiceprint signal is associated with a user's action; processing the bone voiceprint signal to generate a processed signal; identifying an action of the user based on the processed signal; and Notification information is generated in response to the user's action satisfying a threshold condition.

2. The method according to claim 1, further comprising: Responsive to a match between a characteristic of the processed signal and a signal characteristic of a predetermined action, it is identified that the user has completed the predetermined action.

3. The method according to claim 1, further comprising: Completion of a set of actions by the user within a threshold time period is identified based on the processed signal.

4. The method according to claim 1, further comprising: A magnitude of the user's motion is identified based on the processed signal.

5. The method according to claim 1, wherein: The bone conduction sensor is disposed in an earbud, and wherein the method further comprises activating the bone conduction sensor in response to determining that the earbud has been worn on the ear of the user.

6. The method according to claim 1, further comprising: An alert message is sent based on the notification information.

7. The method according to any one of claims 1 to 6, wherein: The user's action includes one or more of the following: yawning, blinking, burping, coughing, sneezing, and chewing.

8. A device for motion recognition, comprising: Memory; A processor is coupled to the memory, wherein the processor is configured to execute the method according to any one of claims 1 to 7.

9. A computer readable medium storing a computer program comprising instructions which, when executed by a processor, cause the processor to be configured to perform the method according to any one of claims 1 to 7.

10. An apparatus for action recognition, comprising a unit for executing the method according to any one of claims 1-7.