Motion analysis method and device based on sound feature recognition, terminal and medium

Abstract

The invention discloses a motion analysis method and device based on sound feature recognition, a terminal and a medium, and the method comprises the steps: when a user moves, determining a sound feature period based on a sound signal collected in real time by a sound collection module, the sound feature period being used for reflecting a change rule of the sound signal, the sound acquisition module is arranged on one side of the motion auxiliary equipment; based on the sound characteristic period, determining a gait period corresponding to the sound characteristic period, the gait period being used for reflecting a gait rule of user motion; and based on the sound characteristic period and the gait period, determining the motion posture of the user. According to the method, the motion posture of the user can be accurately analyzed, and the analyzed motion posture can be used for controlling motion auxiliary equipment or recording motion habits of the user, so that the motion safety of the user is better guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of motion analysis, and in particular, to a motion analysis method, device, terminal, and medium based on sound feature recognition. Background Art

[0002] As people pay more and more attention to physical health, sports have become more and more popular, and various sports assistance devices have also become the choice of more and more people. However, at present, the motion analysis of people basically stays on the analysis of the user's exercise duration, exercise frequency, and various physiological indicators, etc., lacking real-time analysis of the user's motion posture, and the motion posture can reflect the user's real motion situation. It can be seen that in the prior art, the real motion situation of the user cannot be truly understood, which is not conducive to ensuring the user's motion safety.

[0003] Therefore, there are still deficiencies in the prior art. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a motion analysis method, device, terminal, and medium based on sound feature recognition for the above-mentioned defects of the prior art. The technical solutions adopted by the present invention are as follows: In the first aspect, the present invention provides a motion analysis method based on sound feature recognition, wherein the method includes: When the user is exercising, based on the sound signal collected in real time by the sound collection module, and based on the sound signal, a sound feature period is determined, and the sound feature period is used to reflect the change rule of the sound signal. The sound collection module is arranged on one side of the sports assistance device; Based on the sound feature period, the gait period corresponding to the sound feature period is determined, and the gait period is used to reflect the gait rule of the user's motion; Based on the sound feature period and the gait period, the user's motion posture is determined.

[0005] In one implementation, the sound signal includes a first sound signal and a second sound signal. The determining the sound feature period based on the sound signal includes: Respectively obtaining a first peak value corresponding to the first sound signal and a second peak value corresponding to the second sound signal; If the first peak value is greater than the second peak value, it is determined that the first sound signal is the sound signal emitted by the foot close to the sound collection module, and it is determined that the second sound signal is the sound signal emitted by the foot far from the sound collection module; Obtain the periodic change law of the first sound signal to obtain the sound feature period corresponding to the first sound signal, or obtain the periodic change law of the second sound signal to obtain the sound feature period corresponding to the second sound signal.

[0006] In one implementation, the sound signal includes a first sound signal and a second sound signal. Determining the sound feature period based on the sound signal includes: Obtain the first reception time corresponding to the first sound signal and the second reception time corresponding to the second sound signal respectively; If the first reception time is earlier than the second reception time, determine that the first sound signal is the sound signal emitted by the foot close to the sound collection module, and determine that the second sound signal is the sound signal emitted by the foot far from the sound collection module; Obtain the periodic change law of the first sound signal to obtain the sound feature period corresponding to the first sound signal, or obtain the periodic change law of the second sound signal to obtain the sound feature period corresponding to the second sound signal.

[0007] In one implementation, determining the gait period corresponding to the sound feature period based on the sound feature period includes: Obtain a pre-trained first mapping relationship, where the first mapping relationship is used to reflect the correspondence between the change of the sound signal and the gait; Match the sound feature period with the first mapping relationship to obtain the gait period corresponding to the sound feature period, where the sound feature period includes a sound enhancement period, a sound attenuation period, and a sound stable period, and the gait period includes a support period, a swing period, and a standing period; Among them, the determination method of the first mapping relationship includes: When the user is exercising on the motion assistance device, collect sound signal samples under different gait samples and determine the sound sample period in which the sound signal samples are located under different gaits; Map the gait sample to the sound sample period to obtain a first mapping relationship.

[0008] In one implementation, determining the user's motion posture based on the sound feature period and the gait period includes: Based on the sound feature period, determine the fluctuation amplitude of the sound signal; Based on the gait period, determine the period duration of the gait period; Based on the fluctuation amplitude of the sound signal and the period duration of the gait period, determine the user's motion posture.

[0009] In one implementation, determining the user's motion posture based on the fluctuation amplitude of the sound signal and the period duration of the gait cycle includes: If the fluctuation amplitude of the sound signal is 0, determine that the motion posture is a stop posture; If the fluctuation amplitude of the sound signal is less than the first amplitude value and the period duration of the gait cycle is greater than the period threshold, determine that the motion posture is a walking posture; If the fluctuation amplitude of the sound signal is greater than the second amplitude value and the period duration of the gait cycle is less than the period threshold, determine that the motion posture is a running posture, where the first amplitude value is less than the second amplitude value.

[0010] In one implementation, the method further includes: When the user is moving, real-time collect the current load data of the motion assistance device, and determine the motion posture information based on the current load data, and set the motion posture information determined based on the current load data as the posture information to be verified; Match the motion posture determined based on the sound signal with the posture information to be verified, and determine the correctness of the posture information to be verified based on the matching result.

[0011] In a second aspect, an embodiment of the present invention further provides a motion analysis device based on sound feature recognition. The device is used to implement the steps of the motion analysis method based on sound feature recognition described in any one of the above solutions. The device includes: A sound feature recognition module, which is used to, when the user is moving, based on the sound signal collected in real time by the sound collection module, and determine the sound feature period based on the sound signal. The sound feature period is used to reflect the change law of the sound signal. The sound collection module is arranged on one side of the motion assistance device; A gait cycle determination module, which is used to determine the gait cycle corresponding to the sound feature period based on the sound feature period. The gait cycle is used to reflect the gait law of the user's motion; A motion posture determination module, which is used to determine the user's motion posture based on the sound feature period and the gait cycle.

[0012] In a third aspect, an embodiment of the present invention further provides a terminal. The terminal includes a memory, a processor, and a motion analysis program based on sound feature recognition stored in the memory and executable on the processor. When the processor executes the motion analysis program based on sound feature recognition, the steps of the motion analysis method based on sound feature recognition described in any one of the above solutions are implemented.

[0013] Fourthly, an embodiment of the present invention further provides a computer-readable storage medium. The computer-readable storage medium stores a motion analysis program based on voice feature recognition. The motion analysis program based on voice feature recognition implements the steps of the motion analysis method based on voice feature recognition described in any one of the above solutions on the computer-readable storage medium.

[0014] Beneficial effects: Compared with the prior art, the present invention provides a motion analysis method based on voice feature recognition. When the user is exercising, the present invention is based on the voice signal collected in real time by the voice acquisition module, and determines the voice feature period based on the voice signal. The voice feature period is used to reflect the change law of the voice signal. The voice acquisition module is arranged on one side of the motion assistance device. Then, based on the voice feature period, the gait period corresponding to the voice feature period is determined. The gait period is used to reflect the gait law of the user's movement. Finally, based on the voice feature period and the gait period, the user's motion posture is determined. Since the voice acquisition module of the present invention is arranged on one side of the motion assistance device, when the user is exercising, the voice signals emitted by the left and right feet collected by the voice acquisition module are different. The present invention can analyze the collected voice signals, determine the change law of the voice signals, obtain the voice feature period, and then further determine the gait period based on the voice feature period. Thus, the user's motion posture analyzed based on the voice feature period and the gait period can be used to control the motion assistance device or record the user's motion habits, so as to better ensure the user's motion safety. Description of the Drawings

[0015] Figure 1 It is a flowchart of a preferred embodiment of the motion analysis method based on voice feature recognition provided by an embodiment of the present invention.

[0016] Figure 2 It is a schematic diagram of the architecture of the motion analysis device based on voice feature recognition provided by an embodiment of the present invention.

[0017] Figure 3 It is a schematic block diagram of the principle of the terminal provided by an embodiment of the present invention. Detailed Embodiments

[0018] To make the objectives, technical solutions and effects of the present invention clearer and more definite, the following further describes the present invention in detail with reference to the accompanying drawings and by way of examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0019] The flowcharts shown in the accompanying drawings are merely illustrative examples and do not necessarily include all the content, operations, or steps, nor do they necessarily have to be executed in the described order. For example, some operations or steps can be decomposed, combined, or partially merged, so the actual execution order may change according to the actual situation.

[0020] It should be understood that the terms used in the description of the present invention herein are merely for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the description of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms. It should be understood that, for the convenience of clearly describing the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, terms such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and effects. For example, the first control information and the second control information are only used to distinguish different control information and do not limit their sequence. Those skilled in the art can understand that terms such as "first" and "second" do not limit the quantity and execution order, and "first", "second", etc. do not necessarily mean different. It should also be understood that the term "and / or" used in the description of the present invention and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0021] Based on the problems of the prior art, the present invention provides a motion analysis method based on sound feature recognition. The method based on this embodiment can accurately analyze the motion posture, and the analyzed motion posture can be used to control the motion assistance device or record the user's motion habits, so as to better ensure the user's motion safety. Specifically in application, in this embodiment, when the user is in motion, based on the sound signal collected in real time by the sound collection module, and based on the sound signal, a sound feature period is determined, and the sound feature period is used to reflect the change law of the sound signal. The sound collection module is arranged on one side of the motion assistance device. Then, based on the sound feature period, the gait period corresponding to the sound feature period is determined, and the gait period is used to reflect the gait law of the user's motion. Finally, based on the sound feature period and the gait period, the user's motion posture is determined. The motion postures of this embodiment include a stop posture, a walking posture, and a running posture. It can be seen that the motion analysis method based on sound feature recognition according to this embodiment can analyze the user's real motion situation, which is convenient for providing better motion assistance services for the user, and is also convenient for better ensuring the user's motion safety by understanding the user's real motion habits.

[0022] The motion analysis method based on voice feature recognition in this embodiment can be applied to terminals, such as computer, mobile phone, smart TV and other terminal products. In addition, since the application scenario of this embodiment is a motion scenario, in order to facilitate the analysis of the user's real motion situation and to ensure the real-time performance of the motion analysis method based on voice feature recognition, the motion analysis method based on voice feature recognition in this embodiment can also be applied to wearable devices, that is, the above terminal is the user's wearable device, such as smart watches, smart bracelets and other portable intelligent products. In addition, the motion analysis method based on voice feature recognition in this embodiment can also be applied to motion assistance devices, such as treadmills. At this time, the above terminal is the treadmill. In practical applications, the specific form of the terminal is not limited in this embodiment. Specifically, as Figure 1 shown in, the motion analysis method based on voice feature recognition in this embodiment includes the following steps: Step S100, when the user is exercising, based on the voice signal collected in real time by the voice collection module, and determining a voice feature period based on the voice signal, the voice feature period is used to reflect the change rule of the voice signal, and the voice collection module is arranged on one side of the motion assistance device.

[0023] Taking running exercise as an example, the motion assistance device in this embodiment is a treadmill, and the voice collection module can be arranged on one side under the treadmill track. For example, a microphone device is arranged on the left side under the track to collect the voice signals generated by the left and right feet. When the user is exercising, the left and right feet of the user alternately contact the track, and when exercising, sounds will be generated due to the impact vibration and friction between the sole of the foot and the track, and are collected by the voice collection module, so as to obtain the voice signal. When the user is running, the left and right feet alternately contact the track, and the voice signals generated by the left and right feet change periodically. In this embodiment, the voice signal can be analyzed to further determine the voice feature period, and this voice feature period is used to reflect the change rule of the voice signal. Based on this voice feature period, it is helpful to analyze the gait period in the subsequent steps.

[0024] In one implementation manner, when determining the voice signal period in this embodiment, it includes the following steps: Step S101, respectively obtaining the first peak value corresponding to the first voice signal and the second peak value corresponding to the second voice signal; Step S102, if the first peak value is greater than the second peak value, determining that the first voice signal is the voice signal emitted by the foot close to the voice collection module, and determining that the second voice signal is the voice signal emitted by the foot far from the voice collection module; Step S103: Obtain the periodic change pattern of the first sound signal to get the sound feature period corresponding to the first sound signal, or obtain the periodic change pattern of the second sound signal to get the sound feature period corresponding to the second sound signal.

[0025] In order to improve the accuracy when analyzing sound signals and avoid noise interference, in this embodiment, after obtaining the sound signal, preprocessing can be performed on the sound signal. The preprocessing in this embodiment includes band-pass filtering and denoising. Specifically, in this embodiment, the sound signal can be filtered to retain the sound signal in a preset frequency band (such as 20 - 500 Hz) to highlight the low-frequency impact characteristics of the footsteps. Then, the double-threshold energy detection method can be used to distinguish the footsteps from the background noise, so as to remove the background noise in the sound signal of the preset frequency band. For example, by setting two energy thresholds and combining the zero-crossing rate to determine whether it is an effective footstep, the background noise can be better removed, thus obtaining an effective sound signal. In practical applications, the sound acquisition module can be set at a distance of 5 - 10 cm from the surface of the treadmill track under the track and fixed by a rubber bracket to isolate the motor vibration interference.

[0026] Since the sound acquisition module is set on one side under the track, the sound signals emitted by the left and right feet collected are different. For example, if the sound acquisition module is set on the left side under the track, the sound signal emitted by the left foot can be transmitted to the sound acquisition module faster, so the sound signal emitted by the left foot collected by the sound acquisition module is more obvious and clearer. Therefore, the sound signals collected by the sound acquisition module in this embodiment include two, namely the first sound signal and the second sound signal, and one of these two sound signals corresponds to the footsteps of the left foot and the other corresponds to the footsteps of the right foot. In this embodiment, these two sound signals need to be identified and analyzed to extract the sound feature period.

[0027] Specifically, since the footstep attenuation of the nearby sound collection module is less, the peak value of the signal collected by the sound collection module is greater. Therefore, in this embodiment, the first peak value corresponding to the first sound signal and the second peak value corresponding to the second sound signal can be obtained respectively. If the first peak value is greater than the second peak value, it is determined that the first sound signal is the sound signal emitted by the foot close to the sound collection module, and the second sound signal is the sound signal emitted by the foot away from the sound collection module. For example, if the sound collection module is set on the left side under the crawler and is closer to the left foot, the peak value of the sound signal of the left footstep will be greater. Therefore, the first sound signal corresponds to the left footstep, and the second sound signal corresponds to the right footstep. Since the first sound signal and the second sound signal change periodically during the user's running, the periodic change rule of the first sound signal and the second sound signal can be obtained at this time, and the sound feature period corresponding to the first sound signal and the sound feature period corresponding to the second sound signal can be obtained. The sound feature period may include information such as wave peaks, wave valleys, fluctuation amplitudes, rising edge slopes, falling edge slopes, and period durations. Based on the difference between the signal peak values, this embodiment can accurately judge the sound signals of the left and right feet, facilitating the accurate analysis of the user's motion posture in subsequent steps.

[0028] In another implementation, when determining the sound feature period in this embodiment, the following steps may further be included: Step S11: Obtain the first reception time corresponding to the first sound signal and the second reception time corresponding to the second sound signal respectively; Step S12: If the first reception time is earlier than the second reception time, determine that the first sound signal is the sound signal emitted by the foot close to the sound collection module, and determine that the second sound signal is the sound signal emitted by the foot away from the sound collection module; Step S13: Obtain the periodic change rule of the first sound signal to obtain the sound feature period corresponding to the first sound signal, or obtain the periodic change rule of the second sound signal to obtain the sound feature period corresponding to the second sound signal.

[0029] Since the sound of footsteps from the nearby sound collection module can be collected more quickly, there is a difference in the reception time of the sound signals of the footsteps of the left and right feet. Therefore, in this embodiment, the first reception time corresponding to the first sound signal and the second reception time corresponding to the second sound signal can be obtained respectively. If the first reception time is earlier than the second reception time, it is determined that the first sound signal is the sound signal emitted by the foot close to the sound collection module, and it is determined that the second sound signal is the sound signal emitted by the foot far from the sound collection module. For example, if the sound collection module is set on the left side under the track belt and is closer to the left foot, the sound signal of the footsteps of the left foot will be received by the sound collection module earlier. Therefore, the first sound signal corresponds to the footsteps of the left foot, and the second sound signal corresponds to the footsteps of the right foot. Since the first sound signal and the second sound signal change periodically during the user's running, the periodic change rule of the first sound signal and the second sound signal can be obtained at this time, and the sound characteristic period corresponding to the first sound signal and the sound characteristic period corresponding to the second sound signal can be obtained.

[0030] Step S200: Based on the sound characteristic period, determine the gait period corresponding to the sound characteristic period, and the gait period is used to reflect the gait rule of the user's movement.

[0031] In this embodiment, after obtaining the sound characteristic period, the sound characteristic period is analyzed to determine the gait period. In this embodiment, the gait period includes a support period, a swing period, and a standing period. The support period, the swing period, and the standing period constitute a complete gait period. Among them, the support period refers to the stage when the foot starts to contact the treadmill and bears the body weight, and the swing period refers to the stage when the foot swings forward after leaving the ground. It should be noted that the standing period does not necessarily exist. If there is no obvious stay during the user's movement and the support period is directly connected and transitioned with the swing period, there is no standing period. In the user's running exercise, taking the user's single leg as an example, when the user is in the support period, the sole of the foot touches the ground and exerts a vertical impact force and a backward frictional force on the track belt. The impact vibration and friction between the sole of the foot and the track belt will generate sound. At this time, the sound signal is increasing, so the corresponding sound characteristic period is the sound increasing period. Especially in the extension stage, the muscle exerts a greater propulsive force and the sound signal reaches the peak. When the user is in the swing period, after the foot leaves the ground, the force exerted by the user on the running belt decreases. At this time, the sound signal is attenuating, so the corresponding sound characteristic period is the sound attenuation period. When the user is in the standing period, the sound signal approaches 0, and the corresponding sound characteristic period is the sound stable period. The sound enhancement period, the sound attenuation period, and the sound stable period constitute a complete sound characteristic period. Based on this, this embodiment can train the first mapping relationship and determine the gait period corresponding to the sound characteristic period at this time based on this first mapping relationship.

[0032] Further, since the movement of the left and right feet is symmetric when the user is running, to facilitate the analysis of the movement postures of the left and right feet, reduce the computational amount, and improve the analysis efficiency, after obtaining the sound feature period of the first sound signal and the sound feature period of the second sound signal, since the first sound signal is the sound signal emitted by the foot closer to the sound collection module, the attenuation of the first sound signal is small, and it is more accurate and clearer. Therefore, in this embodiment, the sound feature period of the first sound signal can be selected for subsequent analysis.

[0033] In one implementation manner, this embodiment includes the following steps when determining the gait period: Step S201: Obtain a pre-trained first mapping relationship, where the first mapping relationship is used to reflect the correspondence between the change of the sound signal and the gait; Step S202: Match the sound feature period with the second mapping relationship to obtain the gait period corresponding to the sound feature period.

[0034] In practical applications, this embodiment first trains the first mapping relationship, which is used to reflect the correspondence between the change of the sound signal and the gait and can be used to determine the gait period corresponding to the sound feature period. Specifically, when the user is exercising on the movement assistance device in this embodiment, sound signal samples under different gait samples are collected, that is, the sound signal samples corresponding to the user's support period, swing period, and standing period are collected and form a sound signal set. Then, the sound sample period in which the sound signal sample is located under different gaits is further determined. At this time, there is a certain correspondence between the sound sample period and the gait sample. Therefore, the gait sample and the sound sample period can be mapped to obtain the first mapping relationship. The first mapping relationship at this time is: gait sample - sound sample period. Therefore, after obtaining the sound feature period based on the above step S100, by matching the sound feature period with the first mapping relationship, the corresponding gait period can be obtained. For example, if the determined sound feature period is the sound enhancement period, the corresponding gait period is the support period; if the determined sound feature period is the sound attenuation period, the corresponding gait period is the swing period; if the determined sound feature period is the sound stable period, the corresponding gait period is the standing period. In addition, when mapping the gait sample and the sound sample period in this embodiment, the sound features in the gait sample and the sound sample period can be mapped, such as mapping the signal peak value or sound frequency and other features between the support period and the sound enhancement period.

[0035] In other implementation manners, when training the first mapping relationship in this embodiment, it is also possible to collect sound signal samples of multiple user samples with different genders, different body types, and different running habits under different gaits, so as to determine the sound sample period in which the sound signal sample of each user sample is located under different gait samples, and then synthesize the sound sample periods of all user samples to obtain a synthesized sound sample period, and then map the synthesized sound sample period and the gait sample to obtain the first mapping relationship. Since the first mapping relationship at this time is also trained based on different users, it can be applied to different users, improving the applicability of this embodiment.

[0036] In other implementation manners, this embodiment can train a convolutional neural network model to obtain a gait analysis model for outputting gait labels. The input data of this gait analysis model is the collected sound signal. The gait analysis model can automatically analyze the sound signal to determine the corresponding sound feature period, and then further determine the gait label corresponding to the sound feature period. According to this gait label, the gait period at this time can be determined. Preferably, when analyzing the sound signal, the gait analysis model can automatically classify the waveform of the sound signal based on a clustering algorithm, identify the clustering results under different gaits, and then output the gait label corresponding to the input sound signal at this time.

[0037] In other implementation manners, this embodiment can also directly determine the average signal intensity of each stage in the calculated sound feature period based on the collected sound signal, and then compare the average signal intensity with a preset intensity threshold. If the average signal intensity of a certain stage is greater than the intensity threshold, it can be determined that the gait period of this stage is the stance phase. If the average signal intensity of a certain stage is less than the intensity threshold, it can be determined that the gait period of this stage is the swing phase. For example, if the average signal intensity in the sound increasing period is greater than the intensity threshold, it is determined that the gait period at this time is the stance phase. If the average signal intensity in the sound decaying period is less than the intensity threshold, it is determined that the gait period at this time is the swing phase. Or, this embodiment can also compare the sound signal at a certain moment with the intensity threshold to judge the stance phase and the swing phase. For example, when the sound signal at a certain moment is greater than the intensity threshold, it can be determined that the stance phase starts. If the sound signal at a certain moment is less than the intensity threshold, it can be determined that the stance phase ends and the swing phase starts.

[0038] Step S300: Determine the user's motion posture based on the sound feature period and the gait period.

[0039] Since the sound feature period and the gait period under different motion postures are different and have certain rules, after determining the sound feature period and the gait period in this embodiment, the sound feature period and the gait period can be comprehensively analyzed to determine the user posture.

[0040] In one implementation, when determining the user's motion posture, the present embodiment includes the following steps: Step S301: Determine the fluctuation amplitude of the sound signal based on the sound feature period; Step S302: Determine the period duration of the gait cycle based on the gait cycle; Step S303: Determine the user's motion posture based on the fluctuation amplitude of the sound signal and the period duration of the gait cycle.

[0041] The sound feature period of the present embodiment includes information such as wave peaks, wave valleys, fluctuation amplitudes, rising edge slopes, falling edge slopes, etc. The gait cycle also reflects a lot of information, such as gait frequency, period duration, etc. The present embodiment can extract the fluctuation amplitude of the sound signal based on the sound feature period, and extract the period duration of the gait cycle based on the gait cycle. Then, based on the fluctuation amplitude of the sound signal and the period duration of the gait cycle, the user's motion posture is determined. Specifically, when judging the motion posture, if the fluctuation amplitude of the sound signal is 0, the motion posture is determined to be a stop posture. The present embodiment can also determine the motion posture to be a stop posture when the average value of the sound signal has no fluctuation for a preset duration (such as 5 seconds) and there is no obvious periodic change in the sound signal. If the fluctuation amplitude of the sound signal is less than the first amplitude value and the period duration of the gait cycle is greater than the period threshold, it indicates that the user's running speed is relatively slow at this time, and the motion posture can be determined to be a walking posture. To more accurately determine the motion posture, the present embodiment can further analyze the duration ratio of the support phase in the gait cycle at this time. If the support phase accounts for 60%-70% of the entire gait cycle, the user's motion posture can be more accurately determined to be a walking posture. If the fluctuation amplitude of the sound signal is greater than the second amplitude value and the period duration of the gait cycle is less than the period threshold, it indicates that the user's running speed is relatively fast at this time, and the motion posture can be determined to be a running posture. When in the running posture, the swing phase and the support phase of the user are clearly demarcated. In the present embodiment, the first amplitude value is less than the second amplitude value. For example, the first amplitude value is 8 dB, and the second amplitude value is 10 dB. Similarly, the present embodiment can further analyze the duration ratio of the swing phase in the gait cycle at this time. If the swing phase accounts for 60%-70% of the entire gait cycle, the user's motion posture can be more accurately determined to be a running posture.

[0042] In another implementation, this embodiment can also extract the gait frequency of the gait cycle. The calculation formula for the gait frequency is 60 / T, where T is the cycle duration corresponding to the entire gait cycle. Then, based on the comprehensive analysis of the sound feature cycle and the gait frequency, the motion posture is determined. Specifically, when judging the motion posture, if the gait frequency is 0, it means that the user is not running at this time, and the motion posture can be determined as the stop posture. If the fluctuation amplitude of the sound signal is less than the first amplitude value and the gait frequency of the gait cycle is less than the step frequency threshold, for example, the gait frequency is less than 120 steps / minute, it means that the running speed of the user is relatively slow at this time, and the motion posture can be determined as the walking posture. If the fluctuation amplitude of the sound signal is greater than the first amplitude value and the gait frequency of the gait cycle is greater than the step frequency threshold, that is, the gait frequency is greater than 120 steps / minute, it means that the running speed of the user is relatively fast at this time, so the motion posture can be determined as the running posture. Since more than one parameter is reflected in the electro-acoustic feature cycle and the gait cycle, multiple parameter combinations can be used for the analysis of the motion posture.

[0043] It should be noted that if the movement conditions of the user's left and right feet are the same, in this embodiment, when analyzing the motion posture, the analysis can also be performed taking a single foot as an example. The motion posture is analyzed by analyzing the sound feature cycle and the gait cycle of a single foot (such as the left foot). Of course, this embodiment can also use the above method to separately analyze the sound feature cycles and gait cycles of the left and right feet to respectively analyze the motion postures of the left and right feet, and then determine whether the motion postures of the left and right feet are the same. If they are not the same, it can be determined that an abnormality has occurred. Further, when the user is in the running posture, this embodiment can further obtain the motion parameters of the user in the running posture, and then analyze the motion parameters. For example, the motion parameters obtained at this time are compared with the preset reference motion parameters to analyze whether the running posture of the user is abnormal at this time. If an abnormality occurs, such as a single-foot slip, an alternative run, or a two-handed support (fall), a pop-up prompt or a voice alarm prompt can be automatically issued on the user's mobile terminal, sports assistance device, or wearable device at this time to prompt the user to ensure sports safety. If the duration of the alarm prompt exceeds the preset duration, a control command can be issued to the sports assistance device, such as controlling the treadmill to slow down or even stop working. In addition, this embodiment can also monitor the running state of the sports assistance device in real time when the user is in the running posture. If an abnormality is found, a pop-up prompt or a voice alarm prompt can also be issued to ensure the sports safety of the user.

[0044] In addition, in this embodiment, after analyzing the motion posture, the motion posture can be uploaded to a preset virtual reality device, and a virtual animation can be generated based on the motion posture. The virtual animation is generated in real time based on the user's motion posture and will be synchronized with the user's motion posture, which can help the user understand their own motion posture in real time so as to make adjustments in case of non-standard postures.

[0045] Furthermore, in this embodiment, the motion posture information of the user can also be analyzed by collecting the current load data of the motion assistance device. Specifically, taking running as an example, the motion assistance device in this embodiment is a treadmill. Since the current load data on the treadmill changes periodically when the user is running, and the gait of the user running is also periodically changing, therefore, the motion posture can be analyzed based on the periodic change rules of the current load data and the gait. For this purpose, in this embodiment, when the user is exercising, the current load data of the motion assistance device can be collected in real time, and then the current cycle corresponding to the current load data can be determined based on the second mapping relationship. The second mapping relationship in this embodiment is: current load sample - current sample period. Based on this second mapping relationship, the current load data of the motion assistance device can be collected in real time during the user's running process, and then the current load data can be matched with the second mapping relationship to obtain the corresponding current cycle, which includes: current load rising period, current load falling period, and current load stable period.

[0046] The change in the current load data of a motion assistance device (such as a treadmill) essentially reflects the power demand of the user for the motion assistance device during running, and this power demand is directly related to the mechanical characteristics of each stage in the gait cycle. In this embodiment, the gait cycle includes a stance phase, a swing phase, and a standing phase, and the stance phase, the swing phase, and the standing phase constitute a complete gait cycle. In the user's running motion, taking the user's single leg as an example, when the user is in the stance phase, the foot touches the ground and exerts a vertical impact force and a backward frictional force on the running belt. The treadmill motor needs to increase the torque to maintain the running belt speed, and at this time, the current load data of the treadmill increases significantly. Therefore, the corresponding current cycle is the current load rising period. Especially in the extension phase, the muscles exert more force to generate a greater propulsive force, and the current load data reaches the peak. When the user is in the swing phase, after the foot leaves the ground, the force exerted by the user on the running belt decreases, and the treadmill motor only needs to maintain the basic operation of the running belt. At this time, the current load data drops to a stable level. Therefore, the corresponding current cycle is the current load falling period. When the user is in the standing phase, the current load data of the treadmill remains stable and lasts for a period of time. The corresponding current cycle is the current load stable period. The current load rising period, the current load falling period, and the current load stable period constitute a complete current cycle. Based on this, this embodiment trains the third mapping relationship and determines the gait cycle corresponding to the current cycle at this time based on this third mapping relationship. The third mapping relationship in this embodiment is: gait sample - current sample cycle. When the current cycle is obtained, by matching the current cycle with the third mapping relationship, the corresponding gait cycle can be obtained. For example, if the determined current cycle is the current load rising period, the corresponding gait cycle is the stance phase; if the determined current cycle is the current load falling period, the corresponding gait cycle is the swing phase; if the determined current cycle is the current load stabilizer, the corresponding gait cycle is the standing phase.

[0047] Since the current cycle and the gait cycle under different motion postures are different and have certain rules, after obtaining the current cycle and the gait cycle, this embodiment can perform comprehensive analysis based on the current cycle and the gait cycle to determine the user's posture. Specifically, the current characteristics in the current cycle and the gait characteristics in the gait cycle are extracted respectively, and then comprehensive analysis is performed based on the current characteristics and the gait characteristics. The current characteristics in this embodiment include the current fluctuation amplitude and the current frequency reflected by the entire current cycle, and the gait characteristics include: cycle duration or gait frequency.

[0048] In practical applications, when the gait characteristic is the cycle duration, if both the current fluctuation amplitude and the current frequency are 0, it is determined that the motion posture is the stop posture; if the current fluctuation amplitude is less than the first amplitude value, the current frequency is less than the first frequency value, and the cycle duration of the gait cycle is greater than the cycle threshold, it is determined that the motion posture is the walking posture; if the current fluctuation amplitude is greater than the second amplitude value, the current frequency is greater than the second frequency value, and the cycle duration of the gait cycle is less than the cycle threshold, it is determined that the motion posture is the running posture. In another implementation, when the gait characteristic is the gait frequency, if the gait frequency is 0, it is determined that the motion posture is the stop posture; if the current fluctuation amplitude is less than the first amplitude value, the current frequency is less than the first frequency value, and the gait frequency of the gait cycle is less than the step frequency threshold, it is determined that the motion posture is the walking posture; if the current fluctuation amplitude is greater than the second amplitude value, the current frequency is greater than the second frequency value, and the gait frequency of the gait cycle is greater than the step frequency threshold, it is determined that the motion posture is the running posture. Thus, it can be seen that the motion posture information can also be analyzed based on the current load data in this embodiment.

[0049] Furthermore, in this embodiment, the motion posture information determined based on the current load data can be set as the posture information to be verified. Then, the motion posture determined based on the sound signal is matched with the posture information to be verified, and the correctness of the posture information to be verified is determined based on the matching result. Specifically, if at the same moment, the motion posture determined based on the sound signal is the same as the motion posture information that can also be analyzed based on the current load data, such as both being the running posture, it can be determined that the motion posture information that can also be analyzed based on the current load data is correct. If at the same moment, the motion posture determined based on the sound signal is different from the motion posture information that can also be analyzed based on the current load data, it can be determined that the motion posture information that can also be analyzed based on the current load data is incorrect. This embodiment can verify the accuracy of other methods for analyzing the motion posture based on the motion posture analyzed in this embodiment. On the one hand, it can verify the analysis accuracy of other methods, and on the other hand, it can also analyze the accuracy of the method in this embodiment, which can more effectively ensure the user's motion safety.

[0050] Based on the above embodiments, the present invention also provides a motion analysis device based on sound feature recognition. The motion analysis device based on sound feature recognition is used to implement any one of the steps in the above method embodiments, such as Figure 2As shown in the figure, the device includes: a voice feature recognition module 10, a gait cycle determination module 20, and a motion posture determination module 30. Specifically, the voice feature recognition module 10 is configured to, when the user is exercising, based on the voice signals collected in real time by the voice acquisition module, and determine a voice feature cycle based on the voice signals, where the voice feature cycle is used to reflect the variation law of the voice signals, and the voice acquisition module is disposed on one side of the motion assistance device. The gait cycle determination module 20 is configured to determine the gait cycle corresponding to the voice feature cycle based on the voice feature cycle, where the gait cycle is used to reflect the gait law of the user's motion. The motion posture determination module 30 is configured to determine the motion posture of the user based on the voice feature cycle and the gait cycle.

[0051] In the motion analysis device based on voice feature recognition of this embodiment, the working principles of the various modules are the same as those of the various steps in the above method embodiment, and will not be elaborated here.

[0052] The various modules in the above motion analysis device based on voice feature recognition can be implemented in whole or in part by software, hardware, and their combinations. The above modules can be embedded in the processor in the terminal in the form of hardware or be independent of the processor, or can be stored in the memory in the terminal in the form of software, so as to facilitate the processor to call and execute the operations corresponding to the above various modules.

[0053] Based on the above embodiments, the present invention further provides a terminal, and the principle block diagram of the terminal can be as Figure 3 shown. The terminal may include one or more processors 100 ( Figure 3 only one is shown in the figure), a memory 101, and a computer program 102 stored in the memory 101 and executable on one or more processors 100. For example, a motion analysis program based on voice feature recognition. When the one or more processors 100 execute the computer program 102, the various steps in the method embodiment of the motion analysis method based on voice feature recognition can be implemented. Or, when the one or more processors 100 execute the computer program 102, the functions of the various modules / units in the system embodiment of the motion analysis system based on voice feature recognition can be implemented, which is not limited here.

[0054] In one embodiment, the processor 100 may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0055] In one embodiment, the memory 101 may be an internal storage unit of the electronic device, such as the hard disk or memory of the electronic device. The memory 101 may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the electronic device. Further, the memory 101 may also include both an internal storage unit and an external storage device of the electronic device. The memory 101 is used to store computer programs and other programs and data required by the terminal. The memory 101 may also be used to temporarily store data that has been output or is to be output.

[0056] Those skilled in the art can understand that Figure 3 the principle block diagram shown in

[0057] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, operational database, or other medium used in the embodiments provided by the present invention can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0058] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or equivalently replace some of the technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A motion analysis method based on sound feature recognition, characterized in that The method includes: When the user is exercising, based on the sound signals collected in real time by the sound collection module, and based on the sound signals, determining a sound feature period, where the sound feature period is used to reflect the variation law of the sound signals, and the sound collection module is arranged on one side of the motion assistance device; Based on the sound feature period, determining the gait period corresponding to the sound feature period, where the gait period is used to reflect the gait law of the user's movement; Based on the sound feature period and the gait period, determining the user's motion posture.

2. The motion analysis method based on voice feature recognition according to claim 1, wherein The sound signals include a first sound signal and a second sound signal, and determining the sound feature period based on the sound signals includes: Respectively obtaining a first peak value corresponding to the first sound signal and a second peak value corresponding to the second sound signal; If the first peak value is greater than the second peak value, determining that the first sound signal is the sound signal emitted by the foot close to the sound collection module, and determining that the second sound signal is the sound signal emitted by the foot far from the sound collection module; Obtaining the periodic variation law of the first sound signal to obtain the sound feature period corresponding to the first sound signal, or obtaining the periodic variation law of the second sound signal to obtain the sound feature period corresponding to the second sound signal.

3. The motion analysis method based on voice feature recognition according to claim 1, wherein, The sound signals include a first sound signal and a second sound signal, and determining the sound feature period based on the sound signals includes: Respectively obtaining a first reception time corresponding to the first sound signal and a second reception time corresponding to the second sound signal; If the first reception time is earlier than the second reception time, determining that the first sound signal is the sound signal emitted by the foot close to the sound collection module, and determining that the second sound signal is the sound signal emitted by the foot far from the sound collection module; Obtaining the periodic variation law of the first sound signal to obtain the sound feature period corresponding to the first sound signal, or obtaining the periodic variation law of the second sound signal to obtain the sound feature period corresponding to the second sound signal.

4. The motion analysis method based on voice feature recognition according to claim 1, characterized in that Based on the sound feature period, determining the gait period corresponding to the sound feature period includes: Obtaining a pre-trained first mapping relationship, where the first mapping relationship is used to reflect the corresponding relationship between the variation of the sound signals and the gait; Matching the sound feature period with the first mapping relationship to obtain the gait period corresponding to the sound feature period, where the sound feature period includes a sound enhancement period, a sound attenuation period, and a sound stable period, and the gait period includes a support period, a swing period, and a standing period; Wherein, the determination method of the first mapping relationship includes: When the user is exercising on the motion assistance device, collecting sound signal samples under different gait samples, and determining the sound sample period in which the sound signal samples are located under different gaits; Mapping the gait samples and the sound sample period to obtain a first mapping relationship.

5. The motion analysis method based on voice feature recognition according to claim 1, characterized in that Based on the sound feature period and the gait period, determining the user's motion posture includes: Determine the fluctuation amplitude of the sound signal based on the sound feature period; Determine the period duration of the gait cycle based on the gait cycle; Determine the user's motion posture based on the fluctuation amplitude of the sound signal and the period duration of the gait cycle.

6. The motion analysis method based on voice feature recognition according to claim 5, wherein The determining the user's motion posture based on the fluctuation amplitude of the sound signal and the period duration of the gait cycle includes: If the fluctuation amplitude of the sound signal is 0, determine that the motion posture is a stop posture; If the fluctuation amplitude of the sound signal is less than the first amplitude value and the period duration of the gait cycle is greater than the period threshold, determine that the motion posture is a walking posture; If the fluctuation amplitude of the sound signal is greater than the second amplitude value and the period duration of the gait cycle is less than the period threshold, determine that the motion posture is a running posture, where the first amplitude value is less than the second amplitude value.

7. The motion analysis method based on voice feature recognition according to claim 1, characterized in that The method further includes: When the user is moving, real-time collect the current load data of the motion assistance device, and determine the motion posture information based on the current load data, and set the motion posture information determined based on the current load data as the to-be-verified posture information; Match the motion posture determined based on the sound signal with the to-be-verified posture information, and determine the correctness of the to-be-verified posture information based on the matching result.

8. A motion analysis device based on sound feature recognition, characterized in that, The device is used to implement the steps of the motion analysis method based on sound feature recognition according to any one of claims 1-7. The device includes: A sound feature recognition module, configured to, when the user is moving, based on the sound signal collected in real time by the sound collection module, and determine the sound feature period based on the sound signal, where the sound feature period is used to reflect the change rule of the sound signal, and the sound collection module is disposed on one side of the motion assistance device; A gait cycle determination module, configured to determine the gait cycle corresponding to the sound feature period based on the sound feature period, where the gait cycle is used to reflect the gait rule of the user's motion; A motion posture determination module, configured to determine the user's motion posture based on the sound feature period and the gait cycle.

9. A terminal, characterized in that, The terminal includes a memory, a processor, and a motion analysis program based on sound feature recognition stored in the memory and executable on the processor. When the processor executes the motion analysis program based on sound feature recognition, the steps of the motion analysis method based on sound feature recognition according to any one of claims 1-7 are implemented.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a motion analysis program based on sound feature recognition. The motion analysis program based on sound feature recognition implements the steps of the motion analysis method based on sound feature recognition according to any one of claims 1-7 on the computer-readable storage medium.

Images