Motion analysis method and device based on sound feature recognition, terminal and medium

Abstract

The application discloses a motion analysis method and device based on sound feature recognition, a terminal and a medium. The method comprises the following steps: when a user is in motion, a sound signal is collected in real time based on a sound collection module, and a sound feature period is determined based on the sound signal. The sound feature period is used for reflecting the change rule of the sound signal. The sound collection module is arranged on one side of a motion auxiliary device. Based on the sound feature period, a gait period corresponding to the sound feature period is determined. The gait period is used for reflecting the gait rule of the user's motion. Based on the sound feature period and the gait period, the motion posture of the user is determined. The motion posture of the user can be accurately analyzed. The analyzed motion posture can be used for controlling the motion auxiliary device or recording the motion habit of the user, so that the motion safety of the user can be better ensured.

Description

Technical Field

[0001] The present invention relates to the field of motion analysis technology, and in particular to a motion analysis method, device, terminal and medium based on sound feature recognition. Background Art

[0002] As people increasingly prioritize health and fitness, exercise is becoming increasingly popular, and a variety of exercise-assistive devices are becoming increasingly popular. However, current analysis of people's exercise routines primarily focuses on analyzing exercise duration, frequency, and various physiological indicators. It lacks real-time analysis of a user's movement posture, which can reflect their true movement. Consequently, existing technologies fail to truly understand a user's true movement patterns, hindering their safety during exercise.

[0003] Therefore, the prior art still has defects. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a motion analysis method, device, terminal and medium based on sound feature recognition in order to address the above-mentioned defects of the prior art. The technical solution adopted by the present invention is as follows:

[0005] In a first aspect, the present invention provides a motion analysis method based on sound feature recognition, wherein the method comprises:

[0006] When the user exercises, a sound characteristic period is determined based on the sound signal collected in real time by the sound collection module, and the sound characteristic period is used to reflect the change pattern of the sound signal. The sound collection module is provided on one side of the exercise assisting device;

[0007] Based on the sound characteristic period, determining a gait period corresponding to the sound characteristic period, wherein the gait period is used to reflect a gait pattern of the user's movement;

[0008] The user's movement posture is determined based on the sound feature period and the gait period.

[0009] In one implementation, the sound signal includes a first sound signal and a second sound signal, and determining the sound characteristic period based on the sound signal includes:

[0010] respectively obtaining a first peak value corresponding to the first sound signal and a second peak value corresponding to the second sound signal;

[0011] If the first peak value is greater than the second peak value, it is determined that the first sound signal is a sound signal emitted by the foot close to the sound collection module, and the second sound signal is determined to be a sound signal emitted by the foot far away from the sound collection module;

[0012] Obtain the periodic variation pattern of the first sound signal to obtain the sound characteristic period corresponding to the first sound signal, or obtain the periodic variation pattern of the second sound signal to obtain the sound characteristic period corresponding to the second sound signal.

[0013] In one implementation, the sound signal includes a first sound signal and a second sound signal, and determining the sound characteristic period based on the sound signal includes:

[0014] respectively obtaining a first reception time corresponding to the first sound signal and a second reception time corresponding to the second sound signal;

[0015] If the first receiving time is earlier than the second receiving time, determining that the first sound signal is a sound signal emitted by a foot close to the sound collection module, and determining that the second sound signal is a sound signal emitted by a foot far away from the sound collection module;

[0016] Obtain the periodic variation pattern of the first sound signal to obtain the sound characteristic period corresponding to the first sound signal, or obtain the periodic variation pattern of the second sound signal to obtain the sound characteristic period corresponding to the second sound signal.

[0017] In one implementation, determining the gait period corresponding to the sound characteristic period based on the sound characteristic period includes:

[0018] Acquiring a pre-trained first mapping relationship, where the first mapping relationship is used to reflect a correspondence between a change in the sound signal and a gait;

[0019] Matching the sound characteristic period with the first mapping relationship to obtain a gait period corresponding to the sound characteristic period, wherein the sound characteristic period includes a sound enhancement period, a sound attenuation period, and a sound stabilization period, and the gait period includes a support period, a swing period, and a stance period;

[0020] The method for determining the first mapping relationship includes:

[0021] When the user exercises on the exercise assisting device, collecting sound signal samples under different gait samples, and determining the sound sample periods of the sound signal samples under different gaits;

[0022] The gait sample is mapped to the sound sample period to obtain a first mapping relationship.

[0023] In one implementation, determining the user's motion posture based on the sound feature period and the gait period includes:

[0024] determining a fluctuation amplitude of the sound signal based on the sound characteristic period;

[0025] Based on the gait cycle, determining a period length of the gait cycle;

[0026] The user's movement posture is determined based on the fluctuation amplitude of the sound signal and the period length of the gait cycle.

[0027] In one implementation, determining the user's movement posture based on the fluctuation amplitude of the sound signal and the period duration of the gait cycle includes:

[0028] If the fluctuation amplitude of the sound signal is 0, determining that the motion posture is a stop posture;

[0029] If the fluctuation amplitude of the sound signal is less than a first amplitude value and the period length of the gait cycle is greater than a period threshold, determining that the motion posture is a walking posture;

[0030] If the fluctuation amplitude of the sound signal is greater than the second amplitude value and the period length of the gait cycle is less than the period threshold, it is determined that the motion posture is a running posture, wherein the first amplitude value is less than the second amplitude value.

[0031] In one implementation, the method further includes:

[0032] When the user is exercising, current load data of the exercise assisting device is collected in real time, and exercise posture information is determined based on the current load data, and the exercise posture information determined based on the current load data is set as the posture information to be verified;

[0033] The motion posture determined based on the sound signal is matched with the posture information to be verified, and the correctness of the posture information to be verified is determined based on the matching result.

[0034] In a second aspect, an embodiment of the present invention further provides a motion analysis device based on sound feature recognition, wherein the device is used to implement the steps of the motion analysis method based on sound feature recognition in any one of the above solutions, and the device includes:

[0035] A sound feature recognition module is configured to identify a sound feature period based on the sound signal collected in real time by the sound collection module when the user is exercising. The sound feature period is configured to reflect the changing pattern of the sound signal. The sound collection module is disposed on one side of the exercise assistive device.

[0036] a gait cycle determination module, configured to determine a gait cycle corresponding to the sound characteristic cycle based on the sound characteristic cycle, wherein the gait cycle is used to reflect the gait pattern of the user's movement;

[0037] The motion posture determination module is used to determine the user's motion posture based on the sound feature period and the gait period.

[0038] In a third aspect, an embodiment of the present invention further provides a terminal, wherein the terminal includes a memory, a processor, and a motion analysis program based on sound feature recognition stored in the memory and runnable on the processor. When the processor executes the motion analysis program based on sound feature recognition, the steps of the motion analysis method based on sound feature recognition in any one of the above-mentioned schemes are implemented.

[0039] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein a motion analysis program based on sound feature recognition is stored on the computer-readable storage medium, and the motion analysis program based on sound feature recognition implements the steps of the motion analysis method based on sound feature recognition described in any one of the above-mentioned schemes on the computer-readable storage medium.

[0040] Beneficial Effects: Compared to the prior art, the present invention provides a motion analysis method based on sound feature recognition. When a user exercises, the present invention uses a sound signal collected in real time by a sound acquisition module and determines a sound feature period based on the sound signal. The sound feature period is used to reflect the changing pattern of the sound signal. The sound acquisition module is disposed on one side of a motion-assisting device. Then, based on the sound feature period, the gait period corresponding to the sound feature period is determined. The gait period is used to reflect the gait pattern of the user's motion. Finally, the user's motion posture is determined based on the sound feature period and the gait period. Because the sound acquisition module of the present invention is disposed on one side of the motion-assisting device, when the user exercises, the sound signals emitted by the left and right feet collected by the sound acquisition module differ. The present invention can analyze the collected sound signal, determine the changing pattern of the sound signal, obtain the sound feature period, and then determine the gait period based on the sound feature period. The user's motion posture can be analyzed based on the sound feature period and the gait period. The analyzed motion posture can be used to control the motion-assisting device or record the user's motion habits to better ensure the user's motion safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 The present invention provides a flowchart of a preferred embodiment of a motion analysis method based on sound feature recognition.

[0042] Figure 2 A schematic diagram of the architecture of a motion analysis device based on sound feature recognition provided by an embodiment of the present invention.

[0043] Figure 3This is a functional block diagram of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0045] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents, operations, or steps, nor must they be executed in the order described. For example, some operations or steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0046] It should be understood that the terms used in this specification are only for the purpose of describing particular embodiments and are not intended to limit the present invention. As used in the specification and appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0047] It should be understood that, to facilitate a clear description of the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, terms such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. For example, the first control information and the second control information are merely used to distinguish different control information and do not limit their order.

[0048] Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0049] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0050] To address the problems of the prior art, the present invention provides a motion analysis method based on sound feature recognition. The method of this embodiment can accurately analyze motion postures, which can be used to control motion-assistance devices or record a user's motion habits to better ensure the user's motion safety. In specific applications, this embodiment uses a sound signal collected in real time by a sound acquisition module while the user is exercising to determine a sound feature period based on the sound signal. The sound feature period is used to reflect the changing pattern of the sound signal. The sound acquisition module is disposed on one side of the motion-assistance device. Then, based on the sound feature period, the gait period corresponding to the sound feature period is determined. The gait period is used to reflect the gait pattern of the user's motion. Finally, based on the sound feature period and the gait period, the user's motion posture is determined. The motion postures of this embodiment include a stop posture, a walking posture, and a running posture. Therefore, the motion analysis method based on sound feature recognition of this embodiment can analyze the user's actual motion situation, facilitate providing better motion-assistance services to the user, and facilitate better ensuring the user's motion safety by understanding the user's actual motion habits.

[0051] The motion analysis method based on sound feature recognition of this embodiment can be applied to a terminal, and the terminal can be a terminal product such as a computer, a mobile phone, and a smart TV. In addition, since the application scenario of this embodiment is a sports scenario, in order to facilitate the analysis of the user's actual motion situation and to ensure the real-time performance of the motion analysis method based on sound feature recognition, the motion analysis method based on sound feature recognition of this embodiment can also be applied to wearable devices, that is, the above-mentioned terminal is the user's wearable device, such as a smart watch, a smart bracelet and other portable intelligent products. In addition, the motion analysis method based on sound feature recognition of this embodiment can also be applied to sports auxiliary equipment, such as a treadmill. In this case, the above-mentioned terminal is a treadmill. In actual applications, the specific form of the terminal is not limited in this embodiment. Specifically, as Figure 1 As shown in , the motion analysis method based on sound feature recognition of this embodiment includes the following steps:

[0052] Step S100: When the user exercises, the sound signal collected in real time by the sound collection module is used to determine the sound characteristic period based on the sound signal, and the sound characteristic period is used to reflect the change pattern of the sound signal. The sound collection module is set on one side of the exercise assisting device.

[0053] Taking running as an example, the exercise-assisting device of this embodiment is a treadmill. The sound collection module can be disposed on one side below the treadmill's track. For example, a microphone device can be disposed on the left side below the track to collect sound signals generated by the left and right feet. When a user exercises, the user's left and right feet alternately contact the track. During movement, the impact, vibration, and friction between the soles of the feet and the track generate sound, which is collected by the sound collection module to obtain a sound signal. When a user runs, the left and right feet alternately contact the track, and the sound signals generated by the left and right feet vary periodically. This embodiment can analyze the sound signals and determine a characteristic sound period. This characteristic sound period is used to reflect the variation pattern of the sound signal. Based on this characteristic sound period, it is helpful to analyze the gait cycle in subsequent steps.

[0054] In one implementation, determining the sound signal period in this embodiment includes the following steps:

[0055] Step S101: obtaining a first peak value corresponding to the first sound signal and a second peak value corresponding to the second sound signal respectively;

[0056] Step S102: If the first peak value is greater than the second peak value, determining that the first sound signal is a sound signal emitted by a foot close to the sound collection module, and determining that the second sound signal is a sound signal emitted by a foot far away from the sound collection module;

[0057] Step S103: Obtain the periodic variation pattern of the first sound signal to obtain the sound characteristic period corresponding to the first sound signal, or obtain the periodic variation pattern of the second sound signal to obtain the sound characteristic period corresponding to the second sound signal.

[0058] To improve the accuracy of sound signal analysis and avoid noise interference, this embodiment can preprocess the sound signal after obtaining it. The preprocessing in this embodiment includes bandpass filtering and denoising. Specifically, this embodiment can filter the sound signal to retain the sound signal in a preset frequency band (e.g., 20-500Hz) to highlight the low-frequency impact characteristics of footsteps. Next, a dual-threshold energy detection method can be used to distinguish footsteps from background noise to remove background noise from the sound signal in the preset frequency band. For example, setting two energy thresholds and combining the zero-crossing rate to determine whether it is a valid footstep can better remove background noise and obtain a valid sound signal. In actual application, the sound acquisition module can be placed below the treadmill track at a distance of 5-10 cm from the track surface and fixed with a rubber bracket to isolate motor vibration interference.

[0059] Because the sound collection module is located on one side below the track, the sound signals collected by the left and right feet are different. For example, if the sound collection module is located on the left side below the track, the sound signal emitted by the left foot can be transmitted to the sound collection module more quickly, so the sound signal emitted by the left foot collected by the sound collection module is more obvious and clearer. Therefore, the sound collection module of this embodiment collects two sound signals, namely a first sound signal and a second sound signal, one of which corresponds to the footsteps of the left foot and the other corresponds to the footsteps of the right foot. This embodiment requires identifying and analyzing these two sound signals to extract the sound characteristic period.

[0060] Specifically, because footsteps closer to the sound collection module are less attenuated, the peak value of the signal collected by the sound collection module is larger. Therefore, this embodiment can separately obtain a first peak value corresponding to the first sound signal and a second peak value corresponding to the second sound signal. If the first peak value is greater than the second peak value, the first sound signal is determined to be emitted by a foot closer to the sound collection module, and the second sound signal is determined to be emitted by a foot farther from the sound collection module. For example, if the sound collection module is located on the left side below the track, the sound collection module is closer to the left foot, and the peak value of the left footstep sound signal will be larger. Therefore, the first sound signal corresponds to the left footstep, and the second sound signal corresponds to the right footstep. Since the first and second sound signals vary periodically while the user is running, the periodic variation patterns of the first and second sound signals can be obtained to obtain the sound characteristic period corresponding to the first and second sound signals. The sound characteristic period can include information such as peaks, troughs, fluctuation amplitude, rising edge slope, falling edge slope, and cycle duration. This embodiment can accurately determine the sound signals of the left and right feet based on the difference between the signal peaks, which facilitates accurate analysis of the user's movement posture in subsequent steps.

[0061] In another implementation, when determining the sound characteristic period, this embodiment may further include the following steps:

[0062] Step S11, respectively obtaining a first receiving time corresponding to the first sound signal and a second receiving time corresponding to the second sound signal;

[0063] Step S12: If the first receiving time is earlier than the second receiving time, determining that the first sound signal is a sound signal emitted by a foot close to the sound collection module, and determining that the second sound signal is a sound signal emitted by a foot far away from the sound collection module;

[0064] Step S13: Obtain the periodic variation pattern of the first sound signal to obtain the sound characteristic period corresponding to the first sound signal, or obtain the periodic variation pattern of the second sound signal to obtain the sound characteristic period corresponding to the second sound signal.

[0065] Because footsteps from a closer sound collection module can be collected more quickly, the sound signals of the left and right footsteps differ in reception time. To this end, this embodiment can separately obtain a first reception time corresponding to the first sound signal and a second reception time corresponding to the second sound signal. If the first reception time is earlier than the second reception time, the first sound signal is determined to be emitted by the foot closer to the sound collection module, and the second sound signal is determined to be emitted by the foot farther from the sound collection module. For example, if the sound collection module is located on the left side below the track, the sound collection module is closer to the left foot, and the sound signal of the left footstep will be received by the sound collection module earlier. Therefore, the first sound signal corresponds to the left footstep, and the second sound signal corresponds to the right footstep. Since the first and second sound signals change periodically while the user is running, the periodic variation patterns of the first and second sound signals can be obtained to obtain the sound characteristic period corresponding to the first and second sound signals.

[0066] Step S200: Based on the sound characteristic period, determine the gait period corresponding to the sound characteristic period, and the gait period is used to reflect the gait pattern of the user's movement.

[0067] After obtaining the sound characteristic period, this embodiment analyzes the post-sound characteristic period to determine the gait cycle. In this embodiment, the gait cycle includes the stance phase, swing phase, and stance phase, which constitute a complete gait cycle. The stance phase refers to the stage when the foot begins to contact the treadmill and bears weight, while the swing phase refers to the stage when the foot swings forward after leaving the ground. It should be noted that the stance phase does not necessarily exist. If the user does not pause significantly during exercise, the stance phase directly connects to and transitions from the swing phase, and the stance phase does not exist. During a user's running motion, taking a single leg as an example, during the stance phase, the sole of the foot contacts the ground, exerting vertical impact force and backward friction on the track. The impact, vibration, and friction between the sole of the foot and the track generate sound, and the sound signal increases during this period. Therefore, the corresponding sound characteristic period is the sound growth period. In particular, during the extension phase, muscles exert greater propulsion force, resulting in a peak sound signal. During the swing phase, after the foot leaves the ground, the force exerted on the treadmill by the user decreases, and the sound signal decays during this period. Therefore, the corresponding sound characteristic period is the sound decay period. When the user is in the standing phase, the sound signal approaches 0, and the corresponding sound characteristic period is the sound stable period. The sound enhancement period, the sound decay period, and the sound stable period constitute a complete sound characteristic period. Based on this, this embodiment can train a first mapping relationship and determine the gait period corresponding to the sound characteristic period based on this first mapping relationship.

[0068] Furthermore, because the movement of the left and right feet of the user is symmetrical when running, in order to facilitate the analysis of the movement postures of the left and right feet, reduce the amount of calculation, and improve the analysis efficiency, after obtaining the sound characteristic period of the first sound signal and the sound characteristic period of the second sound signal, because the first sound signal is the sound signal emitted by the foot close to the sound collection module, the first sound signal has less attenuation, is more accurate and clearer, so this embodiment can select the sound characteristic period of the first sound signal for subsequent analysis.

[0069] In one implementation, this embodiment includes the following steps when determining the gait cycle:

[0070] Step S201: Acquire a pre-trained first mapping relationship, where the first mapping relationship is used to reflect the correspondence between sound signal changes and gait;

[0071] Step S202: Match the sound feature period with the second mapping relationship to obtain a gait period corresponding to the sound feature period.

[0072] In actual application, this embodiment first trains a first mapping relationship, which is used to reflect the correspondence between the sound signal change and the gait, and can be used to determine the gait cycle corresponding to the sound feature cycle. Specifically, when the user is exercising on the sports assistive device, this embodiment collects sound signal samples under different gait samples, that is, collects sound signal samples corresponding to the user during the support period, swing period and standing period, and forms a sound signal set. Then, the sound sample period in which the sound signal samples under different gaits are located is further determined. At this time, there is a certain correspondence between the sound sample period and the gait sample, so the gait sample can be mapped to the sound sample period to obtain a first mapping relationship. The first mapping relationship at this time is: gait sample-sound sample period. Therefore, after obtaining the sound feature period based on the above step S100, the sound feature period is matched with the first mapping relationship to obtain the corresponding gait cycle. For example, if the determined sound characteristic period is a sound enhancement period, the corresponding gait period is a stance period; if the determined sound characteristic period is a sound decay period, the corresponding gait period is a swing period; if the determined sound characteristic period is a sound stabilization period, the corresponding gait period is a stance period. In addition, when mapping gait samples to sound sample periods, this embodiment may map the gait samples to sound features within the sound sample period, such as mapping features such as signal peaks or sound frequencies during the stance period to those during the sound enhancement period.

[0073] In other implementations, when training the first mapping relationship, this embodiment may also collect sound signal samples of multiple user samples of different genders, body types, and running habits at different gaits, thereby determining the sound sample period in which the sound signal samples of each user sample at different gaits fall. The sound sample periods of all user samples are then combined to obtain a combined sound sample period, and the combined sound sample period is then mapped to the gait samples to obtain the first mapping relationship. In this case, the first mapping relationship is also obtained based on training for different users and can therefore be applied to different users, thereby improving the applicability of this embodiment.

[0074] In other implementations, this embodiment can train a convolutional neural network model to obtain a gait analysis model for outputting a gait label. The input data of the gait analysis model is the collected sound signal. The gait analysis model can automatically analyze the sound signal, determine the corresponding sound characteristic period, and then determine the gait label corresponding to the sound characteristic period. Based on the gait label, the gait period at that time can be determined. Preferably, when analyzing the sound signal, the gait analysis model can automatically classify the waveform of the sound signal based on a clustering algorithm, identify the clustering results under different gaits, and then output the gait label corresponding to the input sound signal at that time.

[0075] In other implementations, this embodiment can also directly determine the signal intensity mean of each stage in the sound characteristic cycle based on the collected sound signal, and then compare the signal intensity mean with a preset intensity threshold. If the signal intensity mean of a certain stage is greater than the intensity threshold, then the gait cycle of that stage can be determined to be the support period. If the signal intensity mean of a certain stage is less than the intensity threshold, then the gait cycle of that stage can be determined to be the swing period. For example, if the signal intensity mean of the sound growth period is greater than the intensity threshold, then the gait cycle at that time is determined to be the support period. If the signal intensity mean of the sound decay period is less than the intensity threshold, then the gait cycle at that time is determined to be the swing period. Alternatively, this embodiment can also compare the sound signal at a certain moment with the intensity threshold to determine the support period and the swing period. For example, when the sound signal at a certain moment is greater than the intensity threshold, then it can be determined that the support period has begun. If the sound signal at a certain moment is less than the intensity threshold, then it can be determined that the support period has ended and the swing period has begun.

[0076] Step S300: Determine the user's movement posture based on the sound feature period and the gait period.

[0077] Since the sound characteristic period and gait period under different movement postures are different and have certain regularities, after determining the sound characteristic period and gait period, this embodiment can perform a comprehensive analysis of the sound characteristic period and gait period to determine the user posture.

[0078] In one implementation, this embodiment includes the following steps when determining the user's motion posture:

[0079] Step S301: determining the fluctuation amplitude of the sound signal based on the sound characteristic period;

[0080] Step S302: determining the duration of the gait cycle based on the gait cycle;

[0081] Step S303: Determine the user's movement posture based on the fluctuation amplitude of the sound signal and the period length of the gait cycle.

[0082] The sound characteristic cycle of this embodiment includes information such as peaks, troughs, fluctuation amplitude, rising slope, and falling slope. The gait cycle also reflects a wide range of information, such as gait frequency and cycle duration. This embodiment can extract the fluctuation amplitude of the sound signal based on the sound characteristic cycle, and extract the cycle duration of the gait cycle based on the gait cycle. The user's movement posture is then determined based on the fluctuation amplitude of the sound signal and the cycle duration of the gait cycle. Specifically, when determining the movement posture, if the fluctuation amplitude of the sound signal is zero, the movement posture is determined to be a stop posture. This embodiment can also determine that the movement posture is a stop posture when the average value of the sound signal remains unchanged for a preset period of time (e.g., 5 seconds) and the sound signal has no obvious periodic changes. If the fluctuation amplitude of the sound signal is less than a first amplitude value and the cycle duration of the gait cycle is greater than a cycle threshold, it indicates that the user's running speed is slow, and the movement posture can be determined to be a walking posture. In order to more accurately determine the motion posture, this embodiment can further analyze the duration ratio of the support period in the gait cycle at this time. If the support period accounts for as much as 60%-70% of the entire gait cycle, it can be more accurately determined that the user's motion posture at this time is a walking posture. If the fluctuation amplitude of the sound signal is greater than the second amplitude value and the cycle duration of the gait cycle is less than the cycle threshold, it means that the user's running speed is relatively fast at this time, and it can be determined that the motion posture is a running posture. When in the running posture, the user's swing period and support period are clearly demarcated. In this embodiment, the first amplitude value is less than the second amplitude value. For example, the first amplitude value is 8dB and the second amplitude value is 10dB. Similarly, this embodiment can further analyze the duration ratio of the swing period in the gait cycle at this time. If the swing period accounts for as much as 60%-70% of the entire gait cycle, it can be more accurately determined that the user's motion posture at this time is a running posture.

[0083] In another implementation, this embodiment can also extract the gait frequency of the gait cycle. The calculation formula for the gait frequency is 60 / T, where T is the cycle duration corresponding to the entire gait cycle. A comprehensive analysis is then performed based on the sound characteristic cycle and the gait frequency to determine the movement posture. Specifically, when determining the movement posture, if the gait frequency is 0, it indicates that the user is not running at this time, and the movement posture can be determined to be a stop posture. If the fluctuation amplitude of the sound signal is less than a first amplitude value and the gait frequency of the gait cycle is less than a step frequency threshold, for example, the gait frequency is less than 120 steps / minute, it indicates that the user's running speed is relatively slow, and the movement posture can be determined to be a walking posture. If the fluctuation amplitude of the sound signal is greater than the first amplitude value and the gait frequency of the gait cycle is greater than the step frequency threshold, that is, the gait frequency is greater than 120 steps / minute, it indicates that the user's running speed is relatively fast, and the movement posture can be determined to be a running posture. Since the electrical sound characteristic cycle and the gait cycle reflect more than one parameter, multiple parameter combinations can be used to analyze the movement posture.

[0084] It should be noted that if the motion of the user's left and right feet is identical, this embodiment can analyze the motion posture of a single foot as an example by analyzing the sound characteristic cycle and gait cycle of a single foot (e.g., the left foot). Of course, this embodiment can also use the above-mentioned method to analyze the motion posture of the left and right feet separately based on the sound characteristic cycle and gait cycle, and then determine whether the motion postures of the left and right feet are identical. If they are not identical, it can be determined that an abnormality has occurred. Furthermore, when the user is in a running posture, this embodiment can further obtain the user's motion parameters in the running posture and then analyze the motion parameters. For example, the obtained motion parameters can be compared with preset reference motion parameters to analyze whether the user's running posture is abnormal. If an abnormality is present, such as single-foot slipping, alternating running, or double-handed support (falling), a pop-up window or sound alarm can be automatically displayed on the user's mobile terminal, exercise-assistance device, or wearable device to remind the user to ensure exercise safety. If the alarm duration exceeds a preset time, a control instruction can be issued to the exercise-assistance device, such as controlling the treadmill to slow down or even stop operation. In addition, this embodiment can also monitor the operating status of the sports assistive device in real time when the user is in a running posture. If any abnormality is found, a pop-up window prompt or sound alarm prompt can also be issued to ensure the user's exercise safety.

[0085] In addition, after analyzing the movement posture, this embodiment can also upload the movement posture to a preset virtual reality device and generate a virtual animation based on the movement posture. This virtual animation is generated in real time based on the user's movement posture and is synchronized with the user's movement posture. This helps the user understand their movement posture in real time and make adjustments if it is not standard.

[0086] Furthermore, this embodiment can also analyze the user's motion posture information by collecting current load data from a motion-assisting device. Specifically, taking running as an example, the motion-assisting device in this embodiment is a treadmill. Since the current load data on the treadmill changes periodically when the user is running, and the user's running gait also changes periodically, the motion posture can be analyzed based on the periodic variation pattern of the current load data and the gait. To this end, this embodiment can collect the current load data of the motion-assisting device in real time while the user is exercising, and then determine the current cycle corresponding to the current load data based on a second mapping relationship. The second mapping relationship in this embodiment is: current load sample - current sample cycle. Based on this second mapping relationship, the current load data of the motion-assisting device can be collected in real time while the user is running, and then the current load data can be matched with the second mapping relationship to obtain the corresponding current cycle. The current cycle includes: a current load rising period, a current load falling period, and a current load stable period.

[0087] Changes in the current load data of an exercise-assistance device (such as a treadmill) essentially reflect the power demand placed on the device by the user while running. This power demand is directly related to the mechanical characteristics of each phase of the gait cycle. In this embodiment, the gait cycle includes the stance phase, swing phase, and stance phase, which together constitute a complete gait cycle. For example, during a user's running motion, using a single leg as an example, during the stance phase, the foot strikes the ground, exerting vertical impact force and backward friction on the treadmill. The treadmill motor must increase torque to maintain the treadmill speed. During this phase, the treadmill's current load data increases significantly, thus corresponding to the current load rising phase. Especially during the extension phase, when muscles exert greater propulsion, the current load data reaches a peak. During the swing phase, after the user's foot leaves the ground, the force exerted on the treadmill decreases, and the treadmill motor only needs to maintain basic treadmill operation. At this point, the current load data decreases to a stable level, thus corresponding to the current load falling phase. During the stance phase, the treadmill's current load data remains stable for a period of time, corresponding to the current load stabilization phase. The current load rising period, the current load falling period and the current load stabilization period constitute a complete current cycle. Based on this, the present embodiment trains the third mapping relationship, and determines the gait cycle corresponding to the current cycle at this time based on the third mapping relationship. The third mapping relationship of the present embodiment is: gait sample-current sample cycle. After obtaining the current cycle, the current cycle is matched with the third mapping relationship to obtain the corresponding gait cycle. For example, if the current cycle determined is the current load rising period, the corresponding gait cycle is the support period; if the current cycle determined is the current load falling period, the corresponding gait cycle is the swing period; if the current cycle determined is the current load stabilizer, the corresponding gait cycle is the standing period.

[0088] Because the current cycle and gait cycle under different motion postures are different and follow certain regularities, once the current cycle and gait cycle are obtained, this embodiment can perform a comprehensive analysis based on the current cycle and the gait cycle to determine the user's posture. Specifically, the current characteristics within the current cycle and the gait characteristics within the gait cycle are extracted, and then a comprehensive analysis is performed based on the current characteristics and the gait characteristics. The current characteristics in this embodiment include the current fluctuation amplitude and current frequency reflected by the entire current cycle, and the gait characteristics include: cycle duration or gait frequency.

[0089] In actual application, when the gait characteristic is the cycle duration, if the current fluctuation amplitude and the current frequency are both 0, the motion posture is determined to be a stop posture; if the current fluctuation amplitude is less than the first amplitude value and the current frequency is less than the first frequency value, and the cycle duration of the gait cycle is greater than the cycle threshold, the motion posture is determined to be a walking posture; if the current fluctuation amplitude is greater than the second amplitude value and the current frequency is greater than the second frequency value, and the cycle duration of the gait cycle is less than the cycle threshold, the motion posture is determined to be a running posture. In another implementation, when the gait characteristic is the gait frequency, if the gait frequency is 0, the motion posture is determined to be a stop posture; if the current fluctuation amplitude is less than the first amplitude value and the current frequency is less than the first frequency value, and the gait frequency of the gait cycle is less than the step frequency threshold, the motion posture is determined to be a walking posture; if the current fluctuation amplitude is greater than the second amplitude value and the current frequency is greater than the second frequency value, and the gait frequency of the gait cycle is greater than the step frequency threshold, the motion posture is determined to be a running posture. It can be seen from this that this embodiment can also analyze motion posture information based on current load data.

[0090] Furthermore, the present embodiment can set the motion posture information determined based on the current load data as the posture information to be verified. Then, the motion posture determined based on the sound signal is matched with the posture information to be verified, and the correctness of the posture information to be verified is determined based on the matching result. Specifically, if at the same moment, the motion posture determined based on the sound signal is the same as the motion posture information that can be analyzed based on the current load data, for example, both are running postures, then it can be determined that the motion posture information that can also be analyzed based on the current load data is correct. If at the same moment, the motion posture determined based on the sound signal is different from the motion posture information that can also be analyzed based on the current load data, then it can be determined that the motion posture information that can also be analyzed based on the current load data is wrong. The present embodiment can verify the accuracy of other methods of analyzing motion postures by using the motion posture analyzed based on the present embodiment. On the one hand, it can verify the analysis accuracy of other methods, and on the other hand, it can also analyze the accuracy of the method of the present embodiment, which can more effectively protect the user's sports safety.

[0091] Based on the above embodiment, the present invention further provides a motion analysis device based on sound feature recognition, which is used to implement any one of the steps in the above method embodiment, such as Figure 2As shown in , the device includes: a sound feature recognition module 10, a gait cycle determination module 20, and a motion posture determination module 30. Specifically, the sound feature recognition module 10 is used to determine the sound feature cycle based on the sound signal collected in real time by the sound acquisition module when the user is exercising, and the sound feature cycle is used to reflect the changing pattern of the sound signal. The sound acquisition module is set on one side of the exercise assistive device. The gait cycle determination module 20 is used to determine the gait cycle corresponding to the sound feature cycle based on the sound feature cycle, and the gait cycle is used to reflect the gait pattern of the user's exercise. The motion posture determination module 30 is used to determine the user's motion posture based on the sound feature cycle and the gait cycle.

[0092] The working principles of each module in the motion analysis device based on sound feature recognition in this embodiment are the same as the principles of each step in the above method embodiment, and will not be repeated here.

[0093] Each module in the aforementioned motion analysis device based on sound feature recognition can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a terminal in the form of hardware, or can be stored in a memory in the terminal in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0094] Based on the above embodiment, the present invention further provides a terminal, the principle block diagram of the terminal can be as follows: Figure 3 The terminal may include one or more processors 100 ( Figure 3 The processors 100 may further include a memory 101 (only one of which is shown), and a computer program 102 stored in the memory 101 and executable on one or more processors 100. For example, a motion analysis program based on sound feature recognition may be employed. When one or more processors 100 execute computer program 102, the processors 100 may implement the various steps of an embodiment of a motion analysis method based on sound feature recognition. Alternatively, when one or more processors 100 execute computer program 102, the processors 100 may implement the functions of various modules / units in an embodiment of a motion analysis system based on sound feature recognition, without limitation.

[0095] In one embodiment, the processor 100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0096] In one embodiment, memory 101 may be an internal storage unit of an electronic device, such as a hard drive or memory. Memory 101 may also be an external storage device of the electronic device, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash memory card, etc. Furthermore, memory 101 may include both an internal storage unit of the electronic device and an external storage device. Memory 101 is used to store computer programs and other programs and data required by the terminal. Memory 101 may also be used to temporarily store data that has been output or is about to be output.

[0097] Those skilled in the art will understand that Figure 3 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the solution of the present invention, and does not constitute a limitation on the terminal to which the solution of the present invention is applied. The specific terminal may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0098] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, operation database or other media used in the embodiments provided by the present invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), Direct Memory Bus Dynamic RAM (DRDRAM), and Rambus Dynamic RAM (RDRAM), among others.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A motion analysis method based on sound feature recognition, characterized in that: The method comprises: When the user exercises, a sound characteristic period is determined based on the sound signal collected in real time by the sound collection module, and the sound characteristic period is used to reflect the change pattern of the sound signal. The sound collection module is provided on one side of the exercise assisting device; Based on the sound characteristic period, determining a gait period corresponding to the sound characteristic period, wherein the gait period is used to reflect a gait pattern of the user's movement; determining a user's movement posture based on the sound characteristic period and the gait period; The determining the user's motion posture based on the sound characteristic period and the gait period includes: determining a fluctuation amplitude of the sound signal based on the sound characteristic period; Based on the gait cycle, determining a period length of the gait cycle; determining a user's movement posture based on the fluctuation amplitude of the sound signal and the period length of the gait cycle; The determining of the user's motion posture based on the fluctuation amplitude of the sound signal and the period duration of the gait cycle includes: If the fluctuation amplitude of the sound signal is 0, determining that the motion posture is a stop posture; If the fluctuation amplitude of the sound signal is less than the first amplitude value and the period of the gait cycle is greater than the period threshold, and the proportion of the duration of the stance period in the gait cycle is as high as 60%-70% of the entire gait cycle, the motion posture is determined to be a walking posture; If the fluctuation amplitude of the sound signal is greater than the second amplitude value and the period duration of the gait cycle is less than the period threshold, and the proportion of the duration of the swing period in the gait cycle is as high as 60%-70% of the entire gait cycle, the motion posture is determined to be a running posture, wherein the first amplitude value is less than the second amplitude value; or, The determining the user's motion posture based on the sound characteristic period and the gait period includes: extracting a gait frequency of the gait cycle; If the gait frequency is 0, determining that the movement posture is a stop posture; If the fluctuation amplitude of the sound signal is less than a first amplitude value and the gait frequency of the gait cycle is less than a gait frequency threshold, determining that the motion posture is a walking posture; If the fluctuation amplitude of the sound signal is greater than a first amplitude value and the gait frequency of the gait cycle is greater than a gait frequency threshold, it is determined that the movement posture is a running posture.

2. The motion analysis method based on sound feature recognition according to claim 1, characterized in that: The sound signal includes a first sound signal and a second sound signal, and determining the sound characteristic period based on the sound signal includes: respectively obtaining a first peak value corresponding to the first sound signal and a second peak value corresponding to the second sound signal; If the first peak value is greater than the second peak value, it is determined that the first sound signal is a sound signal emitted by the foot close to the sound collection module, and the second sound signal is determined to be a sound signal emitted by the foot far away from the sound collection module; Obtain the periodic variation pattern of the first sound signal to obtain the sound characteristic period corresponding to the first sound signal, or obtain the periodic variation pattern of the second sound signal to obtain the sound characteristic period corresponding to the second sound signal.

3. The motion analysis method based on sound feature recognition according to claim 1, characterized in that: The sound signal includes a first sound signal and a second sound signal, and determining the sound characteristic period based on the sound signal includes: respectively obtaining a first reception time corresponding to the first sound signal and a second reception time corresponding to the second sound signal; If the first receiving time is earlier than the second receiving time, determining that the first sound signal is a sound signal emitted by a foot close to the sound collection module, and determining that the second sound signal is a sound signal emitted by a foot far away from the sound collection module; Obtain the periodic variation pattern of the first sound signal to obtain the sound characteristic period corresponding to the first sound signal, or obtain the periodic variation pattern of the second sound signal to obtain the sound characteristic period corresponding to the second sound signal.

4. The motion analysis method based on sound feature recognition according to claim 1, characterized in that: The determining, based on the sound characteristic period, a gait period corresponding to the sound characteristic period includes: Acquiring a pre-trained first mapping relationship, where the first mapping relationship is used to reflect a correspondence between a change in the sound signal and a gait; Matching the sound characteristic period with the first mapping relationship to obtain a gait period corresponding to the sound characteristic period, wherein the sound characteristic period includes a sound enhancement period, a sound attenuation period, and a sound stabilization period, and the gait period includes a support period, a swing period, and a stance period; The method for determining the first mapping relationship includes: When the user exercises on the exercise assisting device, collecting sound signal samples under different gait samples, and determining the sound sample periods of the sound signal samples under different gaits; The gait sample is mapped to the sound sample period to obtain a first mapping relationship.

5. The motion analysis method based on sound feature recognition according to claim 1, characterized in that: The method further comprises: When the user is exercising, current load data of the exercise assisting device is collected in real time, and exercise posture information is determined based on the current load data, and the exercise posture information determined based on the current load data is set as the posture information to be verified; The motion posture determined based on the sound signal is matched with the posture information to be verified, and the correctness of the posture information to be verified is determined based on the matching result.

6. A motion analysis device based on sound feature recognition, characterized in that: The device is used to implement the steps of the motion analysis method based on sound feature recognition according to any one of claims 1 to 5, and the device includes: A sound feature recognition module is configured to identify a sound feature period based on the sound signal collected in real time by the sound collection module when the user is exercising. The sound feature period is configured to reflect the changing pattern of the sound signal. The sound collection module is disposed on one side of the exercise assistive device. a gait cycle determination module, configured to determine a gait cycle corresponding to the sound characteristic cycle based on the sound characteristic cycle, wherein the gait cycle is used to reflect the gait pattern of the user's movement; The motion posture determination module is used to determine the user's motion posture based on the sound feature period and the gait period.

7. A terminal, characterized in that: The terminal includes a memory, a processor, and a motion analysis program based on sound feature recognition stored in the memory and runnable on the processor. When the processor executes the motion analysis program based on sound feature recognition, the steps of the motion analysis method based on sound feature recognition as described in any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a motion analysis program based on sound feature recognition, and the motion analysis program based on sound feature recognition implements the steps of the motion analysis method based on sound feature recognition according to any one of claims 1 to 5 on the computer-readable storage medium.

Images