Non-contact fitness action recognition method based on near ultrasonic signals

Abstract

The invention discloses a non-contact fitness action recognition method based on near ultrasonic signals. The method comprises the steps that a microphone and a loudspeaker are arranged in a fitness exercise area, the loudspeaker is used for emitting near ultrasonic signals, and the microphone collects echo signals; filtering the collected echo signals to obtain an amplitude spectrogram corresponding to each standard action, then segmenting the amplitude spectrograms to obtain standard action image blocks, and constructing a fitness action data set by using the standard action image blocks; training of an action recognition model is completed through the body-building action data set, echo signals of a body builder in the body-building process are collected through a microphone, the body-building action echo signals are sequentially filtered and segmented, then a result is input into the trained action recognition model, and a body-building action recognition result of the body builder is output. According to the invention, the problem that the recognition of the fitness activity generally depends on a fitness coach, vision or wearable equipment is solved, and the convenience of home fitness and the training recognition accuracy are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of activity recognition, and in particular relates to a non-contact fitness action recognition method based on near-ultrasonic signals. Background Art

[0002] With rapid economic development and the continuous improvement of people's daily quality of life, more and more people are paying attention to their health and body shape, leading to a rapid increase in the popularity of fitness. Effective fitness can bring many benefits, such as increasing muscle strength, improving body shape, and reducing the risk of cardiovascular disease. However, due to the accelerated pace of life, it has become inconvenient for many people to go to a dedicated gym. In addition, gym memberships and consultations with personal trainers are usually expensive. Therefore, people have begun to exercise at home or in the office, which helps save time and money. Fitness enthusiasts hire fitness coaches for effective supervision and guidance, or use mobile phone cameras and fitness apps to follow along. However, these methods carry the risk of personal privacy leakage and are not suitable for private settings such as offices and homes.

[0003] At the same time, many people use wearable devices such as smartwatches to monitor their fitness. However, due to their characteristics and functional requirements, these devices often come into direct contact with the skin, which hinders the freedom of fitness exercise to a certain extent. Furthermore, these devices cannot be used properly while charging. Therefore, it has become urgent to develop a non-invasive, easy-to-deploy, and interference-resistant fitness monitoring method to provide detailed fitness statistics and help improve fitness results. Summary of the Invention

[0004] In order to solve the problems existing in the background technology, the purpose of the present invention is to provide a non-contact fitness action recognition method based on near-ultrasonic signals, aiming to solve the problem that the existing technology for identifying fitness activities usually relies on fitness coaches, vision or wearable devices, which may waste money, leak user privacy, cause discomfort to users, and affect the quality of life.

[0005] The technical solution adopted by the present invention is as follows, comprising the following steps:

[0006] Step S1: First, a microphone and a speaker are arranged in the fitness exercise area, the speaker is used to emit a near-ultrasonic chirp signal, and the microphone is used to collect the echo signal of the fitness movement;

[0007] Step S2: filtering the fitness action echo signal collected in step S1 to obtain an amplitude spectrum, then segmenting the amplitude spectrum to obtain standard action image blocks, and constructing a fitness action dataset using the standard action image blocks;

[0008] Step S3: Next, construct an action recognition model, input the fitness action data set into the constructed action recognition model for training, and obtain a trained action recognition model;

[0009] The action recognition model of the present invention can adopt a conventional image classification neural network.

[0010] Step S4. Finally, use a microphone to collect the echo signal of the fitness person during the fitness process, filter and segment the fitness action echo signal in sequence to obtain a standard action image block, and then input the standard action image block into the trained action recognition model to output the recognition result of the fitness person's fitness action.

[0011] Step S3 uses the fitness action dataset to train the DenseNet-264 network. When the exerciser performs fitness activities in the fitness area, the collected user action echo data is analyzed to obtain the amplitude spectrum corresponding to the user's fitness movement and perform action unit segmentation. The resulting data is input into the network for fitness action recognition. Specifically, the fitness action dataset is randomly divided into three sub-datasets, with a training dataset accounting for 75%, a validation dataset accounting for 15%, and a test dataset accounting for 10%. The DenseNet-264 network is used as the deep learning network for action recognition. Specifically, a DenseNet structure with four dense blocks is used, the input image size is 224×224, and the growth rate k is 32.

[0012] The step S1 is specifically as follows:

[0013] First, in a fitness exercise area, a microphone and a speaker are arranged at the boundary of the fitness exercise area. The microphone and the speaker are located at the same height and both face the front of the fitness person. Then, the speaker is used to transmit a customized near-ultrasonic chirp signal to the fitness person, and the microphone is used to receive the echo signal reflected by the fitness person during various standard movements when performing fitness exercises. The transmission frequency of the customized near-ultrasonic chirp signal is 17kHz-20kHz, and the total signal transmission time is 3 to 5 minutes. The standard movements include squats, lunges, dumbbell presses, dumbbell lateral raises, dumbbell front raises and dumbbell flyes.

[0014] In the specific implementation, a rectangular area was selected as the fitness area. The microphone and speaker were respectively a mobile phone microphone and a commercial-grade speaker JBL GO 2. The microphone and speaker were placed on the same plane on the left side of the fitness area. Both were at the same height, 1.2 meters above the ground, and both were facing the fitness person. A commercial-grade speaker was used to send a customized near-ultrasonic chirp signal with a total duration of 3 minutes and a frequency of 17kHz-20kHz. Each frame lasted for T s=30ms, the inter-frame interval is T interval =20ms, total time T ref = 50ms, with a refresh rate of 20Hz. The phone's recording function is enabled, and the phone's microphone receives echo signals reflected from a person performing six fitness exercises: squats, lunges, dumbbell presses, dumbbell lateral raises, dumbbell front raises, and dumbbell flyes. The resulting audio files are in the 48Hz, 16-bit WAV format.

[0015] In step S2, the specific method of filtering the echo signal of the fitness action to obtain the amplitude spectrum is as follows:

[0016] Step S2.1.1. First, synchronously input the echo signals collected M times under the same standard action into a Butterworth bandpass filter for filtering to obtain the echo signal y[n] with a frequency range of 14 kHz-23 kHz. The Butterworth bandpass filter is used to remove noise and other irrelevant frequencies, retaining only the frequency components in the 14 kHz-23 kHz range.

[0017] Step S2.1.2: The near-ultrasonic chirp signal emitted by the loudspeaker is then used as the reference signal x[n], and the cross-correlation function R between the filtered echo signal y[n] and the reference signal is calculated. xy The expression for [k] is as follows:

[0018]

[0019] Where n is the number of sampling points of the signal; k is the sample delay;

[0020] Step S2.1.3: Then use the peak extraction algorithm to calculate the cross-correlation function R between the echo signal and the reference signal. xy [k] performs peak extraction and obtains the cross-correlation function R xy The maximum peak of [k];

[0021] Step S2.1.4: Extract the cross-correlation function R xy The echo signal y[n] after the maximum peak of [k] is used as the target signal y'[n], and the pseudo transmission signal s'[n] is defined by the reference signal. The pseudo transmission signal is defined as a reference signal with an equal time-bandwidth ratio extension. The target signal and the pseudo transmission signal are processed according to the following formula to obtain the conversion signal xs[n]:

[0022] xs[n]=y[n]·s'[n]

[0023]

[0024] Among them, f s Indicates the initial frequency of the near-ultrasonic signal; BS represents the bandwidth of the pseudo transmission signal; Ts represents the duration of the pseudo transmission signal; π represents the circumference of the circle; n represents the sampling point, and the conversion signal is the product of the target signal and the pseudo transmission signal;

[0025] Step S2.1.5: Perform fast Fourier transform on the converted signal xs[n], with the number of Fourier transform points being n. fft , so that the conversion signal xs[n] is converted from a time domain signal to a frequency domain signal XF[n fft ], the frequency domain signal XF[n fft ] is combined according to the frame number sequence of the near-ultrasonic chirp signal to obtain the frequency domain signal matrix XF[m,n fft ], m represents the number of frames of the near-ultrasound chirp signal;

[0026] Step S2.1.6: The frequency domain signal matrix XF[m,n fft ] to update:

[0027]

[0028] Where di represents the distance from the i-th obstacle in the fitness area to the microphone; fi is the i-th frequency value in the conversion signal; Ts is the duration of the pseudo-transmission signal, B s is the bandwidth of the pseudo-transmission signal, v is the speed of sound, v = 343 m / s; g() represents the integer function;

[0029] Next, the frequency domain signal matrix XF[m,n fft All elements in the di-th row in ] are replaced with 0 values, and the updated frequency domain signal matrix is ​​obtained as the target signal matrix XF′[m,n fft ];

[0030] Step S2.1.7. Finally, the target signal matrix XF′[m,n fft ] to perform sliding window MTI dynamic filtering and obtain the amplitude spectrum P[m,n fft ]:

[0031] P[m,n fft ]=|P′[m,n fft ]|

[0032] P′[m,n fft ]=XF′[m,n fft ]-4XF′[m-1,n fft ]+6XF′[m-2,n fft ]-4XF′[m

[0033] -3,nfft ]+XF′[m-4,n fft ]

[0034] Among them, P′[m,n fft ] represents the dynamic filtering result; || represents the absolute value function.

[0035] Sliding window MTI dynamic filtering is used to remove interference from static objects and retain only the moving target signal. The data after dynamic filtering is obtained by taking the absolute value to obtain an amplitude spectrum. Each standard action corresponds to an amplitude spectrum.

[0036] In step S2, the specific method of segmenting the amplitude spectrum to obtain standard motion image blocks is as follows:

[0037] Step S2.2.1. First, extract the maximum pixel value in the amplitude spectrum and mark the pixels with a value greater than 45% of the maximum pixel value in the amplitude spectrum. Use a 3×3 traversal window to traverse each marked pixel. Observe whether the pixels above, below, left, right, upper left, lower left, upper right, and lower right of the marked pixel are marked within the traversal window centered on the marked pixel. If more than six pixels are marked within the traversal window, use that traversal window as the target traversal window.

[0038] Step S2.2.2: When there is an overlapping area between two target traversal windows, the union of the areas where the two target traversal windows are located is used as the new target traversal window;

[0039] Step S2.2.3: Repeat step S2.2.2 until there is no overlapping area between all target traversal windows. The smallest rectangular area containing each target traversal window is used as each connected area. That is, one rectangular connected area covers one target traversal window to obtain multiple connected areas.

[0040] Step S2.2.4: Obtain the maximum length and maximum width of all connected regions, and use them as the target length and target width, respectively. Expand the boundaries of each connected region so that the length and width of each connected region reach the target length and target width, respectively, and the connected region before boundary expansion is in the middle of the connected region after boundary expansion.

[0041] Step S2.2.5. Finally, the connected region after boundary expansion is cut in the amplitude spectrum to obtain several image blocks as standard action image blocks.

[0042] The fitness action dataset in step S2 includes the cut standard action image blocks and their corresponding labels. The labels of the fitness action dataset are the actual standard actions corresponding to the standard action image blocks, and the labels are one of squats, lunges, dumbbell presses, dumbbell lateral raises, dumbbell front raises, and dumbbell flyes.

[0043] The action recognition model in step S3 adopts the existing image classification neural network DenseNet-264.

[0044] The expression of the near-ultrasonic chirp signal x[n] is as follows:

[0045]

[0046] Among them, f s represents the initial frequency of the near-ultrasonic signal; Bs represents the bandwidth of the near-ultrasonic signal; T s represents the duration of the near-ultrasonic signal; π represents the circumference of a circle; and n represents the sampling point.

[0047] The expression of the echo signal y[n] received by the microphone is as follows:

[0048]

[0049] Among them, n l represents the attenuation coefficient of the lth propagation path; N represents the total number of signal propagation paths.

[0050] The beneficial effects of the present invention are:

[0051] 1. The present invention uses near-ultrasonic signals that are within the human ear's audible limit for action recognition, taking into account the limitations of receiving and transmitting devices in daily life and the user's fitness experience, achieving a high recognition rate.

[0052] 2. Combining MTI dynamic filtering and bandpass filtering, an efficient processing method is adopted for the echo signals of users' fitness exercises. This method takes into account the distance range of the user's exercise, eliminates the influence of irrelevant obstacles and static objects in the surrounding area, improves the utilization of echo information, extracts the unique characteristics of each type of movement, and reduces the complexity of calculation.

[0053] 2. The present invention performs sliding window splitting on the amplitude spectrum, thereby improving the efficiency of image block activity segmentation.

[0054] 3. The present invention overcomes the problem that identifying fitness activities usually relies on fitness coaches, vision or wearable devices, which may waste money, leak user privacy, cause discomfort to users, and affect the quality of life. It improves the convenience of home fitness and the accuracy of training recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 is a flow chart of the non-contact fitness action recognition method of the present invention;

[0056] Figure 2 2 is a schematic diagram of the structure of the DenseNet-264 network in an embodiment of the present invention;

[0057] Figure 3 is a loss curve diagram of the network training process in an embodiment of the present invention;

[0058] Figure 4 1 is a precision-recall curve of a user fitness data test in an embodiment of the present invention. DETAILED DESCRIPTION

[0059] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numbers in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present application. Rather, they are merely examples of methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0060] The terms used in the present invention are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms "a," "the," and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0061] like Figure 1 As shown, the present invention provides a non-contact fitness action recognition method based on near-ultrasonic signals, which may include the following steps:

[0062] Step S1: First, a microphone and a speaker are arranged in the fitness exercise area, the speaker is used to emit a near-ultrasonic chirp signal, and the microphone is used to collect the echo signal of the fitness movement;

[0063] Wherein, step S1 is specifically as follows:

[0064] First, in the fitness area, the microphone and speaker are placed at the boundary of the fitness area, at the same height, and both are facing the front of the fitness person; then, the speaker is used to transmit a customized near-ultrasonic chirp signal to the fitness person, and the microphone is used to receive the echo signal reflected by the fitness person during various standard movements during fitness exercise. The transmission frequency of the customized near-ultrasonic chirp signal is 17kHz-20kHz, and the total signal transmission time is 3 to 5 minutes. The standard movements include squats, lunges, dumbbell presses, dumbbell lateral raises, dumbbell front raises and dumbbell flyes.

[0065] In the specific implementation, a rectangular area is selected as the fitness area, and the microphone and speaker are respectively a mobile phone microphone and a commercial-grade speaker JBL GO 2. The microphone and speaker are arranged on the same plane of the left boundary of the fitness area and at the same height, both 1.2 meters above the ground. Both are facing the fitness person. A commercial-grade speaker is used to send a customized near-ultrasonic chirp signal with a total duration of 3 minutes and a frequency of 17kHz-20kHz. Each frame lasts for 30ms and the interval between frames is 20ms. The recording function of the mobile phone is turned on, and the mobile phone microphone is used to receive the reflected echo signals when the human body performs 6 kinds of fitness activities. The 6 kinds of fitness exercises include squats, lunges, dumbbell presses, dumbbell lateral raises, dumbbell front raises, and dumbbell flyes, forming audio files in the wav format.

[0066] The expression of the near-ultrasonic chirp signal x[n] is as follows:

[0067]

[0068] Among them, f s represents the initial frequency of the near-ultrasonic signal; Bs represents the bandwidth of the near-ultrasonic signal; T s represents the duration of the near-ultrasonic signal; π represents the circumference of a circle; and n represents the sampling point.

[0069] The expression of the echo signal y[n] received by the microphone through different propagation paths is as follows:

[0070]

[0071] Among them, n l represents the attenuation coefficient of the lth propagation path; N represents the total number of signal propagation paths.

[0072] Step S2: filtering the fitness action echo signal collected in step S1 to obtain an amplitude spectrum, then segmenting the amplitude spectrum to obtain standard action image blocks, and constructing a fitness action dataset using the standard action image blocks;

[0073] In step S2, the echo signal of the fitness action is filtered to obtain the amplitude spectrum as follows:

[0074] Step S2.1.1: First, the echo signals collected M times under the same standard action are synchronously input into the Butterworth bandpass filter for filtering to obtain the echo signal y[n] with a frequency range of 14kHz-23kHz. The Butterworth bandpass filter is used to remove noise and other irrelevant frequencies, retaining only the frequency components within the range of 14kHz-23kHz. The bandpass filter adopts the Butterworth filter, which is a filter with a flat frequency response and no ripple. It has a flat gain response in the passband and is suitable for smoothly filtering the frequency components in the signal. Its characteristics are: no ripple, thus avoiding unnecessary amplification or attenuation of the frequency components of the signal, and suitable for use in various signal processing. The bandpass range of the filter is selected from 14000Hz to 23000Hz to filter out frequency components outside this range and retain key signals. The 6th-order Butterworth filter is used in the step, which can achieve rapid attenuation while maintaining a relatively flat frequency response. Its transfer function |H(jω)| is:

[0075]

[0076] Where ω is the angular frequency of the signal; ω0 is the center frequency of the filter; ω c is the bandwidth, defined as ω c =ω2-ω1, where ω1 and ω2 are the lower cutoff frequency and the upper cutoff frequency respectively; N' is the order of the filter;

[0077] Step S2.1.2: The near-ultrasonic chirp signal emitted by the loudspeaker is then used as the reference signal x[n], and the cross-correlation function R between the filtered echo signal and the reference signal is xy The expression for [k] is as follows:

[0078]

[0079] Where n is the number of sampling points of the signal; k is the sample delay;

[0080] Signal cross-correlation is used to measure the delay between two signals, determine the time difference between the signals (TOA, Time of Arrival), and calculate the matching degree between the echo signal y(t) and the reference signal x(t) to identify the target echo and estimate the distance to the target. For two signals, if the signals are continuous signals, their cross-correlation function R xy (τ) is defined as:

[0081]

[0082] Where t is the time variable and τ is the time delay parameter. The cross-correlation function represents the similarity between signals x(t) and y(t) at different time delays. For discrete signals x[n] and y[n], the cross-correlation is defined as:

[0083]

[0084] When R xy When [k] reaches its maximum value, it means that the signals x[n] and y[n] are most similar at a delay of k samples.

[0085] Step S2.1.3: Then use the peak extraction algorithm to calculate the cross-correlation function R between the echo signal and the reference signal. xy [k] performs peak extraction and obtains the cross-correlation function R xy The maximum peak of [k];

[0086] Step S2.1.4: Extract the cross-correlation function R xy The echo signal y[n] after the maximum peak of [k] is used as the target signal y'[n], and the expression of the target signal y'[n] is as follows:

[0087] y'[n]=y[argmax(R xy [k])-T interval / 4:argmax(R xy [k])+T ref ]

[0088] Among them, argmax(R xy [k]) represents the cross-correlation function R xy [k] maximum peak; argmax(R xy [k])-T interval / 4:argmax(R xy [k])+T ref Indicates the length of the intercepted signal; argmax(R xy [k])-T interval / 4 indicates the start interception time of the target signal;

[0089] Then, the pseudo transmission signal s'[n] is defined using the reference signal. The conversion signal xs[n] is obtained by processing the target signal and the pseudo transmission signal according to the following formula:

[0090] xs[n]=y[n]·s'[n]

[0091]

[0092] Among them, f s Indicates the initial frequency of the near-ultrasonic signal; B Srepresents the bandwidth of the pseudo transmission signal; Ts represents the duration of the pseudo transmission signal; π represents the circumference of the circle; n represents the sampling point, and the conversion signal is the product of the target signal and the pseudo transmission signal;

[0093] Step S2.1.5: Perform fast Fourier transform on the converted signal xs[n], with the number of Fourier transform points being n. fft , so that the conversion signal xs[n] is converted from a time domain signal to a frequency domain signal XF[n fft ], the frequency domain signal XF[n fft ] is combined according to the frame number sequence of the near-ultrasonic chirp signal to obtain the frequency domain signal matrix XF[m,n fft ], m represents the number of frames of the near-ultrasound chirp signal;

[0094] In a specific implementation, the frequency domain signal XF[n fft ] is the number of rows n fft The column vector of the row, m frequency domain signals XF[n fft ] After combining the frame number sequence of the near-ultrasonic chirp signal, the number of rows is n fft The matrix with m rows and columns is the frequency domain signal matrix XF[m,n fft ];

[0095] Step S2.1.6: The frequency domain signal matrix XF[m,n fft ] to update:

[0096]

[0097] Where di represents the distance from the i-th obstacle in the fitness area to the microphone; fi is the i-th frequency value in the conversion signal; Ts is the duration of the pseudo-transmission signal, B s is the bandwidth of the pseudo-transmission signal, v is the speed of sound, v = 343 m / s; g() represents the integer function;

[0098] Next, the frequency domain signal matrix XF[m,n fft All elements in the di-th row in ] are replaced with 0 values, and the updated frequency domain signal matrix XF′[m,n fft ];

[0099] The Fourier transform of the discrete signal x'[n] is expressed as:

[0100]

[0101] Where X′[k] represents the frequency domain signal after fast Fourier transform; x'[n] represents the time domain signal before fast Fourier transform; N is the signal length, ks represents the frequency index, n represents the sampling point; j represents the imaginary part sign;

[0102] Step S2.1.7. Since there are many static objects such as walls and desks that interfere with the movement process, MTI filtering is added in step S2 to filter out the influence of static objects. By suppressing the echo signal of the fixed target, the signal of the moving target is made more significant. The core principle of MTI filtering is to use the phase difference between adjacent pulses to eliminate the influence of the stationary target. A fifth-order MTI filter is used in this algorithm. Finally, the frequency domain signal xf[n] is subjected to sliding window MTI dynamic filtering according to the following formula to obtain the amplitude spectrum P[m,n] under each standard action. fft ]:

[0103] P[m,n fft ]=|P′[m,n fft ]|

[0104] P′[m,n fft ]=XF′[m,n fft ]-4XF′[m-1,n fft ]+6XF′[m-2,n fft ]-4XF′[m

[0105] -3,n fft ]+XF′[m-4,n fft ]

[0106] Among them, P′[m,n fft ] represents the dynamic filtering result; || represents the absolute value function.

[0107] Sliding window MTI dynamic filtering is used to remove interference from static objects and retain only the moving target signal. The data after dynamic filtering is obtained by taking the absolute value to obtain an amplitude spectrum. Each standard action corresponds to an amplitude spectrum.

[0108] In step S2, the specific method of segmenting the amplitude spectrum to obtain standard action image blocks is as follows:

[0109] Step S2.2.1. First, extract the maximum pixel value in the amplitude spectrum and mark the pixels with a value greater than 45% of the maximum pixel value in the amplitude spectrum. Use a 3×3 traversal window to traverse each marked pixel. Observe whether the pixels above, below, left, right, upper left, lower left, upper right, and lower right of the marked pixel are marked within the traversal window centered on the marked pixel. If more than six pixels are marked within the traversal window, use that traversal window as the target traversal window.

[0110] Step S2.2.2: When there is an overlapping area between two target traversal windows, the union of the areas where the two target traversal windows are located is used as the new target traversal window;

[0111] Step S2.2.3: Repeat step S2.2.2 until there is no overlapping area between all target traversal windows. The smallest rectangular area containing each target traversal window is used as each connected area. That is, one rectangular connected area covers one target traversal window to obtain multiple connected areas.

[0112] Step S2.2.4: Obtain the maximum length and maximum width of all connected regions, and use them as the target length and target width, respectively. Expand the boundaries of each connected region so that the length and width of each connected region reach the target length and target width, respectively, and the connected region before boundary expansion is in the middle of the connected region after boundary expansion.

[0113] Step S2.2.5. Finally, the connected region after boundary expansion is cut in the amplitude spectrum to obtain several image blocks as standard action image blocks.

[0114] The fitness action dataset in step S2 includes the cut standard action image blocks and their corresponding labels. The labels of the fitness action dataset are the actual standard actions corresponding to the standard action image blocks, such as Figure 2 As shown, the label is one of squat, lunge, dumbbell press, dumbbell lateral raise, dumbbell front raise and dumbbell fly.

[0115] In the specific implementation of step S2, a large amount of data of 6 kinds of movements is collected from different coaches in multiple scenarios. The amplitude spectrum corresponding to each standard movement is segmented into single movements using the action unit segmentation method, and each single standard movement amplitude spectrum is manually labeled to construct a data set of standard movements:

[0116] Specifically, the multiple scenes refer to four different rooms with different furniture arrangements and fitness areas in different locations. The different coaches refer to three men and three women of varying heights and weights. The heights and weights of the three men are: 176cm / 75kg, 168cm / 60kg, and 185cm / 90kg, respectively; the heights and weights of the three women are: 158cm / 48kg, 171cm / 62kg, and 165cm / 56kg, respectively. The six exercises are squats, lunges, dumbbell presses, dumbbell lateral raises, dumbbell front raises, and dumbbell flyes. Each exercise is recorded with a 3s interval and repeated 20 times, resulting in 2880 audio recordings. The audio recordings are processed using the filtering method in step S2 to produce 2880 amplitude-spectrograms. The amplitude spectrograms are segmented using the action-by-action segmentation method, resulting in 57,600 224×224 amplitude spectrograms. These are manually annotated based on the coaches' movements, and finally, multiple standard action image blocks are segmented.

[0117] Step S3: Next, construct an action recognition model. The specific structure is as follows: Figure 2 As shown, the fitness action dataset is input into the constructed action recognition model for training to obtain the trained action recognition model. The loss curve during the model training process is shown in Figure 3 As shown;

[0118] In the specific implementation of step S3, the obtained fitness action dataset is randomly divided to determine the training dataset, validation dataset and test dataset: the ratio of the training dataset, validation dataset and test dataset is divided into: the training dataset accounts for 75%, the validation dataset accounts for 15%, and the test dataset accounts for 10%, ensuring that the data in each dataset is relatively balanced. The deep learning network is selected as DenseNet-264, and the specific structure is as follows: Figure 3 , specific parameters such as Figure 4 As shown, the DenseNet-264 model contains 1 initial convolution layer (64 7×7 convolutions with a stride of 2), 4 dense blocks (each layer in the block consists of 1×1 convolution and 3×3 convolution, and each block contains 6, 12, 64, and 48 layers respectively), 3 transition layers (1×1 convolution and 2×2 average pooling) and a classification layer. The classification layer uses global average pooling, followed by a 1000-dimensional fully connected layer and classification with a softmax activation function. The growth rate of each layer is 32. It has dense connections. Each layer in DenseNet receives the features of all previous layers as input. The traditional convolutional feedforward network uses the output of the lth layer as the input of the l+1th layer, which can be expressed as:

[0119] x l =H l (x l-1 )

[0120] In order to improve the fluidity of information, direct connections are adopted from any layer to all subsequent layers, and the lth layer receives feature maps from all previous layers as input:

[0121] x l =H l (x l-1 )+x l-1

[0122] Among them, [x1,x2,...,x l-1 ] represents the concatenation of the feature maps generated by the previous layer, H l (·) is defined as a composite function of three consecutive operations: batch normalization (BN), rectified linear unit (ReLU), and 3×3 convolution (Conv).

[0123] This dense connection enhances the transmission of information flow, making features smoother throughout the network. Gradients can be easily propagated in the network, which helps alleviate the problem of gradient disappearance. DenseNet reduces the features that need to be re-learned in each layer through the connection mechanism. The input of each layer contains all the feature maps of the previous layer, allowing the network to learn features more effectively with fewer parameters. Since each layer of DenseNet directly receives the features of all previous layers, the feature maps are reused in the network. This reuse not only increases the expressive power of the model, but also reduces the risk of overfitting of the model. DenseNet achieves a smaller model size and reduces the demand for computing resources by densely connecting each layer.

[0124] There are very narrow layers in the model DenseNet-264, that is, kp=32, and each function H l Generate kp feature maps, then the lth layer has k0+kp×(l-1) input feature maps, where k0 is the number of channels in the input layer. Each layer can access all the previous feature maps in the dense block, that is, the "collective knowledge" of the network. These feature maps are regarded as the global state of the network. Each layer only needs to add new feature maps to this state. A small growth rate is enough to achieve excellent results.

[0125] The l×1 convolution is introduced as a bottleneck layer before each 3×3 convolution to reduce the number of input feature maps, thereby improving computational efficiency. In terms of improving model compactness, the number of feature maps is reduced in the transition layer. Assuming that a dense block contains m feature maps, the subsequent transition layer generates A characteristic graph is formed, where θ (0<θ<1) is a compression factor, and θ=0.5 is selected in the present invention.

[0126] In the specific implementation of step S3, the hardware equipment in step S1 is used to build a fitness area environment. The user stands in the fitness area, facing the microphone and speaker, and performs the six fitness exercises in step S2. The mobile phone microphone is used to collect data, and the image blocks corresponding to the user's fitness exercises are obtained by comprehensive filtering and image segmentation methods. The image blocks are input as input data into the DenseNet-264 network trained with the standard data set to obtain the final classification results.

[0127] Step S4: Finally, the microphone is used to collect the echo signal of the fitness person during the fitness process. The fitness action echo signal is filtered and segmented in sequence to obtain the standard action image block. The standard action image block is then input into the trained action recognition model to output the recognition result of the fitness person's fitness action. The recognition accuracy rate is 94.19%. Its precision-recall rate curve is shown as follows: Figure 4 shown.

[0128] In summary, the example of this application first obtains the coaching exercise audio data collected by commercial speakers and mobile phone microphones, uses the designed filtering method to analyze and process the echo information, and obtains the amplitude spectrum of the collected data. Then, the amplitude spectrum is segmented by the segmentation method to obtain the amplitude spectrum image block corresponding to each action unit, which is manually labeled and randomly divided to construct a standard data set. This data set is used to train the deep learning network DenseNet-264 to obtain a trained fitness action recognition network. Finally, the user's fitness data is collected using the original commercial speakers and mobile phone microphones, and after the same processing, it is input into the trained network to realize the recognition and classification of the user's fitness exercise movements, so as to provide detailed fitness statistics and help improve fitness effects.

[0129] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the contents disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0130] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A non-contact fitness action recognition method based on near-ultrasonic signals, characterized in that: The following steps are involved: Step S1: First, a microphone and a speaker are arranged in a fitness exercise area, a near-ultrasonic chirp signal is emitted by the speaker, and an echo signal of a fitness exercise is collected by the microphone; Step S2, filtering the fitness action echo signal collected in step S1 to obtain an amplitude spectrum, then segmenting the amplitude spectrum to obtain standard action image blocks, and using the standard action image blocks to construct a fitness action data set; Step S3: Next, construct an action recognition model, input the fitness action data set into the constructed action recognition model for training, and obtain a trained action recognition model; Step S4. Finally, use a microphone to collect echo signals of the fitness person during the fitness process, filter and segment the fitness action echo signals in turn to obtain standard action image blocks, and then input the standard action image blocks into the trained action recognition model to output the recognition results of the fitness person's fitness actions.

2. The non-contact fitness action recognition method based on near-ultrasonic signals according to claim 1, characterized in that: The step S1 is specifically as follows: First, in a fitness area, a microphone and a speaker are arranged at the boundary of the fitness area, the microphone and the speaker are located at the same height, and both are facing the front of the fitness person; then, the speaker is used to transmit a customized near-ultrasonic chirp signal to the fitness person, and the microphone is used to receive the echo signal reflected by the fitness person under various standard movements when performing fitness exercises. The sending frequency of the customized near-ultrasonic chirp signal is 17kHz-20kHz, and the total signal sending time is 3 to 5 minutes. The standard movements include squats, lunges, dumbbell presses, dumbbell lateral raises, dumbbell front raises and dumbbell flyes.

3. The non-contact fitness action recognition method based on near-ultrasonic signals according to claim 1, characterized in that: In step S2, the specific method of filtering the echo signal of the fitness action to obtain the amplitude spectrum is as follows: Step S2.1.1, firstly synchronously input the echo signals collected M times under the same standard action into the bandpass filter for filtering, and obtain the echo signal y[n] with a frequency in the range of 14kHz-23kHz; Step S2.1.2: The near-ultrasonic chirp signal emitted by the loudspeaker is then used as the reference signal x[n], and the cross-correlation function R between the filtered echo signal y[n] and the reference signal is xy The expression for [k] is as follows: Where n is the number of sampling points of the signal; k is the sample delay; Step S2.1.3: Then use the peak extraction algorithm to extract the cross-correlation function R between the echo signal and the reference signal. xy [k] performs peak extraction and obtains the cross-correlation function R xy The maximum peak of [k]; Step S2.1.4: Extract the cross-correlation function R xy The echo signal y[n] after the maximum peak of [k] is used as the target signal y'[n], and the pseudo transmission signal s'[n] is defined by the reference signal. The conversion signal xs[n] is obtained by processing the target signal and the pseudo transmission signal according to the following formula: xs[n]=y[n]·s'[n] Wherein, f1 represents the initial frequency of the near-ultrasonic signal; B2 represents the bandwidth of the pseudo-transmission signal; Ts represents the duration of the pseudo-transmission signal; π represents the circumference of a circle; n represents the sampling point; Step S2.1.5: Perform a fast Fourier transform on the conversion signal xs[n], so that the conversion signal xs[n] is converted from a time domain signal to a frequency domain signal XF[n fft ], the frequency domain signal XF[n fft ] is combined according to the frame number sequence of the near-ultrasonic chirp signal to obtain the frequency domain signal matrix XF[m,n fft ], m represents the frame number of the near-ultrasound chirp signal; Step S2.1.6: The frequency domain signal matrix XF[m,n fft ] to update: Wherein, di represents the distance from the ith obstacle in the fitness area to the microphone; fi is the ith frequency value in the conversion signal; Ts is the duration of the pseudo transmission signal, B9 is the bandwidth of the pseudo transmission signal, and v is the speed of sound; g() represents an integer function; Next, the frequency domain signal matrix XF[m,n fft All elements in the di-th row in ] are replaced with 0 values, and the updated frequency domain signal matrix is ​​obtained as the target signal matrix XF′[m,n fft ]; Step S2.1.

7. Finally, the target signal matrix XF′[m,n fft ] to perform sliding window MTI dynamic filtering to obtain the amplitude spectrum P[m,n fft ]: P[m,n fft ]=|P′[m,n fft ]| P′[m,n fft ]=XF′[m,n fft ]-4XF′[m-1,n fft ]+6XF′[m-2,n fft ]-4XF′[m-3,n fft ]+XF′[m-4,n fft ] Among them, P′[m,n fft ] represents the dynamic filtering result; | | represents the absolute value function.

4. The non-contact fitness action recognition method based on near-ultrasonic signals according to claim 1, characterized in that: In step S2, the specific method of segmenting the amplitude spectrum to obtain the standard action image block is as follows: Step S2.2.1, first extract the maximum pixel value in the amplitude spectrum, and mark the pixel points that are greater than 45% of the maximum pixel value in the amplitude spectrum, and use the traversal window to traverse each marked pixel point. If there are more than six marked pixels in the traversal window, the traversal window is used as the target traversal window; Step S2.2.2, when there is an overlapping area between two target traversal windows, the union of the areas where the two target traversal windows are located is used as the new target traversal window; Step S2.2.3, repeat step S2.2.2 until there is no overlapping area between all target traversal windows, and take the minimum rectangular area containing each target traversal window as each connected area; Step S2.2.4, obtaining the maximum length and the maximum width of all connected regions, taking them as the target length and the target width respectively, and expanding the boundaries of each connected region so that the length and the width of each connected region reach the target length and the target width respectively; Step S2.2.

5. Finally, the connected region after boundary expansion is cut in the amplitude spectrum to obtain several image blocks as standard action image blocks.

5. The non-contact fitness action recognition method based on near-ultrasonic signals according to claim 1, characterized in that: The fitness action data set in step S2 includes the cut standard action image blocks and their corresponding labels. The labels of the fitness action data set are the actual standard actions corresponding to the standard action image blocks. The labels are one of squats, lunges, dumbbell presses, dumbbell lateral raises, dumbbell front raises and dumbbell flyes.

6. The non-contact fitness action recognition method based on near-ultrasonic signals according to claim 1, characterized in that: The action recognition model in step S3 uses the image classification neural network DenseNet-264.

7. The non-contact fitness action recognition method based on near-ultrasonic signals according to claim 2, characterized in that: The expression of the near-ultrasonic chirp signal x[n] is as follows: Among them, f s represents the initial frequency of the near-ultrasonic signal; Bs represents the bandwidth of the near-ultrasonic signal; T s represents the duration of the near-ultrasonic signal; π represents the circumference of a circle; and n represents the sampling point.

8. The non-contact fitness action recognition method based on near-ultrasonic signals according to claim 2, characterized in that: The expression of the echo signal y[n] received by the microphone is as follows: Among them, n l represents the attenuation coefficient of the lth propagation path; N represents the total number of signal propagation paths.

Images