Method and device for analysing a swimming style of a user on the basis of structure-borne sound

Abstract

The present invention relates to a device (100) and a method for analysing a swimming style of a user on the basis of structure-borne sound. Sensor signals are generated by one or more sensors (103) and output to a processing device (102). The sensor signals are preprocessed by the processing device (102), the preprocessing comprising applying a high-pass filter. The swimming style of the user is evaluated on the basis of the preprocessed sensor signals, and the result of the evaluation is output to the user.

Description

[0001] Description

[0002] title

[0003] Method and device for analyzing a user's swimming style based on structure-borne sound

[0004] State of the art

[0005] The present invention relates to a method for analyzing a user's swimming style based on structure-borne sound and a device for analyzing a user's swimming style based on structure-borne sound.

[0006] JP 2005-311 428 A2 describes a hydrophone that can be used to record underwater sounds. The hydrophone is attached to a cable, allowing it to be lowered into the water. Detected underwater events can originate, in particular, from swimming.

[0007] US Patent 3,273,138 discloses a device for monitoring a swimming pool. The device includes a hydrophone for recording underwater sounds, such as those generated when a child or animal falls into the pool. In this case, the pool operator is informed or alerted. The pool operator also has the option of listening to the hydrophone signal to determine the source of the underwater sound.

[0008] US Patent 5,012,457 describes a system for generating high-fidelity audio signals underwater. The system can be installed in the side wall of a swimming pool. US Patent 5,031,637 discloses a system for recording sounds produced by the human body. For this purpose, the human body is immersed in a fluid used for sound transmission. The sounds produced by the human body are recorded by underwater microphones. The recorded sounds can, for example, provide clues to certain diseases.

[0009] German patent DE 10 2014 202 638 A1 relates to a device and a method for monitoring swimming performance. The device has both inertial and magnetic field sensors for detecting the user's swimming movements. The signals are acquired at a sampling rate of 10 Hz. The device is worn on the user's wrist.

[0010] US Patent 2012 / 072 165 A1 describes a device for monitoring a user's swimming activity. The device can be attached to various parts of the body. It has a sensor that can detect movement. A "hidden Markov model" is used to recognize the swimming style.

[0011] US patent 2018 / 065 022 A1 discloses a device for determining a user's swimming style. It can also determine other information, such as energy consumption, speed, and the number of turns. This information is captured using an accelerometer that records body movement.

[0012] Disclosure of the invention

[0013] The problem addressed by the invention is solved by a method for analyzing a user's swimming style based on structure-borne sound according to claim 1. Furthermore, according to a second aspect of the invention, a device for analyzing a user's swimming style based on structure-borne sound according to claim 8 is provided. Preferred embodiments of the invention are the subject of the dependent claims, the drawings, and the description of exemplary embodiments.

[0014] The method for analyzing a user's swimming style includes a step to generate sensor signals using one or more sensors and output these signals to a processing unit. Several different types of sensors can be used, in particular pressure sensors, accelerometers, gyroscopes, and acoustic sensors. This allows for the reliable detection of the user's movement in multiple dimensions. Furthermore, structure-borne sound can be detected, from which a variety of movement characteristics can be determined.

[0015] A further step involves preprocessing the sensor signals using the processing unit. This preprocessing includes at least the application of a high-pass filter. Preprocessing allows the sensor signals to be processed more easily, quickly, and therefore more efficiently. In particular, preprocessing serves to remove unwanted components of the sensor signals.

[0016] A further step involves evaluating the user's swimming style based on the pre-processed sensor signals and outputting the results of the evaluation to the user. This allows the user to draw conclusions about the quality of their movements and improve their swimming style.

[0017] According to preferred embodiments, the preprocessing further comprises the application of a low-pass filter and / or the application of a band-pass filter and / or the removal of interference signals. By applying frequency filters, the components of the signal relevant for evaluation are filtered out. This improves the efficiency of the subsequent processing.

[0018] Preferably, the high-pass filter can have a cutoff frequency of 20 Hz. Tests have shown that low-frequency components of the sensor signals largely contain unwanted noise. Evaluating the swimming style preferably provides at least one of the following pieces of information or results: the type of swimming style used by the user; and / or the period of the user's arm movements; and / or the user's leg kick frequency; and / or the user's arm immersion depth; and / or the user's breathing rate; and / or the user's heart rate; and / or the quality of the user's swimming style.

[0019] Exemplary methods for determining this information are explained further below in the description of the implementation examples.

[0020] According to a preferred embodiment, the evaluation of the swimming style comprises a step to determine a moving average of the amplitude of the preprocessed sensor signals and a step to determine a period by calculating an average of the time intervals between maxima of the moving average. Calculating the moving average enables an efficient evaluation of the amplitude of the generated sensor signal.

[0021] Preferably, the evaluation of swimming style includes a step involving the application of a neural network trained to recognize swimming styles based on pre-processed sensor signals. A neural network can distinguish between different swimming styles particularly efficiently and quickly. Previously recorded sensor data, along with information about the swimming styles used, can be used as training data.

[0022] Preferably, the evaluation of the swimming style can include a step to determine the time intervals during which a user's hand is above the water's surface, and a step to calculate the period of the arm movement based on these time intervals. The swimming style used can be determined by evaluating the hand movement. Furthermore, the quality of the movement can be assessed, for example, based on the timing and duration of the hand(s) entering and emerging from the water.

[0023] The device according to the invention for analyzing a swimming style based on structure-borne sound comprises a housing, a processing unit arranged in the housing, and a sensor arranged in the housing. The processing unit is configured to perform a method according to the invention for analyzing a user's swimming style based on structure-borne sound.

[0024] According to preferred embodiments, the processing device and the sensor can each be designed as separate units or integrated into a common electronic component.

[0025] The device may preferably comprise a wearable and / or a hearable. The device may therefore consist of a single unit or of a unit with additional components. For example, a smartwatch on the user's wrist may serve as the host device, with additional sensors arranged in hearables worn in the user's ears. This allows, for example, the position and orientation of the head to be detected.

[0026] The case is advantageously waterproof for use while swimming in water.

[0027] According to a preferred embodiment, the device may include a plurality of sensors and / or a display device for displaying information and / or a storage device for storing sensor signals and / or an interface for transmitting sensor signals and / or evaluation results to a user's terminal device.

[0028] The present invention is explained in more detail below with reference to the exemplary embodiments shown in the schematic figures. Figure 1 shows a schematic representation of an exemplary device for analyzing a swimming style based on structure-borne sound;

[0029] Fig. 2 shows a schematic representation of an exemplary sequence of a method according to the invention for analyzing a swimming style based on structure-borne sound;

[0030] Fig. 3 shows an exemplary high-pass filtered signal from one axis of an accelerometer in a device for analyzing a swimming style based on structure-borne sound during breaststroke;

[0031] Fig. 4 shows a moving average of the high-pass filtered signal from Fig. 3;

[0032] Fig. 5 shows another exemplary high-pass filtered signal from one axis of an accelerometer in a device for analyzing a swimming style based on structure-borne sound during crawl stroke;

[0033] Fig. 6 shows another exemplary high-pass filtered signal from one axis of an accelerometer in a device for analyzing a swimming style based on structure-borne sound with a magnification of a sub-area;

[0034] Fig. 7 shows another exemplary high-pass filtered signal from one axis of an accelerometer in a device for analyzing a swimming style based on structure-borne sound, with a magnification of a sub-area; and

[0035] Fig. 8 shows another exemplary high-pass filtered signal from one axis of an accelerometer in a device for analyzing a swimming style based on structure-borne sound during backstroke swimming.

[0036] The accompanying figures are intended to provide a further understanding of the embodiments of the invention. They illustrate embodiments and, in conjunction with the description, serve to explain the principles and concepts of the invention. Other embodiments and many of the advantages mentioned will become apparent with reference to the drawings. The elements of the drawings are not necessarily shown to scale.

[0037] In the figures of the drawing, identical, functionally equivalent and similarly acting elements, features and components - unless otherwise stated - are each provided with the same reference symbols.

[0038] First, an exemplary setup of a device 100 according to the invention for analyzing swimming styles based on structure-borne sound is described. Subsequently, an area of ​​application is explained in detail, and signal characteristics are named on the basis of which the described application can be realized.

[0039] Fig. 1 shows an exemplary setup of a device 100 according to the invention for analyzing swimming styles based on structure-borne sound. The device comprises a housing 101 in which a processing unit 102 and at least one sensor 103 are arranged. Fig. 1 shows, by way of example, a sensor 103, which here is implemented as part of the processing unit 102. However, the sensor 103 and the processing unit 102 can also be separate components. Furthermore, several sensors 103 can be arranged in the housing 101, for example, to enable sensor fusion.

[0040] The device 100 can, for example, be a so-called "wearable" or "hearable." A wearable is preferably a smartwatch or the like, worn on the user's wrist. More preferably, the user can wear a wearable on each wrist or arm. A wearable can also be worn on a finger, such as a smart ring. In preferred embodiments, a wearable can also be implemented as smart swimming goggles or a chest strap. A hearable is, for example, headphones or a pair of headphones that can be worn directly in or on the user's ear. If the wearable is preferably a smartwatch, then according to the invention, the user can preferably wear at least one wearable or hearable, one wearable and one hearable, or two wearables and / or two hearables.

[0041] Particularly preferably, when using multiple wearable and / or hearable devices, wireless connections can be used to communicate and / or transmit sensor data between the different devices.

[0042] For recording structure-borne sound, the sensors 103 are preferably arranged directly on an inner surface of the housing 101. If the device 100 is a wearable, the sensors 103 can preferably be arranged on the back of a display.

[0043] The sensor 103 can preferably be an acoustic sensor, an accelerometer, a gyroscope, a pressure sensor, or a combination of two or more such sensors. A pressure sensor can, for example, determine the immersion depth of the user's hands. An acoustic sensor can, in particular, detect structure-borne sound as well as background noise caused by air bubbles generated in the water. Furthermore, an acoustic sensor can detect whether the sensor is above or below water.

[0044] The sensor 103, or the plurality of sensors 103 of the device 100, generate sensor signals and output them to the processing unit 102. The processing unit 102 preferably includes a memory for storing received sensor signals and evaluated data before and after processing.

[0045] Furthermore, for signal evaluation, it is advantageous if the sensor signals are acquired with a high sampling rate and bandwidth. The signals shown in Figures 3 to 8, for example, were recorded with an accelerometer with a sampling rate of 8 kHz and a bandwidth of approximately 2.4 kHz.

[0046] Fig. 2 first illustrates a schematic sequence of a method according to the invention for evaluating a user's swimming style of the device 100. The illustrated process sequence consists of three steps S1 to S3. Before the first step S1 and after the last step S3, the start and end of the process sequence are shown, respectively.

[0047] In the first step S1 of the process, the sensor signal or the multitude of sensor signals are read. The sensor 103, for example, can be either a single-axis or multi-axis sensor. Furthermore, it can be one or more sensors. In particular, the same or different types of sensors can be installed at different locations within the housing 101. The device 100 can also consist of several different devices that are combined with one another. For example, a smartwatch can serve as a wearable host device, which has its own sensors 103, the data from which is used by the processing unit 102 to evaluate the swimming style.

[0048] Additionally, one or two hearables can be connected to the wearable, especially via a wireless connection such as Bluetooth or similar.

[0049] Accordingly, the process sequence shown in Fig. 2 can also be carried out on the hearables, or the sensor signals from the hearable can be transmitted to the host device and processed there. In the second case, the process sequence shown in Fig. 2 is only executed on the host device.

[0050] In the second step S2 of the process flow shown in Fig. 2, the sensor signal(s) are preprocessed. This preprocessing can involve various tasks. For example, the sensor signal can be filtered with a simple high-pass, low-pass, or band-pass filter to isolate the frequency ranges required for evaluating the swimming style.

[0051] The preprocessing includes at least the application of a high-pass filter to filter out the signal components resulting from the arm movement itself. In Figures 3 to 8, for example, the cutoff frequency of the high-pass filter was set to a value of 20 Hz. However, other cutoff frequencies can also be set.

[0052] According to preferred embodiments, the preprocessing can further include the removal of interference signals from the sensor signal. When the device 100 is used in a swimming pool in which other swimmers are present and moving, the sensors 103 can record structure-borne sound from the other swimmers. Since this structure-borne sound from the other swimmers can negatively affect the evaluation of the user's swimming style, it is preferably removed from the sensor signals in S2. This can be achieved, for example, by applying spectral masks with a predefined frequency range. Furthermore, suitable filter banks can be used to filter out relevant frequency ranges.

[0053] Preprocessing in step S2 reduces the frequency range of the sensor signals, which can advantageously suppress interference. Furthermore, reducing the frequency range allows for effective compression of the sensor signals, thus requiring less storage space and less RAM, and making subsequent processing less complex.

[0054] The third step, S3, involves the actual evaluation of the user's swimming style. The following information can be recorded during the evaluation of the swimming style:

[0055] • Swimming style: The sensor signal can be used to determine whether the user is swimming breaststroke, backstroke, or crawl, for example. • Arm movement period: The recorded sensor signal has a period that can be used to determine the arm movement period.

[0056] • Leg kick frequency o When the wearable or hearable is underwater, the leg kick frequency can be determined by periodic signal components.

[0057] • Arm immersion depth: One aspect that can be used to evaluate swimming style is the fact that air bubbles rise after the hands enter the water and burst at the surface. When the arm is moved close to the surface, air bubbles are formed throughout the movement. The deeper the arm is moved below the surface, the fewer air bubbles remain. Therefore, a rough assessment of the arm's depth can be made based on the air bubbles and the water turbulence.

[0058] • Breathing rate o Especially when using a hearable, it's possible to detect when a swimmer, for example, moves their head to the side while swimming freestyle, as the hearable's entry into the water can be detected. o In addition to breathing rate based on head movement, breathing rate can also be determined based on bone conduction. When the user breathes, the sound of breathing is transmitted directly to the hearable via bone conduction.

[0059] • Heart rate: A user's heart rate can be derived from periodic signals in the sensor signals. In particular, sensor data that has not been pre-processed with a high-pass filter can be used for this purpose, especially to detect frequencies in the range of 50 to 220 beats per minute.

[0060] • Quality of the executed swimming style: The quality of the executed swimming style can be evaluated based on the following characteristics, for example. A desired goal in freestyle is for the hand to enter the water without forming air bubbles or turbulence. The type of noise recorded in the sensor signals due to air bubbles and water turbulence therefore allows conclusions to be drawn about the quality of the executed swimming style. Furthermore, body movement can be evaluated. In freestyle, for example, one arm should enter the water before the other arm leaves the water. This timing can be recorded from the sensor signals. With the additional use of hearables, it can also be recorded whether the user's head is turned to the side at the desired times during freestyle or lifted out of the water to breathe at the desired times during breaststroke.

[0061] Figure 3 shows an example of a high-pass filtered signal from one axis of an accelerometer. The abscissa represents time in seconds, and the ordinate represents the value of the high-pass filtered signal in arbitrary units. The high-pass filtered signal indicates the breaststroke swimming style. This was recognized, for example, by the fact that the hands were always below the water's surface while performing this swimming style.

[0062] The periodic arm movements during breaststroke are clearly visible in the periodic signal components of the high-pass filtered signal. Furthermore, the high-pass filtered signal in Fig. 3 shows significant increases in signal amplitude, even though the hands were always moved below the water's surface and no or very few air bubbles were generated. This demonstrates that even turbulence in the water itself, generated by the hands, arms, or the wearable device's casing, can be detected as structure-borne noise.

[0063] The period of arm movements during breaststroke can be evaluated in various ways. For example, the amplitude of the high-pass filtered signal can be analyzed. In particular, a moving average of the amplitude of the high-pass filtered signal can be determined. Alternatively, other calculation methods, such as an exponential smoother or another type of low-pass filter, can be used.

[0064] The moving average with a length of, for example, 2000 samples for the high-pass filtered signal shown in Fig. 3 is shown in Fig. 4. In Fig. 4, the period of the arm movement is clearly visible and can be determined, for example, by measuring the intervals between the signal peaks. A two-stage method can be used to determine the period. In this two-stage method, the swimming style itself is first identified, such as breaststroke in this case. In the next step, the period of the arm movements is calculated.

[0065] In addition to evaluating the high-pass filtered signal in the time domain, it is also conceivable to perform a frequency analysis of the high-pass filtered signal to determine the period of the arm movements.

[0066] Figure 5 shows an example of a high-pass filtered signal from one axis of an accelerometer. The abscissa represents time in seconds, and the ordinate represents the value of the high-pass filtered signal in arbitrary units. Here, the high-pass filtered signal represents the crawl stroke. As can be seen in Figure 5, the high-pass filtered signal differs significantly from the high-pass filtered signal shown in Figure 3. Different swimming strokes can therefore be distinguished based on the signal characteristics. The signal characteristics can be evaluated, for example, using classical algorithms. Alternatively, neural networks can be trained to distinguish between different swimming strokes.Training data can be provided, for example, by a large number of recorded sensor data from a large number of swimmers, where the swimming style used is known in each case.

[0067] Unlike the breaststroke, in the crawl stroke the hands are lifted out of the water and remain above the surface for a specific period of time. As shown in Fig. 5, the sections marked "A", during which the hand is above the water, are clearly visible. The period of the arm movement can be determined from the time interval between these sections.

[0068] Furthermore, it is possible to record how long the hand is moved above and below the water's surface. In addition to the duration itself, variations in that duration can also be recorded. Tracking these variations allows for inferences about the consistency of the swimming motion and, along with other aspects, can be used to evaluate the quality of the swimming style.

[0069] The information described in the previous paragraphs can be provided to the user in a detailed analysis of their swimming style, for example, via a corresponding application on the wearable or a smartphone paired with it, or via a website accessible via the internet, or the like. For this purpose, the data can be transferred from the device 100 to the user's internet-enabled device and subsequently to a cloud or server. The quality of the executed swimming style can also be logged, allowing the user to review their own training progress. A preferred device 100 includes, in particular, a display unit for showing the analysis. The quality of the executed swimming style can also be evaluated using other characteristics. One possible characteristic is described with reference to Figures 6 and 7.Figures 7 and 7 each show a high-pass filtered signal from one axis of an accelerometer. Both figures depict the high-pass filtered signal of the crawl stroke. The quality of the executed stroke differs significantly between Figures 6 and 7. In Figure 6, the hand was entered the water at an angle, resulting in minimal air bubbles and water turbulence. In contrast, in Figure 7, the palm of the hand was entered the water to generate a large number of air bubbles and water turbulence.

[0070] As in the comparison of the magnified high-pass filtered signals in Fig. 6 and Fig. 7, the signal amplitude after the hand enters the water is greater in Fig. 7 than in Fig. 6, which is due to the increased number of air bubbles and water turbulence. Furthermore, the point in time at which the wearable was immersed in the water is more clearly visible in the magnified image of Fig. 6 compared to Fig. 7, as indicated by the signal peak. The occurrence of such a signal peak can also be used to assess the quality of the swimming technique.

[0071] As can be seen in Figures 6 and 7, different ways of entering the water with the hands can be distinguished. It is therefore also possible to differentiate how the hand is shaped at the moment of entry, for example, whether the user has made a fist, spread their fingers, or enters the water loosely without tension. Depending on how the user enters the water, the water displacement can also be assessed as a quality characteristic of the swimming style. The characteristics mentioned above are merely examples of features that can be determined.

[0072] If the user wears two wearables, it is also possible to determine the synchronicity of arm movements. Synchronicity can be assessed for different swimming styles and is also dependent on the specific stroke being performed. For example, in breaststroke, the arms should ideally move in phase, whereas in freestyle or backstroke, the different phases of the arm movement are staggered. Therefore, to assess synchronicity, it is advantageous to base the evaluation on the underlying swimming style.

[0073] Figure 8 shows a high-pass filtered signal from one axis of an accelerometer. The abscissa represents time in seconds, and the ordinate represents the value of the high-pass filtered signal in arbitrary units.

[0074] The high-pass filtered signal in Fig. 8 shows the backstroke swimming style. The backstroke swimming style can also be distinguished from the other swimming styles presented in this invention disclosure based on the signal waveform.

[0075] As with the crawl stroke, the hand is lifted out of the water and then re-entered during the backstroke. The sections marked "A", where the hand is above the water's surface, are clearly visible in the signal and are also highlighted in Fig. 8.

[0076] Figures 3 to 8 have shown that swimming styles can be distinguished from one another. Furthermore, the inventors observed that the same swimming style differs between different users based on the generated sensor signals. For example, the user can also be identified based on the analysis of their swimming style.

[0077] In addition to automated analysis and evaluation of swimming style, the sensor signals can also be recorded, according to a preferred configuration, for later listening and analysis. Users who practice listening to the recorded signals can then, for example, conduct their own assessment of swimming style.

[0078] For example, a swimming coach can listen to the recorded acoustic signals to analyze an athlete's swimming style and derive training plans from it.

[0079] In the present invention, several features are designated as "first" and "second". These designations serve only to clearly distinguish the individual features. In particular, no spatial or functional arrangement or prioritization should be derived from them.

[0080] If the present application contains a list of alternatives marked with the term “or”, it should be understood that the listed alternatives should be understood to be taken individually, but also, where appropriate, as a combination of several or all of the listed alternatives.

Claims

Claims 1. Method for analyzing a user's swimming style based on structure-borne sound, comprising the following steps: Generating sensor signals (S1) by one or more sensors (103) and outputting the generated sensor signals to a processing unit (102); Performing preprocessing (S2) of the sensor signals by the processing device (102), wherein the preprocessing includes applying a high-pass filter; Evaluating the user's swimming style (S3) based on the pre-processed sensor signals and outputting the result of the evaluation to the user.

2. The method of claim 1, wherein the preprocessing (S2) further comprises: Applying a low-pass filter; and / or Applying a bandpass filter; and / or removing interference signals.

3. Method according to claim 1 or 2, wherein the high-pass filter has a cutoff frequency of 20 Hz.

4. A method according to any of the preceding claims, wherein the evaluation of the swimming style (S3) comprises: a type of swimming style of the user; and / or a period of the arm movements of the user; and / or a leg kick frequency of the user; and / or an immersion depth of the arms of the user; and / or a breathing rate of the user; and / or a user's heart rate; and / or the quality of the user's swimming style.

5. Method according to claim 4, wherein the evaluation of the swimming style (S3) comprises: Determining a moving average of the amplitude of the preprocessed sensor signals; and Determining a period by calculating the mean of the time intervals between maxima of the moving average.

6. Method according to any of the preceding claims, comprising evaluating the swimming style (S3): Applying a neural network to recognize swimming styles and / or to determine the quality of swimming styles, wherein the neural network was trained using the preprocessed sensor signals.

7. Method according to claim 6, wherein the evaluation of the swimming style (S3) comprises: Determine the time periods during which a user's hand is above the water's surface; and Calculating one period of arm movement based on the determined time intervals.

8. Device (100) for analyzing a user's swimming style based on structure-borne sound, comprising: a housing (101); a processing device (102) arranged in the housing (101); and a sensor (103) arranged in the housing (101), wherein the processing device (102) is configured to perform a method according to any one of claims 1 to 7.

9. Device (100) according to claim 8, wherein: The device (100) comprises a wearable and / or a hearable; and / or the housing (101) is waterproof.

10. Device (100) according to claim 8 or 9, further comprising: a plurality of sensors (103); and / or a display device for displaying information; and / or a storage device for storing sensor signals; and / or an interface for transmitting sensor signals and / or evaluation results to a user terminal device.

Images