A method of detecting and identifying muscle activations made by a user, corresponding system and computer program product

Abstract

A method for detecting and identifying muscle activations carried out by a user (U) is described. Electromyography sensors provided in a wearable device (B) provide a first data stream (ref) indicative of electromyography signals sampled during a reference session in which the user (U) repeatedly performs a muscle activation. A non-negative matrix factorization algorithm is applied to the first data stream to compute a set of reference matrices indicative of muscle synergies detected during repetitions of the muscle activation. The average value of the reference matrices is computed to define an average reference matrix. The electromyography sensors provide a second data stream (cfr) indicative of the electromyography signals sampled during a comparison session in which the user (U) repeatedly performs the muscle activation. The non-negative matrix factorization algorithm is applied to the second data stream to compute a set of comparison matrices indicative of the muscle synergies detected during repetitions of the muscle activation. A single-repetition electromyography similarity index indicative of the difference between the comparison matrix and the average reference matrix is computed for each of the comparison matrices. An overall electromyography similarity index is computed as the average value of the electromyography similarity indices of the individual repetitions.

Description

[0001] “A method of detecting and identifying muscle activations made by a user, corresponding system and computer program product”

[0002] ****

[0003] TEXT OF THE DESCRIPTION

[0004] Technical field

[0005] The instant description relates to methods and systems for detecting and identifying (e.g., classifying) the muscle activities (or activations) of a user by analysis of electromyography (EMG) signals. Such muscle activations can possibly result in the execution of movements or gestures by the user, and in such a case the muscle activations can be detected and identified by analyzing, in addition, motion signals (e.g., acceleration and / or angular velocity). Electromyography signals and, optionally, motion signals are detected using a user-wearable device.

[0006] Technological background

[0007] Methods and systems of the type indicated above are known in the art. In the instant description, reference will be made to some documents by indicating them with a number in sguare brackets (“[X]”) that identifies the document in a LIST OF CITED DOCUMENTS that appears at the end of the description.

[0008] In particular, EMG signal segmentation technigues and the use of non-negative matrix factorization of the EMG signal for the detection of muscle activations have been studied in the literature. In this regard, document

[0001] reports the statistical demonstration of the minimum time threshold technigue for peak detection of a voluntary muscle activation. Document [2] presents a “Hierarchical Alternating Least Sguare” algorithm for carrying out non-negative matrix factorization (NNMF). Documents [3] and [4] describe the relationship between matrix decompositions of EMG signals and synergy activations from the central nervous system, and study the types of movements expected by upper limb muscles. Document [5] analyzes the relevance of muscle synergy analysis, specifically in the upper limb rehabilitation treatment of post-stroke patients. Changes over time in synergies analyzed by NMF are indicative of treatment progress.

[0009] Such known technigues have found application in some known systems or apparatuses.

[0010] For example, document [6] describes a system that includes a plurality of neuromuscular sensors that can be placed on a user’s body (e.g., wearables), such as electromyography (EMG) sensors, mechanomyography (MMG) sensors, and sonomyography (SMG) sensors. The system is configured to record neuromuscular signals detected by the neuromuscular sensors, and apply a source separation technique (e.g., independent component analysis (ICA) or non-negative matrix factorization (NNMF)) to the detected neuromuscular signals to obtain the source neuromuscular signals and corresponding mixing information (e.g., a mixing matrix or a non-mixing matrix). The system is further configured to identify, for each of the source neuromuscular signals, an associated set of one or more biological structures (e.g., one or more muscles, one or more tendons, and / or one or more motor units) whose neuromuscular activity gave rise to the source neuromuscular signal. The identification phase can be carried out using one or more features derived from the mixing information, source neuromuscular signals, and / or recorded neuromuscular signals. Such a system is hardwired (thus not portable) and does not allow identification of movements or gestures performed by the user.

[0011] Document [7] describes a system for carrying out identification of gestures made by a user. A wearable electromyography (EMG) device includes multiple EMG sensors, a processor, and a memory. The wearable EMG device detects signals when a user performs a gesture and computes a signal vector based on the characteristics of the detected signals. A library of gesture “model” vectors is stored in the memory of the wearable EMG device. A property of each angle formed between the signal vector and the model vectors is analyzed to match the direction of the signal vector to the direction of a particular model vector. Optionally, the system can also include an inertial sensor, and gesture identification can also be based on analysis of the signal detected by the inertial sensor. The system requires a dataset of pre-recorded reference movements, and thus for each new movement, it is necessary to expand the dataset by making signal acquisitions over even a very long period (e.g., weeks).

[0012] As a further example, document [8] describes a system structurally similar to that of document [7] cited earlier, but in which the similarity analysis between gestures is carried out using a decision-tree “machine learning” network. The result of the decision tree analysis is a probability vector that assigns a respective probability score to each gesture in a library of gestures. Also in this case, the system requires a dataset of pre-recorded reference movements, and thus for each new movement it is necessary to expand the dataset by performing signal acquisitions over even a very long period (e.g., weeks). In addition, each new movement also requires retraining the decision-tree network.

[0013] Document [9] describes an apparatus configured to receive EMG information generated by an individual, identify a class of gestures based on the EMG information, and train using the EMG information received and the class of gestures. The apparatus includes a virtual reality visor that provides instructions to the individual regarding the movements to be carried out. Again, the system uses a classification network with associated dataset, providing a solution that is not easily adaptable over time but rather requires readjusting the technology as the patient progresses.

[0014] Systems and methods for detecting and identifying muscle activations and / or movements can find application in a variety of different areas. For example, they may be useful for performing analysis of movements made by an operator performing repetitive manual tasks, for use in ergonomics studies (think of an assembly line worker or an office worker). Another possible application of such systems is in the field of sports, where the analysis of movements made by an athlete can be used to assess their effectiveness, and / or to provide useful indications for learning and improving a particular technical gesture, and / or to check the health condition of the athlete.

[0015] A further possible application of such systems and methods is in the field of motor rehabilitation of subjects affected by any form of motor dysfunction and / or damage to the neuro-motor system. In particular, these include individuals affected by neuromuscular diseases (such as muscular dystrophy, Parkinson’s disease, multiple sclerosis), traumatic accidents, and strokes. These people usually require a specific rehabilitation program in order to regain their motor function, but for many of them the need to repeatedly and periodically go to a rehabilitation center to get help from a physical therapist is a problem (for example, for elderly individuals for whom this commitment can become very costly in terms of time and / or money). Against this backdrop, the recent pandemic of COVID-19 has further limited the options for direct access to some nonessential care facilities, including rehabilitation facilities.

[0016] Known systems, however, do not allow monitoring of exercise in the absence of actual movement. Moreover, in known systems, comparison with a reference exercise requires preparing a large reference dataset for each movement of interest. It also follows that each new movement requires a new dataset and a new training step for the recognition algorithms.

[0017] Thus, there is a need to provide innovative solutions that can empower individuals in need of rehabilitative motor care to recover independently, including using remote technologies, and filling the gaps in known devices.

[0018] Object and summary

[0019] In view of the above, an object of the present invention is of providing methods and systems for evaluating the goodness of performance of a physical exercise (such as a rehabilitative physiotherapy movement), for example by providing an indication of the accuracy of a movement (or even just a muscle activation) performed by a person in comparison with a reference movement (or muscle activation).

[0020] According to one or more embodiments, such an object may be achieved by a method having the features set forth in the claims that follow.

[0021] One or more embodiments may relate to a corresponding system.

[0022] One or more embodiments may relate to a corresponding computer program product loadable into the memory of at least one processing circuit (e.g., a computer) and comprising portions of software code for carrying out the steps of the method when the product is executed on at least one processing circuit. As used herein, a reference to such a computer program product is intended to be equivalent to a reference to a computer-readable medium that contains instructions for controlling the processing system to coordinate implementation of the method according to one or more embodiments. A reference to “at least one processing circuit” is intended to highlight the possibility of one or more embodiments being implemented in modular and / or distributed form. The claims are an integral part of the technical teaching provided here in relation to embodiments.

[0023] According to one aspect of the present description, a method for detecting and identifying muscle activations made by a user includes receiving, from a plurality of surface electromyography sensors provided in a wearable electronic device and configured to sense a plurality of surface electromyography signals on the skin of the user, a first stream of data indicative of the plurality of surface electromyography signals sampled during a first session of use (e.g., a reference session) in which the user repeatedly makes a given muscle activation. A non-negative matrix factorization algorithm is applied to the first stream of data indicative of the plurality of surface electromyography signals to compute a set of reference matrices. Each of the reference matrices is indicative of the muscle synergies detected during a respective repetition of the muscle activation made by the user during the first session of use. The average value of the reference matrices is computed to define an average reference matrix of the muscle synergies associated to the given muscle activation. The method also includes receiving, from the plurality of surface electromyography sensors, a second stream of data indicative of the plurality of surface electromyography signals sampled during a second session of use (e.g., a comparison session) in which the user repeatedly makes the given muscle activation. The non-negative matrix factorization algorithm is applied to the second stream of data indicative of the plurality of surface electromyography signals to compute a set of comparison matrices. Each of the comparison matrices is indicative of the muscle synergies detected during a respective repetition of the muscle activation made by the user during the second session of use. A single-repetition electromyography similarity index indicative of the difference between the comparison matrix and the average reference matrix is computed for each of the comparison matrices. An overall electromyography similarity index between the first session of use and the second session of use is computed as an average value of the single-repetition electromyography similarity indices. The single-repetition electromyography similarity indices and / or the overall electromyography similarity index is transmitted to a user terminal of the user.

[0024] Thus, one or more embodiments provide a method that allows identifying a muscle activation (possibly resulting in the execution of a movement or gesture) made by a user and comparing it with a reference muscle activation using an algorithm that is precise and light in terms of computational resources expended, giving the user the possibility to carry out motor rehabilitation sessions autonomously.

[0025] According to another aspect of this description, an electronic system comprises an electronic device wearable by a user and a user terminal equipped with a processing unit. The electronic device includes a plurality of surface electromyography sensors configured to detect a plurality of surface electromyography signals on the skin of the user. The wearable electronic device and the user terminal are configured to operate according to the method of one or more embodiments.

[0026] Brief description of the figures

[0027] Various embodiments will now be described, purely by way of example, by referring to the annexed figures, in which:

[0028] - Figure 1 is a schematic of a system configured to detect and identify muscle activations made by a user, according to one or more embodiments;

[0029] - Figure 2 is a flowchart that illustrates a method for using the system of Figure 1 ; and

[0030] - Figure 3 is a flowchart that illustrates a method for operating the system of Figure 1 .

[0031] Detailed description

[0032] The following description illustrates one or more specific details, aimed at providing a thorough understanding of examples of embodiments of this description. Embodiments can be obtained without one or more of the specific details, or other processes, components, materials, etc. In other cases, known structures, materials, or operations are not illustrated or described in such detail that certain aspects of the embodiments are not made unclear.

[0033] Reference to “one embodiment” in the context of this description is intended to indicate that a particular configuration, structure, or feature described in relation to the embodiment is included in at least one embodiment. Therefore, phrases such as “in one embodiment” that may occur in one or more places in this description do not necessarily refer to the single and same embodiment. Also, particular configurations, structures, or features may be combined in any appropriate way in one or more embodiments.

[0034] In all figures annexed herein, unless the context indicates otherwise, similar parts or elements are indicated by similar references / numbers and a corresponding description will not be repeated for brevity.

[0035] The references used here are provided merely for convenience and therefore do not define the extent of protection or the scope of embodiments.

[0036] In brief, Figure 1 is a schematic that illustrates a system 10 configured to detect and identify (e.g., classify) a muscle activation (such as a rehabilitative physiotherapy exercise) made by a person II. The muscular activation may result in a gesture or movement of the person (e.g., wrist flexion) but may also result in no net movement of the person’s limbs (e.g., in the case of grasping an object, where the person clasps his or her hand around an object without resulting in a macroscopically appreciable movement). Figure 2 is a block diagram illustrating a method 20 for using the system 10, and Figure 3 is a block diagram illustrating a method 30 implemented by system 10 (e.g., a data sampling and analysis method). As discussed earlier, the system and the method can be used to evaluate the goodness of execution of a physical rehabilitation exercise, in support of patients (users) and therapists (experts), in order to verify the effectiveness of rehabilitation sessions conducted remotely. Although this description primarily refers to this application area, various applications of the process and system (such as use for ergonomic studies or sports performance evaluation) are intended to be included in the scope of the present invention. For example, one or more embodiments also find application for prevention purposes, insofar as the analysis of data on the goodness of execution of a movement may allow to determine if a subject (an athlete, worker, etc.) is close to a potential injury (e.g., an injury to a ligament such as the cruciate ligament), or if he or she is overexerting a limb (e.g., an arm, resulting in the risk of developing epicondylitis), and other similar applications.

[0037] As shown in Figure 1 , the system 10 includes:

[0038] - an electronic device B (e.g., a wristband) wearable by a user II and equipped with surface electromyography sensors and, optionally, inertial sensors (e.g., accelerometer and / or gyroscope and / or magnetometer);

[0039] - a firmware provided on device B;

[0040] - a software that includes a front-end part (e.g., executable on a terminal S of user II, such as a smart phone, tablet, personal computer, or similar) and a back-end part (e.g., executable on a remote computing system C or in the “cloud”, or executable on the same terminal S); and

[0041] - optionally, a feedback device F for user II (e.g., a tablet of user II), which may coincide with terminal S in some embodiments.

[0042] Surface electromyography (sEMG) implemented using the electromyography sensors provided in device B is a noninvasive technique that uses conductive and passive electrodes applied to the skin to detect the electrophysiological signals associated with the user’s muscle contractions. An inertial measurement unit (IMU), optionally implemented in device B, is an electronic device that measures the acceleration, angular velocity, and / or orientation of the body with which it is associated, whether a physical object or a limb, using a combination of accelerometers, gyroscopes, and / or magnetometers. In some embodiments, device B includes an 8-channel sEMG acquisition device with integrated IMU, which is used to record electromyography and kinematic data of the motion made by the limb to which the device B is attached (e.g., an arm, leg, etc.). In some embodiments, the sEMG and / or IMU sensors are configured to acquire data at a certain (high) frequency; the firmware running on device B may carry out filtering of the raw data collected by the sensors and compute the root-mean-square (RMS) value thereof, thus reducing the frequency. For example, the sampling frequency of the sEMG and / or IMU sensors can be about 2 kHz, and filtering can be done at a frequency at least ten times lower, e.g., 16 Hz (other values are possible, e.g., 32 Hz, 64 Hz, etc.). Device B communicates, via a communication channel that is preferably wireless (e.g., Bluetooth), with terminal S to transmit sampled (e.g., at a frequency of 2 kHz) and possibly filtered (e.g., at a frequency of 16 Hz) sEMG and / or IMU data streams to it. In turn, terminal S communicates (e.g., via an Internet connection) with a remote processing platform C (or cloud) that processes the data to detect the execution of a given muscle activation (e.g., a given movement) and compute a similarity index with respect to a reference muscle activation. It will be noted, however, that in some embodiments the remote processing platform C may be absent, as data processing may be carried out locally on terminal S. For the sake of brevity, in the remainder of this description reference will be made primarily to the execution of “movements” or “gestures” by the user, it being understood that muscle activations that do not necessarily result in a net movement of the user’s limbs (such as “grasps” of objects) are also included.

[0043] Specifically, as exemplified in Figures 1 and 2, in a first phase, user II records his or her activity (i.e. , performance of one or more rehabilitation physiotherapy exercises) while wearing device B in the presence of an expert (the physiotherapist) at a rehabilitation facility, or even in private. The sEMG and / or IMU data for that first session, indicated by ref in Figure 1 , are saved in cloud C (or terminal S) as indicative of a reference session. Later, user II performs again the same exercise or series of exercises while wearing the device B, and the related sEMG and / or IMU data, indicated by cfr in Figure 1 , are saved in cloud C (or terminal S) as indicative of a comparison session. In the cloud C (or the terminal S), an algorithm processes and compares the two data streams (reference ref and comparison cfr) and determines one or more similarity indices GSI between the two executed exercises (e.g., one or more numerical values between 0 and 100). Once computed, the value of the GSI index is transmitted from cloud C to terminal S and / or feedback device F for it to be made available to user U (e.g., displayed on the screen and / or played acoustically by terminal S), who consequently can understand whether he or she is performing the rehabilitation exercise correctly (if the GSI index has a value above a certain threshold, e.g., above 90) or not (if the GSI index has a value below the threshold). Acquisition of the sEMG and / or IMU data by device B, transmission from device B to terminal S and from it possibly to cloud C, data analysis and calculation of the similarity index, and its transmission and display in terminal S can be carried out, advantageously and thanks to the algorithm described in more detail below, almost in real time and with a low demand on computational resources.

[0044] Figure 2 is a flowchart that illustrates a method 20 of using system 10. Specifically, a first phase 200 of method 20 relates to the execution of the reference session, which is to be carried out at the expert’s place or otherwise under his or her supervision (e.g., even under remote supervision). In an (optional) step 201 , user II goes to the expert rehabilitator (e.g., a physical therapist). In the next step 202, user II wears device B on the body segment that is the subject of the rehabilitation plan (e.g., an arm, leg, finger, etc.). In the next step 203, the experienced rehabilitator teaches user II the exercise to be carried out to rehabilitate the impaired part (i.e., the limb on which device B is applied). In the next step 204, device B detects sEMG, and optionally IMU, signals (data streams) while the exercise (which may include a series of repetitions of the rehabilitative movement) is being carried out, optionally filters them, and sends them to the terminal S to which it is connected (e.g., via Bluetooth). In the next step 205, terminal S records the received data related to the single reference session and optionally sends such data ref to cloud C for analysis according to the algorithm described below to determine a reference or “gold standard”. Once completed the first phase 200 of method 20, a second phase 250 is related to the execution of one (or more) comparison session, which the user II can carry out independently without the presence of the expert rehabilitator, at any time and any place. In a step 251 , user II wears device B on the body segment that is the subject of the rehabilitation plan. In a step 252, the user selects via the software running on the terminal S an exercise session dedicated to him or her, and carries it out. Optionally, the selection of the exercise session may include the playback, on terminal S, of an explanatory video of the exercise to be performed. In the next step 253, device B detects sEMG, and optionally IMU, signals (data streams) while the exercise (or multiple repetitions of the exercise) is being carried out, optionally filters them, and sends them to terminal S to which it is connected (e.g., via Bluetooth). In the next step 254, the terminal S records the received data related to the comparison session and optionally sends such data cfr to the cloud C for analysis according to the algorithm described below. In the next step 255, the software running on cloud C or terminal S computes one or more values of the GSI similarity index (e.g., depending on how many repetitions of the exercise were performed by user U) and transmits it to terminal S. In the next step 256, the value of the GSI similarity index is made available to user U via terminal S or another feedback device F (visual and / or auditory and / or haptic). In some embodiments, feedback may also be provided to the user II via the wearable device B, which for this purpose may include a light indicator (LED) and / or a sound indicator (buzzer) and / or a motor to generate a vibration. The feedback provided to the user may include:

[0045] - the level of accuracy of the exercises carried out by the user during the session;

[0046] - which exercises (e.g., movements) were performed best, i.e., most similar to the reference session;

[0047] - which exercises were performed worse, i.e., less similar to the reference session; and

[0048] - which exercises were skipped, that is, they were performed so dissimilarly from the reference session that they could not be graded.

[0049] As anticipated, the data streams (sEMG and optionally IMU) sensed by device B are analyzed remotely by platform C or locally by terminal S via a method 30 (e.g, a software-implemented algorithm) such as the one exemplified in the flowchart of Figure 3. For the analysis of each of the data streams (i.e., the reference data stream ref and the comparison data stream cfr), it is first useful to break down the stream into the various portions that correspond to the individual repetitions of the exercise; in fact, each session (both the reference and comparison sessions) usually includes the repetition in succession of the same rehabilitation exercise. To perform such an analysis, one or more embodiments exploit data indicative of human limb (e.g., arm) muscle synergies using a non-negative matrix factorization algorithm (“Hierarchical alternating least squares non-negative matrix factorization” or “HALS NMF”). The HALS NMF algorithm recursively divides an initial data matrix V into two separate matrices, W and H, both with non- negative elements, the product of which returns the original matrix V minus an established error. Specifically, the W matrix is indicative of the time series of the principal components of the original signal, while the H matrix represents the muscle synergies of the muscle groups involved, i.e., it contains the weights with which the individual channels of the sEMG sensor are involved in recording muscle contraction. The terminology “muscle synergies” is widely used in the technical field of reference of the present invention, as shown in documents [3], [4], [5],

[0010] , and

[0011] , and therefore does not need to be further defined in the text of the present description.

[0050] Therefore, an initial phase 300 of method 30 relates to the analysis of the data ref of the reference session.

[0051] In a step 301 , data are collected from a plurality of EMG channels (e.g., 8 EMG channels) and stored in a matrix V of size n x m, where n is the number of samplings made (and possibly filtered) during the entire recording of the reference session, and m is the number of sEMG sensor channels. For example, if the data streams are filtered at a frequency of 16 Hz and the recording of a session lasts 30 seconds (e.g., repeating ten times a single movement lasting about 3 seconds), n is equal to 30*16=4800.

[0052] In step 302, the matrix V of size n x m is factorized into two smaller matrices, W of size n x 1 and H of size 1 x m, using a HALS NMF algorithm with a single component.

[0053] In step 303, an adaptive threshold algorithm is applied to the matrix W, which contains the time series of the principal components of the electromyography signal, in order to choose a signal amplitude threshold above which the signal is considered indicative of voluntary muscle activation.

[0054] For example, in some embodiments, a copy of the matrix W is created in step 303 and its elements w are sorted in ascending order to subdivide the values from lowest to highest. The first p values (i.e., the smallest p values) are classified as the baseline values of the signal, while the remaining values (i.e., the largest n-p values) are classified as belonging to the peaks of the signal. As an example, p can be chosen equal to n / 10, n / 5, or n / 2. Thus, the threshold value wTH can be chosen as the average value between the rms value (RMS) computed on the p baseline values (RMSBaseline) and the rms value computed on the n-p peak values (RMSPeaks), according to the following formulas (where w_basei and w_peaki denote the elements of the matrix W, classified into the “baseline” and “peak” groups, respectively): RMSPeaks—

[0055] RMSBaseline + RMSpeaps

[0056] WTH 2

[0057] In other embodiments, the threshold value wTH can be determined in step 303 by a different algorithm. For example, the threshold value wTH can be determined by computing the average value of all the n elements w of the matrix W, and subtracting from this average value a correction factor that is determined as a percentage (e.g., 6.1 %) of the maximum value wMAX recorded in the matrix W, according to the following formula (where wi denotes the generic elements of the matrix W, wMAX denotes the largest of all values wi saved in the matrix W, and £ denotes the percentage, e.g., £ = 0.061 if the selected percentage is 6.1 %):

[0058] In still other embodiments, the threshold value wTH can be determined in step 303 by a different algorithm, such as by setting the threshold at a value equal to half of the maximum recorded value wMAX , according to the following formula:

[0059] _WMAX WTH ~ 2

[0060] In step 304, once the threshold wTH that represents a voluntary muscle activation has been determined, the matrix W is searched for elements w whose value is greater than the threshold wTH. Each sequence comprising a plurality of consecutive elements whose value is greater than the threshold wTH is then interpreted as a single repetition of the rehabilitative exercise. For each repetition thus identified, the element ws corresponding to the beginning of the contraction and the element wf corresponding to the end of the contraction are identified (e.g., for the hypothetical sequence [0, 0, 0, 2, 2, 2, 2, 0, 0], assuming that the activation threshold wTH is equal to 1 , we would have ws = 3 and wf= 7 if we assume that the elements wi are numbered starting from 0), obtaining a vector of time windows representative of the individual repetitions, which can be as many as r.

[0061] In step 305, the original matrix V is taken again and, using again the HALS NMF algorithm, matrices H (i.e., matrices indicative of muscle synergies) are computed only within the r time windows previously identified, thus obtaining a number r of matrices Hrip,ref, each of which describes the muscle synergies detected during a single repetition of the exercise. It will be noted that, even in the case where the time windows have different durations from each other, the respective matrices Hrip, ref all have the same dimension 1 x m.

[0062] In step 306, the average between all the r matrices Hrip, ref obtained in step 305 is computed to define a reference matrix Hgold representing the expected (average) muscle synergies for a given exercise performed by a given user. Such an average between matrices is computed element by element, i.e., the matrix Hgold has the same dimension 1 x m as the individual matrices Hrip, ref, and each element of the matrix Hgold is computed as the average value of the elements of all the r matrices Hrip, ref that occupy the same position.

[0063] A second phase 350 of method 30 relates to the analysis of the data cfr from the comparison session. Phase 350 includes steps 351 , 352, 353, 354, and 355, which are essentially the same processing steps 301 , 302, 303, 304, and 305 but applied to the comparison data cfr instead of the reference data ref. Thus, at the end of step 355, a set of q matrices H of size 1 x m (referred to herein as Hrip, cfr) is produced, each describing the muscle synergies detected during a single repetition of the exercise during the comparison session. It will be noted that again all matrices Hrip, cfr have the same dimension 1 x m, and that in general the number q of matrices Hrip, cfr determined at the output of step 355 may be different from the number r of matrices Hrip, ref determined at the output of step 305, because the number of repetitions performed during the comparison session may be different from the number of repetitions performed during the reference session.

[0064] In step 356, each of the q matrices Hrip, cfr produced in step 355 is compared with the reference matrix Hgold to compute a respective similarity index GSIEMG, single, varying between 0 and 100, between a single repetition of the exercise (made during the comparison session) and the reference exercise, according to the following formula:

[0065] Specifically, in the above formula, the difference Hgold-Hrip,cfr represents the error between the two matrices (element by element). Of the error matrix, the absolute value (element by element) is then calculated, and the resulting matrix is divided by the matrix Hgold, specifically by multiplying the absolute value matrix | Hgold-Hrip,cfr| by the pseudo-inverse of the matrix Hgold.

[0066] In step 357, an overall electromyography similarity index GSIEMG (i.e. , indicative of the whole session) is computed as the average value of the q indices GSIEMG, single, according to the following formula:

[0067] In step 360, the overall electromyography similarity index GSIEMG and / or the individual similarity indices GSIEMG, single computed by the algorithm run by the cloud C (or the terminal S) are transmitted to the terminal S to provide feedback to user II. The index GSIEMG takes a value from 0 to 100, giving user II a feedback on the unsupervised exercise session. The feedback can be interpreted according to one or more thresholds, for example: i) if the value of the index GSIEMG is between the maximum value (e.g., 100) and a threshold value (e.g., 70), the movement is identified as correct; and ii) if the value of the index GSIEMG is less than the threshold value, the movement is considered wrong.

[0068] In cases where the muscle activation results in a movement, even a complex movement, various embodiments may also use one or more data streams detected by the inertial sensor in the execution of method 30.

[0069] For example, in steps 301 and 351 , acceleration data from the accelerometer axes are collected, and stored in an matrix A of size n x t, where n is the number of samplings made (and possibly filtered) during the entire session recording (as for the matrix V), and t is the number of accelerometer axes (e.g., three axes). In steps 302 and 352, matrix A is factorized into two smaller matrices, T of size n x 1 and Z of size 1 x t, using, for example, a PCA (“Principal Component Analysis”) algorithm with a single component. Unlike a HALS NMF algorithm, a PCA algorithm returns a matrix T that can include both positive and negative values. In steps 303 and 353, an adaptive dual threshold algorithm is applied to the matrix T, using a positive threshold and a negative threshold (these thresholds being computable according to any of the approaches described above with reference to step 303 of method 30). In steps 304 and 354, elements whose value is greater than the positive threshold and elements whose value is less than the negative threshold are searched for in the matrix T in order to determine time windows representative of individual repetitions of the movement (where positive elements are indicative of a forward movement, and negative elements are indicative of a return movement). For each repetition thus identified, the element corresponding to the beginning of the movement and the element corresponding to the end of the movement are identified, resulting in a vector of time windows representative of the individual repetitions, which can be u in number. Since usually each repetition of the rehabilitative movement results in a continuous muscle activation that produces a forward movement and a return movement, the number of time windows u is usually equal to twice the number of time windows r (u = 2*r). In steps 305 and 355, the original matrix A is taken again and, again using the PCA algorithm, matrices Z (i.e., matrices indicative of accelerations) are computed only within the previously identified time windows, resulting in a number u of Z matrices, each of which describes the accelerations detected during a part of the rehabilitative movement. Of these matrices Z, ideally half are indicative of a forward movement, and half are indicative of a return movement. It will be noted that even in the case where the time windows have different durations from each other, the respective matrices Z all have the same dimension 1 x t. In step 306, the average between all “forward” matrices Za, rip, ref obtained in step 305 is computed (ideally, in the number of u / 2) to define a reference “forward” matrix Z Za,gold, which represents the expected accelerations during the forward leg of a given movement performed by a given user. Such an average between matrices is computed element by element, i.e., the matrix Za,gold has the same dimension 1 x t as the individual matrices Za, rip, ref, and each element of the matrix Za,gold is computed as the average value of the elements of all the u / 2 “forward” matrices Za, rip, ref that occupy the same position. Still in step 306, the average between all “return” matrices Zr, rip, ref obtained in step 305 (ideally, in the number of u / 2) is computed to define a reference “return” matrix Zr, gold , which represents the expected accelerations during the return leg of a given movement made by a given user. Such an average between matrices is computed element by element, i.e., the matrix Zr,gold has the same dimension 1 x t as the individual matrices Zr, rip, ref, and each element of the matrix Zr,gold is computed as the average value of the elements of all the u / 2 “return” matrices Zr, rip, ref that occupy the same position. In step 356, each of the u / 2 “forward” matrices Za,rip,cfr produced in step 355 is compared with the reference matrix Za,gold to compute a respective similarity index GSIACC, a, single, varying between 0 and 100, between the forward part of a single repetition of the movement (made during the comparison session) and the forward part of the reference movement, according to the following formula:

[0070] Still in step 356, each of the u / 2 “return” matrices Zr, rip,cfr produced in step 355 is compared with the reference matrix Zr,gold to compute a respective similarity index GSIACC, r, single, varying between 0 and 100, between the return portion of a single repetition of the movement (made during the comparison session) and the return portion of the reference movement, according to the following formula: 100

[0071] In step 357, a forward accelerometric similarity index GSIACC, a is computed as the average value of the u / 2 forward indices GSIACC, a, single, according to the following formula:

[0072] Still in step 357, a backward accelerometric similarity index GSIACC, r is computed as the average value of the u / 2 return indices GSIACC, r, single, according to the following formula: Still in step 357, an overall accelerometric similarity index GSIACC is computed as the average value of the two indices GSIACC, a and GSI ACC, r.

[0073] In some embodiments, the steps described above in relation to the processing of acceleration data collected by the accelerometer can be applied in addition to or as an alternative to the processing of angular velocity data collected by the gyroscope provided in the wristband B. Therefore, an overall gyroscopic similarity index GSIGIRO can be computed in a similar manner.

[0074] In one or more embodiments, a global similarity index GSI can be computed by fusing together data from the sEMG sensor, accelerometer, and / or gyroscope, for example, by computing a weighted average of the indices GSIEMG, GSIACC, and GSIGIRO. The weights aEMG, aACC and aGIRO associated with each of the indices GSIEMG, GSIACC and GSIGIRO, respectively, can be chosen depending on the exercise performed by the user. For example, if the exercise is a strength or resistance movement, the index GSIEMG will carry more weight than the indices GSIACC and GSIGIRO in the calculation of the overall index GSI (even, one could have aEMG = 1 , aACC = 0, and aGIRO = 0). In contrast, if the exercise is a mobility movement, the index GSIEMG will have less weight than the indices GSIACC and GSIGIRO in computing the overall index GSI (even, one could have aEMG = 0, aACC = 1 and aGIRO = 1 ).

[0075] The global similarity index GSI computed by the algorithm executed by cloud C (or terminal S) is transmitted to terminal S to provide feedback to user II in the same way as described above.

[0076] System 10 and its operating method can be used, for example, to detect and identify a number of rehabilitative exercises of a user’s arm and / or hand, such as the following twelve wrist exercises: wrist extension (shoulder angle: 0°, elbow angle: 90°); wrist flexion (shoulder angle: 0°, elbow angle: 90°); ulnar deviation (shoulder angle: 0°, elbow angle: 90°); radial deviation (shoulder angle: 0°, elbow angle: 90°); maximum pronation (shoulder angle: 0°, elbow angle: 90°); maximum supination (shoulder angle: 0°, elbow angle: 90°); wrist extension (shoulder angle: 30°, elbow angle: 0°); wrist flexion (shoulder angle: 30°, elbow angle: 0°); ulnar deviation (shoulder angle: 30°, elbow angle: 0°); radial deviation (shoulder angle: 30°, elbow angle: 0°); maximum pronation (shoulder angle: 30°, elbow angle: 0°); maximum supination (shoulder angle: 30°, elbow angle: 0°); and the following nine finger exercises: finger flexion; finger extension; thumb abduction; thumb adduction; grip I (proximal and distal flexion of the phalanges); grip II (thumb pincer grip); grip III (pen grip); grip IV (cylindrical grip); grip V (ball grip).

[0077] Therefore, the present invention provides a novel technique for comparing the similarity between two rehabilitative exercises carried out by a person: one carried out in the presence of an expert (e.g., a physical therapist, or a trainer if the use is intended in the sports sector) and another carried out independently (e.g., at home) by the user, without the use of classical machine learning models and without having to create a special training dataset. The invention also renders the expert largely autonomous, who will be free to administer to the user the exercises best suited to his or her rehabilitation path without having to choose from a set of predetermined exercises and without having to generate extensive databases of data for comparison, the latter operation that could also pose problems of sensitive data management and privacy.

[0078] One or more embodiments of the present invention may therefore provide one or more of the following advantages:

[0079] - quick and easy use of the system, which features a simple setup including a common electronic device (e.g., smart phone, computer, tablet) and an inexpensive and easy-to-wear sensor;

[0080] - accurate comparison of gestures / movements;

[0081] - operation that is not based on classical classification networks, and therefore not very onerous in terms of the computational resources required;

[0082] - operation that does not require the organization of a dataset for training a hypothetical classification network, and thus allows the use immediately after acquisition of a single reference with which to compare exercises performed later;

[0083] - accurate recognition for any type of chosen gesture / movement, with the ability to recognize even exercises that do not result in a net movement of the user’s limb; and

[0084] - possibility of autonomously defining new gestures / movements that best suit the user’s needs, the benchmark being produced by the user himself and not through predefined datasets.

[0085] Without prejudice to the underlying principles, the details and embodiments may vary even considerably from what has been described here only as an example, without departing from the scope of protection. The scope of protection is defined by the annexed claims.

[0086] LIST OF DOCUMENTS CITED

[0087] [1] Bonato P., D’Alessio T., Knaflitz M., “A statistical method for the measurement of muscle activation intervals from surface myoelectric signal during gait,” IEEE Trans. Biomed. Eng., 1998 Mar; 45(3):287-99, doi: 10.1109 / 10.661154

[0088] [2] Cichocki A., Phan A., “Fast Local Algorithms for Large Scale Nonnegative Matrix and Tensor Factorizations,” IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, 2009, vol. E92-A., issue 3, pp. 708-721 , doi: 10.1587 / transfun.E92.A.708

[0089] [3] J. Ma, N. V. Thakor and F. Matsuno, “Hand and Wrist Movement Control of Myoelectric Prosthesis Based on Synergy,” IEEE Transactions on Human-Machine Systems, vol. 45, no. 1 , pp. 74-83, Feb. 2015, doi: 10.1109 / THMS.2014.2358634

[0090] [4] Ahn, S.H., Kwon, S., Na, Y. et al., “Grasp Behavior Analysis Using Muscle and Postural Hand Synergies for Smartphones,” International Journal of Precision Engineering and Manufacturing, 22, 697-707 (2021), doi: 10.1007 / s12541 -020-00467-w

[0091] [5] Maistrello L., Rimini D., Cheung V. C. K., Pregnolato G., Turolla A., “Muscle Synergies and Clinical Outcome Measures Describe Different Factors of Upper Limb Motor Function in Stroke Survivors Undergoing Rehabilitation in a Virtual Reality Environment,” Sensors, 2021 ; 21 (23):8002; doi: 10.3390 / s21238002

[0092] [6] US 2019 / 0121306 A1

[0093] [7] US 9367139 B2

[0094] [8] US 9389694 B2

[0095] [9] US 10796599 B2

[0010] V. C. K. Cheung, K. Devarajan, G. Severini, A. Turolla and P. Bonato, “Decomposing time series data by a non-negative matrix factorization algorithm with temporally constrained coefficients,” 2015 37th Annual International Conference of the IEEE Engineering in Medicine and

[0096] Biology Society (EMBC), 2015, pp. 3496-3499, doi: 10.1109 / EMBC.2015.7319146

[0097]

[0011] Pale II., Atzori M., Muller H., Scano A., “Variability of Muscle Synergies in Hand Grasps: Analysis of Intra- and Inter-Session Data,”

[0098] Sensors (Basel), 2020 Aug 1 ; 20(15):4297, doi: 10.3390 / s20154297

Claims

CLAIMS1. A method of detecting and identifying muscle activations made by a user (II), the method comprising: receiving (200), from a plurality of surface electromyography sensors provided in a wearable electronic device (B) and configured to sense a plurality of surface electromyography signals on the skin of the user (II), a first stream of data (ref) indicative of said plurality of surface electromyography signals sampled during a first session of use in which the user (II) repeatedly makes a given muscle activation; applying (301 , 302, 303, 304, 305) a non-negative matrix factorization algorithm to said first stream of data (ref) indicative of said plurality of surface electromyography signals to compute a set of reference matrices (Hrip,ref), each of said reference matrices (Hrip,ref) being indicative of the muscle synergies detected during a respective repetition of said muscle activation made by the user (II) during said first session of use; computing (306) the average value of said reference matrices (Hrip,ref) to define an average reference matrix (Hgold) of the muscle synergies associated to said given muscle activation; receiving (250), from said plurality of surface electromyography sensors, a second stream of data (cfr) indicative of said plurality of surface electromyography signals sampled during a second session of use in which the user (II) repeatedly makes said given muscle activation; applying (351 , 352, 353, 354, 355) said non-negative matrix factorization algorithm to said second stream of data (cfr) indicative of said plurality of surface electromyography signals to compute a set of comparison matrices (Hrip, cfr), each of said comparison matrices (Hrip,cfr) being indicative of the muscle synergies detected during a respective repetition of said muscle activation made by the user (II) during said second session of use; computing (356), for each of said comparison matrices (Hrip, cfr), a single-repetition electromyography similarity index (GSIEMG, single) indicative of the difference between said comparison matrix (Hrip, cfr) and said average reference matrix (Hgold); computing (357) an overall electromyography similarity index(GSIEMG) between said first session of use and said second session of use as an average value of said single-repetition electromyography similarity indices (GSIEMG, single)', and transmitting (360) said single-repetition electromyography similarity indices (GSIEMG, single) and / or said overall electromyography similarity index (GSIEMG) to a user terminal (S) of the user (II).

2. The method of claim 1 , comprising the step of visually, acoustically and / or haptically reproducing the values of said singlerepetition electromyography similarity indices (GSIEMG, single) and / or of said overall electromyography similarity index (GSIEMG) via said user terminal (S), via said wearable electronic device (B) and / or via a feedback device (F).

3. The method of claim 1 or claim 2, wherein the steps of applying said non-negative matrix factorization algorithm to said first stream of data (ref) and to said second stream of data (cfr) to compute, respectively, a set of reference matrices (Hrip,ref) and a set of comparison matrices (Hrip,cfr) each comprise: storing (301 , 351 ) the values of said plurality of surface electromyography signals sampled during the first session of use and during the second session of use in a first sampling matrix and in a second sampling matrix (V), respectively; factoring (302, 352) said first sampling matrix and said second sampling matrix (V) by applying a hierarchical alternating least squares nonnegative matrix factorization algorithm, HALS NMF, producing respectively a first component matrix and a second component matrix (W) indicative of the time series of the components of said plurality of surface electromyography signals and a first muscle synergy matrix and a second muscle synergy matrix (H); applying a threshold algorithm (303, 353) to said first component matrix and to said second component matrix (W) to determine, respectively, a first threshold value and a second threshold value (wTH) indicative of a voluntary muscle activation; identifying (304, 354), in said first component matrix and in saidsecond component matrix (W), the elements whose value is higher than said first threshold value and than said second threshold value (wTH), respectively, thereby identifying a first plurality of time windows and a second plurality of time windows, respectively; factoring (305, 355) said first sampling matrix and said second sampling matrix (V) by applying said hierarchical alternating least squares non-negative matrix factorization algorithm, HALS NMF, limitedly respectively first to said first plurality of time windows and to said second plurality of time windows, producing respectively said set of reference matrices (Hrip, ref) and said set of comparison matrices (Hrip,cfr).

4. The method of any of the previous claims, wherein the step of computing (356), for each of said comparison matrices (Hrip,cfr), a singlerepetition electromyography similarity index (GSIEMG, single) comprises computing the difference matrix between said average reference matrix (Hgold) and said comparison matrix (Hrip, cfr) and multiplying said difference matrix by the pseudoinverse matrix of said average reference matrix (Hgold).

5. The method of any of the previous claims, comprising: receiving (200), from an accelerometer provided in said wearable electronic device (B) and configured to sense a plurality of acceleration signals, a first stream of data (ref) indicative of said plurality of acceleration signals sampled during said first session of use; applying (301 , 302, 303, 304, 305) a principal component analysis algorithm, PCA, to said first stream of data (ref) indicative of said plurality of acceleration signals to compute a set of reference acceleration matrices (Zrip,ref), each of said reference acceleration matrices (Zrip,ref) being indicative of the accelerations detected during a respective repetition of said muscle activation made by the user (II) during said first session of use; computing (306) a positive average value of the positive reference acceleration matrices (Za, rip, ref) to define an average positive reference acceleration matrix (Za,gold) of the accelerations associated to said given muscle activation, and computing (306) a negative average value of the negative reference acceleration matrices (Zr, rip, ref) to define an averagenegative reference acceleration matrix (Zr,gold) of the accelerations associated to said given muscle activation; receiving (250), from said accelerometer, a second stream of data (cfr) indicative of said plurality of acceleration signals sampled during said second session of use; applying (351 , 352, 353, 354, 355) said principal component analysis algorithm, PCA, to said second stream of data (cfr) indicative of said plurality of acceleration signals to compute a set of comparison acceleration matrices (Zrip,cfr), each of said comparison acceleration matrices (Zrip,cfr) being indicative of the accelerations detected during a respective repetition of said muscle activation made by the user (II) during said second session of use; computing (356), for each of said comparison acceleration matrices (Hrip,cfr), a single-repetition acceleration similarity index (GSIACC, single) indicative of the difference between said comparison acceleration matrix (Zrip,cfr) and one of said average positive reference acceleration matrix (Za,gold) and said average negative reference acceleration matrix (Zr,gold); computing (357) an acceleration similarity index of the positive accelerations (GSIACC, a) as an average value of said single-repetition acceleration similarity indices (GSIACC, single) corresponding to positive accelerations; computing (357) an acceleration similarity index of the negative accelerations (GSIACC, r) as an average value of said single-repetition acceleration similarity indices (GSIACC, single) corresponding to negative accelerations; computing (357) an overall acceleration similarity index (GSIACC) between said first session of use and said second session of use as an average value of said acceleration similarity index of the positive accelerations (GSIACC, a) and said acceleration similarity index of the negative accelerations (GSIACC, r) and transmitting (360) said single-repetition acceleration similarity indices (GSIACC, single) and / or said overall acceleration similarity index (GSIACC) to said user terminal (S).

6. The method of any of the previous claims, comprising:receiving (200), from a gyroscope provided in said wearable electronic device (B) and configured to sense a plurality of angular velocity signals, a first stream of data (ref) indicative of said plurality of angular velocity signals sampled during said first session of use; applying (301 , 302, 303, 304, 305) a principal component analysis algorithm, PCA, to said first stream of data (ref) indicative of said plurality of angular velocity signals to compute a set of reference angular velocity matrices, each of said reference angular velocity matrices being indicative of the angular velocities detected during a respective repetition of said muscle activation made by the user (II) during said first session of use; computing (306) a positive average value of the positive reference angular velocity matrices to define an average positive reference angular velocity matrix of the angular velocities associated to said given muscle activation, and computing (306) a negative average value of the negative reference angular velocity matrices to define an average negative reference angular velocity matrix of the angular velocities associated to said given muscle activation; receiving (250), from said gyroscope, a second stream of data (cfr) indicative of said plurality of angular velocity signals sampled during said second session of use; applying (351 , 352, 353, 354, 355) said principal component analysis algorithm, PCA, to said second stream of data (cfr) indicative of said plurality of angular velocity signals to compute a set of comparison angular velocity matrices, each of said comparison angular velocity matrices being indicative of the angular velocities detected during a respective repetition of said muscle activation made by the user (II) during said second session of use; computing (356), for each of said comparison angular velocity matrices, a single-repetition gyroscopic similarity index (GSIGIRO, single) indicative of the difference between said comparison angular velocity matrix and one of said average positive reference angular velocity matrix and said average negative reference angular velocity matrix; computing (357) a gyroscopic similarity index of the positive angular velocities (GSIGIRO, a) as an average value of said single-repetition angular velocity similarity indices (GSIGIRO, single) corresponding to positive angular velocities;computing (357) a gyroscopic similarity index of the negative angular velocities (GSIGIRO, r) as an average value of said single-repetition angular velocity similarity indices (GSIGIRO, single) corresponding to negative angular velocities; computing (357) an overall gyroscopic similarity index (GSIGIRO) between said first session of use and said second session of use as an average value of said gyroscopic similarity index of the positive angular velocities (GSIGIRO, a) and said gyroscopic similarity index of the negative angular velocities (GSIGIRO, r) and transmitting (360) said single-repetition gyroscopic similarity indices (GSIGIRO, single) and / or said overall gyroscopic similarity index (GSIGIRO) to said user terminal (S).

7. The method of claim 5 or claim 6, comprising: computing (357) a global similarity index (GSI) between said first session of use and said second session of use as a weighted average of said overall electromyography similarity index (GSIEMS) and at least one of said overall acceleration similarity index (GSIACC) and said overall gyroscopic similarity index (GSIGIRO)', and transmitting (360) said global similarity index (GSI) to said user terminal (S), wherein preferably the weights associated to said overall electromyography similarity index (GSIEMS), said overall acceleration similarity index (GSIACC) and said overall gyroscopic similarity index (GSIGIRO) are selected as a function of the muscle activation made by the user (II).

8. The method of any of the previous claims, wherein said surface electromyography signals are sampled at a first frequency, the method comprising: filtering at a second frequency said surface electromyography signals sampled during said first session of use to produce said first stream of data (ref)', and filtering at said second frequency said surface electromyography signals sampled during said second session of use to produce said secondstream of data (cfr), said second frequency being lower than said first frequency, preferably at least ten times lower.

9. A system (10) comprising: an electronic device (B) wearable by a user (II) and comprising a plurality of surface electromyography sensors configured to sense a plurality of surface electromyography signals on the skin of the user (II); and a user terminal (S) provided with a processing unit; wherein said wearable electronic device (B) and said user terminal (S) are configured to operate according to the method of any of claims 1 to 8.

10. The system (10) of claim 9, further comprising a remote processing system (C), wherein said wearable electronic device (B), said user terminal (S) and said remote processing system (C) are configured as a whole to operate according to the method of any of claims 1 to 8, in particular wherein: the user terminal (S) is configured to receive from said wearable electronic device (B) and transmit to said remote processing system (C) said streams of sampled data (ref, cfr) indicative of said plurality of surface electromyography signals; and the remote processing system (C) is configured to receive said streams of sampled data (ref, cfr) indicative of said plurality of surface electromyography signals from said user terminal (S), and to transmit said single-repetition electromyography similarity indices (GSIEMG, single) and / or said overall electromyography similarity index (GSIEMG) to said user terminal (S).

11. The system (10) of claim 9 or claim 10, wherein: said wearable electronic device (B) comprises an accelerometer configured to sense a plurality of acceleration signals; and said wearable electronic device (B), said user terminal (S) and optionally said remote processing system (C) are configured as a whole to operate according to the method of claim 5.

12. The system (10) of any of claims 9 to 11 , wherein: said wearable electronic device (B) comprises a gyroscope configured to sense a plurality of angular velocity signals; and said wearable electronic device (B), said user terminal (S) and optionally said remote processing system (C) are configured as a whole to operate according to the method of claim 6.

13. A computer program product loadable in the memory of at least one processing device (B, S, C) and comprising software code portions configured to cause said at least one processing device (B, S, C) to operate according to the method of any of claims 1 to 8 as a result of the product being run on said at least one processing device (B, S, C).

Images