MOTION SENSOR DATA ENCODING METHOD, ASSOCIATED DEVICE AND SYSTEM
The method encodes motion sensor data into reduced-bit descriptors for efficient energy use and processing, addressing high energy consumption in gait study systems, enabling extended battery life and real-time analysis.
Patent Information
- Application Number
- FR2021003875
- Authority / Receiving Office
- FR · FR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-04-14
- Publication Date
- 2024-12-20
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing gait study systems face challenges with high energy consumption due to real-time transmission of raw data, leading to short battery autonomy, and limited data processing capabilities at the measuring device, which hinders long-term, real-time gait analysis.
A method for encoding motion sensor data using processors to calculate and transform descriptors into at most eight bits, allowing for reduced energy consumption and efficient data transmission, combined with machine learning models for real-time gait determination.
Enables extended battery life and detailed real-time gait analysis with minimal data loss, supporting long-term monitoring and accurate gait determination.
Smart Images

Figure 00000040_0000 
Figure 00000041_0000 
Figure 00000042_0000
Abstract
Description
Title of the invention: METHOD FOR ENCODING MOTION SENSOR DATA, ASSOCIATED DEVICE AND SYSTEM
[0001] The invention relates to the field of gait study and more particularly to the processing of raw data from inertial units to facilitate the real-time study of movements, posture or more broadly the activity of a person. The invention relates to a method for encoding data generated by one or more motion sensors preferably coupled to a shoe. The invention further relates to an encoding device and a system for encoding data generated by one or more motion sensors capable of implementing such a method. Prior art
[0002] Human gait is a complex and cyclical process aimed primarily at maintaining an upright position and balance in static or dynamic conditions. Such a process requires the synergy of muscles, bones and the nervous system.
[0003] Many systems have been developed over the years to study human gait. These systems include, for example, the use of shoes equipped with pressure sensors or inertial sensors positioned in the sole, on the shoe or at the ankle. In particular, several teams have focused on the use and processing of inertial sensor data for the analysis of human movement. These systems, based on one or more sensors, today provide values of classic "walking" parameters indicating, for example, information on the number of steps thanks to calculations most often carried out entirely on external terminals.
[0004] Furthermore, these systems are most often based on pressure sensors or inertial sensors coupled to remote computing units in external devices with high computing capacities. Thus, current gait study systems generally involve the transmission of high-frequency data from the onboard sensors to the computing units. These systems must be able to operate continuously without having to change / recharge the battery frequently while providing a detailed analysis of a user's gait. However, in conventional systems, the real-time transmission of raw data results in high energy consumption, whereas these systems must be able to operate autonomously over a long period. Alternatively, the transmission of exclusively pre-processed data limits the possibilities of evolving interpretations of data measured by motion sensors.
[0005] A study carried out in 2018 shows that out of the 14 systems evaluated, the median autonomy of the proposed systems is less than 6 hours. This is clearly insufficient for consumer applications where monitoring of an individual's gait over at least one day is desired ("A survey on smart shoe insole Systems". Salma Saidani et al. 2018 10.1109 / SMARTNETS.2018.8707391; ©2018 IEEE).
[0006] One solution to increase the autonomy of systems for studying an individual's gait is to increase the size of the battery. However, such a solution cannot be applied in all situations, particularly when miniaturization of the tracking devices is desired. For example, inertial measurement units have been proposed that are capable of operating for approximately 35 hours by sending raw data to an external device capable of analyzing them ("Bluetooth Embedded Inertial Measurement Unit for Real-Time Data Collection for Gait Analysis". Nimsiri Abhayasinghe et al. October 2013; Conference: Indoor Positioning and Indoor Navigation. Montbéliard (France)). However, such an inertial unit has dimensions of the order of 50 cm3 and a weight of the order of 50 grams.Such dimensions are imposed in particular by the use of a battery with sufficient capacity to continuously transmit data to an external processing device.
[0007] Other solutions, already proposed by the applicant (WO2019193301, WO2019077266, WO2020217037), are to allow at least partial calculation at the level of the movement measurement device so as not to transfer all of the measured data and thus reduce the consumption associated with the storage and / or transfer of data. However, such a solution presumes having all the algorithms already integrated into the measurement device. However, in certain cases, only part of the data processing algorithms have been integrated into the measurement device. There may also be a need to develop the data processing methods.
[0008] Thus, there is a need for a method of encoding motion sensor data that allows for data storage and transfer that is less energy-intensive at the measuring device level while minimizing data loss as much as possible. [Technical problem]
[0009] The invention aims to overcome the drawbacks of the prior art. In particular, the invention aims to propose a method for encoding data generated by one or more motion sensors preferably coupled to a shoe, capable of generating / transmitting information that is faithful to the measured data and sufficient for studying the gait of the individual wearing said motion sensors. movements. Such a process allows systems to operate for longer periods without having to change / recharge the battery, while still providing a detailed analysis of a user's gait. In addition, such a process makes it possible to determine an individual's gait in real time.
[0010] The invention further aims to propose a data encoding device and a system for determining a user's gait integrating such a device, said system having improved autonomy while being capable of studying the user's gait in real time. [Brief description of the invention]
[0011] To this end, the invention relates to a method for encoding data generated by one or more motion sensors coupled to a shoe, for identifying a position or movement of the shoe over a time sequence,
[0012] said method, executed by one or more processors, comprising:
[0013] - A step of acquiring data generated by the motion sensor(s) coupled to the shoe, said data comprising variable values in the form of time series, preferably said variable values comprising acceleration and angular velocity values; - A step of calculating, from the data acquired over a time window, a plurality of descriptors, said calculation step being repeated for several consecutive time windows, the consecutive time windows forming the time sequence; and - A step of transforming the plurality of descriptors of each of the consecutive time windows forming the time sequence, said transformation step comprising a generation for each of the descriptors of data coded on at most eight bits, preferably on at most four bits
[0014] said data coded on at most eight bits associated with the time sequence being representative of a position or a movement of the shoe on said time sequence.
[0015] The applicant has developed an on-board method capable of processing almost instantaneously (e.g. time less than 1 ms with a processor rate of 64 MHz), preferably by processors coupled to the motion sensors, the raw data generated by the motion sensors. The shoe is for example worn by a user. This makes it possible to encode them in a format which can either be processed at the level of a processor coupled to the motion sensor, for example in a sole, or transmitted punctually to an external terminal with reduced energy consumption and / or reduced storage memory.
[0016] The applicant has developed in particular a data encoding allowing better interpretation by automatic learning of movements. Without being bound In theory, this encoding can be considered as a phonemization of the time series from motion sensors. Thus, the method will generate from a time series of motion sensor data, a set of "phonemes" capable of describing consecutive intervals of the time series. The particular combination or assembly of these phonemes will make it possible to define particular postures or movements.
[0017] Furthermore, such encoding also gives an algorithm the possibility of processing the encoded data in real time to translate it into predictions of an individual's gait (e.g. movement and / or posture).
[0018] The data thus encoded may then be processed or transmitted in the form of a data tensor. The method then comprises a step of generating a data tensor comprising the data coded on at most eight bits obtained for several of the time windows, so that the data tensor comprises an encoding representative of a posture or a movement of the shoe. Furthermore, the data tensor may comprise an encoding representative of a plurality of postures or movements of the shoe.
[0019] According to other optional features of the method, the latter may optionally include one or more of the following features, alone or in combination:
[0020] - the one or more motion sensors as well as the one or more processors are integrated into an encoding device, preferably positioned in a sole. Such a device incorporating a motion sensor and processors makes it possible to carry out part of the processing as close as possible to the generation of raw motion data and therefore to accelerate the processing and reduce the energy cost. - The encoding device is positioned in a removable sole. The presence of the encoding device in a removable sole is possible thanks to the low consumption of the process and allows the process to be used on several footwear items. - the calculation step involves pre-processing of the acquired data. It may be advantageous to modify the data, for example, via filtering, in order to speed up subsequent processing and in some cases reduce the associated energy consumption. - the descriptors are calculated from mathematical functions applied to the data, pre-processed or not, generated by the motion sensor(s) coupled to the shoe, and said applied mathematical functions comprising for example a calculation of average, statistical asymmetry, flattening coefficient, comparisons of values of the
[0021] - extrema or quartiles, positions of extrema in a time series, and / or comparisons of edge values of subsets of the movement and / or a calculation of a topology of the data. it comprises a step of extracting a time sequence that may correspond to a movement unit, said extraction step implementing a learning model, preferably an unsupervised learning model. A prior extraction of a movement unit can make it possible to improve the prediction accuracy of the method while reducing the associated energy consumption. It comprises a step of analyzing the data coded on at most eight bits, preferably the data tensor, as a function of a user activity datum, said analysis step comprising a comparison between the data coded on at most eight bits or preferably the data tensor and a database comprising reference data for the given activity. This advantageously makes it possible to determine which data are associated with movements relating to an activity and which data correspond to risky or unintentional movements. The given activity may, for example, correspond to a sporting activity but also a leisure or professional activity. The time windows of two consecutive time sequences have different durations. Thus, the analysis can adapt to different gait types, different gait cycle temporalities and does not require lengthy data segmentation. during the transformation step, at least four descriptors are used, each generating data coded on at least one bit, more preferably on at least two bits. during the transformation step, several descriptors are transformed into a single data item of at most eight bits, preferably at most three bits. In certain cases, “composite” descriptors, formed from several descriptors may be the only ones required. In particular, during the transformation step, at least eight descriptors are used, each generating data item coded on at least one bit, more preferably at least two bits. one or more data encoded on at most eight bits, preferably at most three bits, are calculated from values of descriptors calculated from data acquired on consecutive time windows. In certain cases, a "composite" descriptor, formed from several descriptors coming from different time windows can improve the performance. - during the transformation stage, between four and twenty descriptors are used, each generating data coded on a number of bits selected from: one, two or three. Such values allow for better performance in terms of data retention, storage volume and power consumption.
[0022] According to a second object, the invention relates to a method for determining the gait of a user of a shoe, from data likely to have been encoded according to an encoding method according to the invention, said determination method, executed by one or more processors, comprising:
[0023] - A step of loading one or more data tensors comprising an encoding representative of a position or movement of the user's shoe; and - A step of processing the data tensor(s) by a machine learning model trained to identify a posture or movement of the user.
[0024] According to other optional features of the method, the latter may optionally include one or more of the following features, alone or in combination:
[0025] - the user's gait comprises one or more movements and / or postures of the user. - the step of processing the data tensor(s) by a machine learning model comprises the identification of event classes from one or more data tensors, preferably one motion class per time sequence. - it comprises a step of selection, preferably by the trained machine learning model, of a posture or movement of the user from a comparison of the postures or movements of the user identified from contiguous time windows or contiguous time sequences. - it involves the identification, by the trained learning model, of a posture or movement for each of several contiguous time sequences, the method then comprising a selection of the posture or movement most represented within the contiguous time sequences. Such a step makes it possible to reduce the risks of false positives by taking advantage of real-time recognition of time series and multiple validation of the same posture or movement. For example, a movement can be selected from an analysis of at least 3, preferably at least 4 and even more preferably at least five time sequences contiguous pores. - it comprises a step of comparing postures and / or movements of the user identified for a given time window or time sequence from data from different sensors so as to identify a posture or movement of the user. In particular, it may comprise, for a unit of movement, a comparison of the movement identified for each of the feet, or a comparison with respect to a reference system, so as to determine a posture or movement.
[0026] The invention further relates to a device for encoding data generated by one or more motion sensors coupled to a shoe, for identifying a position or movement of the shoe over a time sequence, said encoding device comprising one or more processors, said one or more processors being configured to:
[0027] - Acquire the data generated by the motion sensor(s) coupled to the shoe, said data comprising variable values in the form of time series, preferably acceleration and angular velocity values; - Calculate, from the data acquired over a time window, a plurality of descriptors, the calculation being repeated for several consecutive time windows, the consecutive time windows forming a time sequence; - Transforming the plurality of descriptors of each of the consecutive time windows forming the time sequence, said transformation comprising a generation for each of the descriptors of data coded on at most eight bits, preferably at most three bits, said data coded on at most eight bits associated with a time window being representative of a movement of the shoe on said time sequence.
[0028] Further, the encoding device may include the one or more motion sensors. Further, the one or more processors may be configured to generate a data tensor including the encoded data on at most eight bits obtained for several of the time windows, such that the data tensor includes an encoding representative of a position or motion of the shoe.
[0029] The invention further relates to a system for determining the gait of a user, said determination system comprising at least one encoding device according to the invention and an external terminal configured to receive the generated data tensor and then to determine the gait of a user from the generated data tensor.
[0030] Other advantages and characteristics of the invention will appear on reading the following description given as an illustrative and non-limiting example, with reference to the attached Figures:
[0031] [Fig.l], [Fig.l] represents a diagram of a method according to an embodiment of the invention. The dotted steps are optional.
[0032] [Fig.2], [Fig.2] represents a diagram of a method according to another embodiment of the invention. The dotted steps are optional.
[0033] [Fig.3A] and [Fig.3B and 3C], Figures 3A, 3B each represent a graph showing data generated, divided into a time sequence subdivided into time windows, by a motion sensor along the x axis. Figure 3C corresponds to second-order data tensors that can together form a third-order tensor.
[0034] [Fig.4], [Fig.4] represents a diagram of a method for determining the gait of a user according to an embodiment of the invention. The dotted steps are optional.
[0035] [Fig.5], [Fig.5] represents a diagram of an open encoding device seen from above comprising in particular an electronic card, a power source, a connector and a means of communication.
[0036] [Fig.6], [Fig.6] represents a diagram of a system for determining the gait of a user according to the invention.
[0037] Aspects of the present invention are described with reference to flowcharts and / or functional diagrams of methods, apparatuses (systems) and computer program products according to embodiments of the invention.
[0038] In the figures, flowcharts and block diagrams illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a system, device, module, or code, which includes one or more executable instructions for implementing the specified logical function(s). In some implementations, the functions associated with the blocks may appear in a different order than shown in the figures. For example, two blocks shown in succession may, in fact, be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order, depending on the functionality involved.Each block of the block diagrams and / or flowchart, and combinations of blocks in the block diagrams and / or flowchart, may be implemented by special hardware systems that perform the specified functions or acts or carry out combinations of special hardware and computer instructions. [Description of the invention]
[0039] In the remainder of the description, the “approach” within the meaning of the invention corresponds to the user's posture, movements, locomotion, and balance. Balance refers in particular to postural balance linked to the stability of the body and more specifically to the stability of a user's center of gravity. However, it can integrate both static and dynamic balance.
[0040] “Determining the gait” corresponds, within the meaning of the invention, to the attribution of one or more values, for example a score, a ranking or a note to a trajectory or to the movement of a user’s foot. This determination of the gait can also make it possible to obtain one or more numerical or alphanumeric values of biomechanical parameters representative of the gait.
[0041] The term "activity" within the meaning of the invention may correspond to an action carried out by the user over a prolonged period. This may, for example, correspond to a sporting activity such as football, cycling or basketball, but also to a sedentary activity such as an office activity or any other activity such as recreational walking or hiking.
[0042] The term “Descriptor” within the meaning of the invention may preferably correspond to a value obtained via a mathematical transformation of the data acquired from the motion sensors. A mathematical transformation may, for example, correspond to the calculation of a mean, a median, a variance, statistical asymmetry, the flattening coefficient, extreme values, quartile values, comparisons of extreme or quartile values, positions of extremes in a time series, and comparisons of the edge values of the subsets of the motion. For example, for a window, an edge value corresponds to the initial and final value of the window.
[0043] The expression “movement phoneme” within the meaning of the invention may correspond to a set of data calculated from several descriptors generated over a given time window. These data advantageously take the form, within the framework of the invention, of data encoded over at most eight bits. Thus, the movement phoneme corresponds to several values encoded over at most eight bits describing at least in part a movement over the time window studied. These movement phonemes may thus correspond to data calculated from several descriptors describing a sub-part of a movement called a micromovement, that is to say a movement which may, for example, take place over less than 500 ms.
[0044] The expression “data tensor” within the meaning of the invention may correspond to an object, or to a data structure, such as a one-dimensional table, a two-dimensional matrix or even a cube comprising data relating to descriptor values transformed into values coded in at most eight bits, preferably in at most two bits. The representation in the form of a tensor is a representation of the data allowing, for example, the use of matrix calculation operations, optimized in terms of computational and memory complexity. In addition, data tensors, when corresponding to the values of a sensor for a time window, can form a third-order data tensor and can be processed together with data tensors for other sensors and data tensors for other time windows to form a list of third-order data tensors.
[0045] For the purposes of the invention, the term “biomechanical parameter” means a characteristic of the user’s gait.
[0046] The term "sole" means an object that separates the user's foot from the ground. A shoe may comprise an upper sole layer in direct contact with the user's foot and a lower sole layer in direct contact with the ground or more generally the external environment. A shoe may also comprise a removable inner sole.
[0047] “Removable” means the ability to be detached, removed or dismantled easily without having to destroy the fastening means either because there are no fastening means or because the fastening means are easily and quickly removable (e.g. notch, screw, tab, lug, clips). For example, removable means that the object is not fixed by welding or by another means not intended to allow the object to be detached.
[0048] By "model" or "rule" or "algorithm", it is necessary to understand in the sense of the invention a finite sequence of operations or instructions making it possible to calculate a value by means of a classification or a partitioning of the data within previously defined groups Y and to assign a score or to prioritize one or more data within a classification. The implementation of this finite sequence of operations makes it possible for example to assign a label Y to an observation described by a set of characteristics or parameters X thanks for example to the implementation of a function f, capable of reproducing Y having observed X.
[0049] Y = f (X) + e
[0050] where e symbolizes the noise or measurement error.
[0051] By "supervised learning method", we mean in the sense of the invention a method making it possible to define a function f from a base of n labeled observations (Xi n, Yi n) where Y = f (X) + e.
[0052] By "unsupervised learning method" is meant a method aimed at hierarchizing data or dividing a set of data into different homogeneous groups, the homogeneous data groups sharing common characteristics and this without the observations being labeled.
[0053] “Substantially constant” means a value varying by less than 15% compared to the compared value, preferably by less than 10%, even more so preferred by less than 5%.
[0054] For the purposes of the invention, the term "process", "calculate", "determine", "display", "transform", "extract", "compare" or more broadly "executable operation" means an action performed by a device or processor unless the context indicates otherwise. In this regard, operations refer to actions and / or processes of a data processing system, for example a computer system or an electronic computer device, which manipulates and transforms data represented as physical (electronic) quantities in the memories of the computer system or other devices for storing, transmitting or displaying information. These operations may be based on applications or software.
[0055] The terms or expressions "application", "software", "program code", and "executable code" mean any expression, code or notation, of a set of instructions intended to cause data processing to perform a particular function directly or indirectly (e.g. after a conversion operation to another code). Examples of program code may include, but are not limited to, a subroutine, a function, an executable application, source code, object code, a library and / or any other sequence of instructions designed for execution on a computer system.
[0056] For the purposes of the invention, the term "processor" means at least one hardware circuit configured to execute operations according to instructions contained in a code. The hardware circuit may be an integrated circuit. Examples of a processor include, but are not limited to, a central processing unit, a graphics processor, an application-specific integrated circuit (ASIC), and a programmable logic circuit.
[0057] The term "computer device" means any device comprising a processing unit or a processor, for example in the form of a microcontroller cooperating with a data memory, possibly a program memory, said memories being able to be dissociated. The processing unit cooperates with said memories by means of an internal communication bus.
[0058] For the purposes of the invention, the term "IT infrastructure" means a set of IT structures (i.e. IT devices) capable of operating an application or an application chain. The IT structure may be a server and may, for example, be composed of a presentation server, a business server and a data server. Preferably, the IT structure is a server.
[0059] For the purposes of the invention, the term "coupled" means connected, directly or indirectly, with one or more intermediate elements. Two elements can be coupled mechanically, electrically, or linked by a communication channel.
[0060] In this description and even before, the same references are used for designate the same elements.
[0061] The inventors tested several methods and systems to study a user's gait (e.g. posture and / or movement). They favored systems integrating inertial sensors, systems most capable of providing a rapid response and at a cost acceptable for industrialization. Indeed, existing methods are generally based on pressure sensors or on a plurality of sensors distributed in the shoe and / or sole or even on the ankle. Such a distribution of sensors leads to a reduction in the robustness of the system and / or an increase in its cost.
[0062] However, as mentioned, the vast majority of systems for studying an individual's gait suffer from the autonomy of the battery used, this being mainly caused by the real-time transmission of raw data generated by motion sensors and in particular by inertial units. The alternative, namely to size a battery capable of lasting several tens of hours, then concerns systems which could no longer be considered portable or in common use.
[0063] Faced with these shortcomings, the applicant has developed an on-board method capable of processing the raw data generated by the motion sensors almost instantaneously and encoding them in a format allowing transmission with reduced energy consumption and / or reduced storage memory. Furthermore, the method allows encoding such that the transmitted data, although requiring only a small amount of energy, retains a maximum of information and then allows a detailed analysis of the gait of the user of the motion sensors.
[0064] In particular, the applicant has developed a solution for encoding data generated by one or more motion sensors. As mentioned, this encoding could be considered to be based on a phonemization of the movement linked to the gait of an individual (e.g. posture and / or movement).
[0065] According to a first aspect, the invention relates to a method 1 for encoding data generated by one or more motion sensors. In particular, such a method can be implemented for one or more motion sensors 21 coupled to a shoe, in particular a shoe worn by a user. It may in particular be implemented by one or more processors 22.
[0066] In particular, such a method is implemented from data generated by one or more motion sensors 21 of a sole 11, 12, described below. Some of the motion sensors 21 may be distributed in the sole 11, 12 or may also be positioned on the shoe so as to follow the movements of the sole. The motion sensors may also all be integrated into a first encoding device 20 associated with a first shoe and a second encoding device 20 associated with a second shoe of the same pair.
[0067] An encoding method 1 according to the invention will advantageously allow identification of a position or movement of a shoe coupled to one or more movement sensors 21 over a time sequence ST, when the shoe is worn by a user.
[0068] As illustrated in [Fig.l], the encoding method 1 comprises: a step 200 of acquiring data generated by the motion sensor(s) 21, a step 300 of calculating a plurality of descriptors from the data acquired over a time window 31, a step 400 of transforming the plurality of descriptors and a step 500 of generating a data tensor comprising the bits obtained for each of the time windows 31, such that the data tensor comprises an encoding of a combination of several motion phonemes which is representative of a position or movement of the user or a position or movement of the shoe worn by a user.
[0069] Furthermore, a method according to the invention may comprise advantageous steps such as: learning 100, calibration 210 to the user's gait, preprocessing 220 of the data generated by the motion sensors and / or values calculated within the framework of the method, extraction 230 of a time sequence corresponding to a unit of movement, transmission 600 of a data tensor, determination 700 of the user's gait or even storage 900 of the data.
[0070] In the remainder of this description, the solution proposed by the applicant will be described in an embodiment in which the motion sensor(s) 21 comprise at least one inertial unit of at least six axes and this or these motion sensors 21 are integrated into a sole 11, 12. The sole(s) involved in the present invention may or may not be removable. Preferably, the sole(s) involved in the present invention is / are removable.
[0071] Nevertheless, the present invention in these different embodiments, preferred or not, could be applied in the case where the motion sensors are not integrated into a sole. It could in particular be applied when the motion sensors are positioned on a shoe, but also at the level of a user's body, for example integrated into a bracelet or even in clothing worn by the latter. Indeed, the movement data generated by one or more motion sensors positioned on the body of a user, for example at the level of lower limbs or the spine, could be encoded by an encoding method according to the invention and / or used to determine the user's gait.
[0072] The motion sensors could also be pressure sensors which, if positioned appropriately, can provide information on movements. especially of an individual.
[0073] In the context of the following description, preferably, the one or more processors are positioned in a sole. Alternatively, the one or more processors could be positioned in a housing arranged to be attached to a shoe or to a user.
[0074] In particular, the one or more motion sensors as well as the one or more processors are integrated into an electronic box, such as an encoding device 20, preferably positioned in a sole.
[0075] As illustrated in [Fig.2], a method according to the invention may comprise a learning step 100.
[0076] A learning step 100 may include a step of collecting training data and then aggregating and storing this target data in a database. This training data may or may not be annotated and may be used to generate data tensors that can be used in a training step, preferably distributed and parallelized.
[0077] Preferably, the learning step 100 may include the generation and selection of multiple learning models by cross-validation and selection of optimized hyper parameters. Finally, the learning step 100 may include a selection of the learning model providing the best classification performance, i.e. optimal and homogeneous performance between the classes.
[0078] Such a step may in particular comprise the definition of a plurality of reference values for the activity of a user. For example, during the repetition of movements, postures and steps performed by a user over a determined period, reference values for the activity may be recorded and classified according to a plurality of values, including patterns, for example of a static or dynamic type.
[0079] A method according to the invention comprises a step 200 of acquiring data generated by the motion sensor(s) 21, preferably coupled to a shoe. The acquisition, as can be understood from reading the present description, is preferably carried out for each of the two shoes and, where appropriate, for the two soles of the shoes. This step may correspond in particular, within the framework of the invention, to the acquisition of data generated by one or more motion sensors of an encoding device 20 or of several encoding devices 20, for example each integrated into a removable or non-removable sole. This acquisition may, for example, correspond to storing, preferably in a random access memory, the data generated by the sensors. This acquisition may also correspond to loading into the memory of one or more processors intended to process them.
[0080] In particular, the acquired data is raw data generated by the motion sensors 21. Alternatively, the acquired data may also correspond to pre-processed data. As will be described below, the pre-processed acquired data may, for example, correspond to data that has been normalized, filtered, supplemented, or to data that has been merged, for example, by Kalman filtering. Preferably, the acquired data comprises variable values in the form of time series. These variable values may preferably correspond to acceleration and angular velocity values, but they may, for example, also correspond to values generated by pressure sensors. They may optionally comprise magnetic field data or a fusion of these data. Thus, the data generated by the motion sensor(s) may also comprise magnetic field data.
[0081] This acquisition step may also include the acquisition of other data. such as geolocation data, temperature data, or even pressure data.
[0082] In particular, when the acquired data has been generated by motion sensors positioned in a sole, this data will preferably correspond to acceleration and angular velocity values of the sole(s).
[0083] In particular, a method according to the invention may comprise a calibration step 210. As will be described later, the motion sensors 21 may be associated with a shoe and may be integrated into an encoding device 20.
[0084] A method according to the invention may also comprise pre-processing 220 of the data generated by the motion sensors before or after their acquisition.
[0085] In particular, this preprocessing step 220 may correspond to the preprocessing of the acceleration, angular velocity and / or orientation values of a sole or even pressure values. For example, it may include in particular at least one processing selected from: frequency filtering, gravity suppression on the acceleration values, gravity suppression, noise suppression on the acceleration values and drift suppression on the angular velocity and / or orientation values of the sole.
[0086] As mentioned and illustrated in [Fig.2], a method according to the invention may comprise a step 230 of extracting a time sequence which may correspond to a unit of movement.
[0087] The unit of movement within the meaning of the invention may correspond to a walking cycle, a running cycle or even a stair climbing cycle. These cycles may be defined as the period of time from the initial contact of a foot to the next occurrence of the same event with the same foot.
[0088] Within the scope of the invention, numerous partitioning methods can be used. Thus this step can for example include the implementation of classic steps for studying step movement, for example the Pan-Tompkins method or the detection of thresholds, maxima and / or minima.
[0089] The extraction 230 may advantageously implement a machine learning model, preferably an unsupervised machine learning model.
[0090] A method according to the invention also comprises a step 300 of calculating a plurality of descriptors from the data acquired over a time window 31.
[0091] Advantageously, this calculation step can be repeated for several consecutive time windows 31. As will be described later, the consecutive time windows 31 form a time sequence 32. This is notably illustrated in FIGS. 3A and 3B.
[0092] Figures 3A and 3B represent data generated by a motion sensor, for example acceleration data along the x axis, preprocessed or not. This data can be divided into time sequences 32, two in [Fig.3A], then subdivided into time windows 31.
[0093] As has been presented, a method according to the invention may comprise a step 230 of extracting a time sequence 32 which may correspond to a unit of movement. Alternatively, a time sequence 32 may correspond to a sliding window as illustrated in FIG. 3B. Furthermore, as illustrated in FIG. 3B, preferably, the time sequences 32 may be overlapping.
[0094] The time window 31 does not necessarily correspond to a predetermined duration. Furthermore, preferably, it does not correspond to a predetermined duration. For example, the duration of the time window 31 can be defined once a unit of movement, or a type of movement has been identified.
[0095] As illustrated in Figure 3B, all time windows 31 of a time sequence 32 may have an identical duration. Furthermore, the time windows 31 of two consecutive time sequences 32 may have different durations.
[0096] Thus, the duration of a time window can be adapted to the duration of a movement unit. This makes it possible to consider approaches independently of their execution time.
[0097] For example, the duration of each of the time windows 31 of a time sequence 32 may correspond to a predetermined fraction of the duration of the time sequence 32.
[0098] The duration of a time window 31 may for example be between 25 and 1000 ms while the duration of a time sequence 32 may for example be between 100 and 5000 ms. A time window 31 may for example represent between 1 / 20 and 1 / 4 of a time sequence 32, preferably between 1 / 10 and 1 / 5 of a time sequence 32.
[0099] Advantageously, the descriptors are calculated from mathematical functions applied to the data generated by the motion sensor(s) 21, preprocessed or not. The mathematical functions preferably include calculations of statistical asymmetry, flattening coefficient, values of the extremums and / or comparisons of the edge values of the subsets of the motion.
[0100] In addition, the implemented mathematical functions can make it possible to calculate so-called “topological” descriptors making it possible to extract, measure and compare structural information hidden within complex data. In this case, the mathematical functions can be adapted to allow the calculation of the data topology, notably by determining a uni- or multidimensional persistence diagram (as presented in H. Edelsbrunner and J. Harer. Computational Topology: An Introduction. American Mathematical Society, 2009, or in H. Edelsbrunner, D. Letscher, and A. Zomorodian. Topological persistence and simplification. Disc. Compu. Geom., 2002), the Reeb graph (J. Tiemy, A. Gyulassy, E. Simon, and V. Pascucci. Loop surgery for volu-metric meshes: Reeb graphs reduced to contour trees. IEEE Transactions on Visualization and Computer Graphics (Proc, of IEEE VIS), 2009.), or the Morse-Smale complex (A. Gyulassy, PTBremer, B. Hamann, and V. Pascucci. A practical approach to morse-smale complex computation: Scalability and generality. IEEE Transactions on Visualization and Computer Graphics (Proc. of IEEE VIS), 2008). Of course, the computation of these topological descriptors is not limited to the use of the mathematical functions described as an illustrative example; other mathematical functions can be used to determine other types of topological descriptors.
[0101] Preferably, the applied mathematical functions incorporate data from several motion sensors to generate data coded on at most eight bits, preferably at most three bits, more preferably at most two bits. For example, in the case of an inertial unit generating acceleration data on three axes, a descriptor coded on at most three bits, preferably at most two bits, may have been generated from the values of the three axes.
[0102] Furthermore, the applied mathematical functions can advantageously incorporate data from a previous time window to calculate a descriptor of a given window. Thus, the values of the descriptors can transcribe an evolution of the values generated by the motion sensors.
[0103] Preferably, the applied mathematical functions can also incorporate variables not originating from the motion sensors.
[0104] Preferably, the step 300 of calculating a plurality of descriptors will comprise the calculation of at least 4 descriptors, preferably at least six descriptors for each of the time windows 31.
[0105] A method according to the invention comprises a step 400 of transforming the plurality of descriptors.
[0106] This transformation step 400 of the plurality of descriptors advantageously comprises a coding of descriptors, each on at most eight bits, preferably at most five bits, more preferably at most four bits, even more preferably at most three bits, for example each on at most two bits. Preferably, during the transformation step 400, the descriptors of the plurality of descriptors are each transformed into a data item coded on two bits.
[0107] Generally, all calculated descriptors will be encoded. However, it is possible that not all calculated descriptors will be encoded. For example, the method according to the invention may include a step of selecting the descriptors that will be encoded or not.
[0108] Generally, this encoding will be exclusively associated with a descriptor. However, it is possible that several descriptors contribute to the definition of a value encoded on at most eight bits which will be complementary to encoded values specific to a descriptor. Furthermore, it is possible that a descriptor is involved in the definition of several values encoded on at most eight bits. In one embodiment, during the transformation step 400, several descriptors could be used to generate a complementary two-bit encoded data item.
[0109] The inventors tested several methods for processing the data generated by the motion sensors. In addition, they tested several methods for reducing the dimensionality of the data produced by these motion sensors. However, it is the method according to the present invention which made it possible to achieve the best results in terms of energy consumption and data compression while retaining the useful information of the transmitted data to correctly represent the user's gait.
[0110] Table 1 below represents normalized system performance data based on the processing of data acquired from the data sensors before transmission. [YES] Table 1: Battery consumption Retention of useful information Required data storage Data No encoding 100 1 100 One-bit encoding; 4 descriptors 1 38.2 1 Two-bit encoding; 4 descriptors 5.5 23.7 3.7 Three-bit encoding; 4 descriptors 15.1 14.2 5.2 Four-bit encoding; 4 descriptors 21.9 9 16 One-bit encoding; 8 descriptors 3.1 24.1 3.2 Two-bit encoding; 8 descriptors 10.8 11.9 7.3 Three-bit encoding; 8 descriptors 26.5 8 16.5 Four-bit encoding; 8 descriptors 34 4.9 29.1 Eight-bit encoding; 8 descriptors 59 2.4 42.6
[0112]
[0113]
[0114]
[0115]
[0116] Thus, the battery consumption values correspond to battery consumption values normalized according to the highest consumption (100) and the lowest consumption (1). The useful information retention values of the data correspond to values normalized according to a capacity to retain a maximum amount of information (1) and a minimum retention (100). In particular, the information retention of the data is estimated through the accuracy of the movement predictions by a set of machine learning and supervised models based on the encoded data. Finally, the required storage values of the data correspond to the volume in terms of memory space normalized from the highest value (100) to the lowest value (1). Preferably, during the transformation step 400, at least eight descriptors are used and each coded on at least one bit, more preferably at least two bits. Generally, during the transformation step 400, descriptors are each coded on at most eight bits, preferably at most six bits and even more preferably at most four bits. For example, among the preferred embodiments, during the transformation step 400, the coding is carried out on: - one bit for between 4 and 20 descriptors, for example four descriptors or eight descriptors; - two bits for between 4 and 20 descriptors, for example four descriptors or eight descriptors; - three bits for between 4 and 20 descriptors, for example four descriptors. The coding will preferably be carried out for each of the time windows 31.
[0117] Thus, the data encoded on at most two bits and associated with a time window 31 will preferably be representative of what could be considered as a phoneme of movement of the shoe. Alternatively, these data encoded on at most two bits and associated with a time window 31 can be considered as a phoneme if a phonemization of the movements associated with the gait is considered.
[0118] Advantageously, several encoded data (eg a plurality of descriptors each encoded on at most two bits) characterize a movement phoneme and several movement phonemes characterize a movement.
[0119] Thus, the data encoded under a series of values encoded on at most two bits associated with a time window 31 are representative of a micromovement of the user, in particular of a shoe and more particularly of a sole 11,12.
[0120] Table 2 below illustrates an example of data obtained following a transformation step according to the invention. FT1 FT2 FT3 Calculated Encoded Calculated Encoded Calculated Encoded Descriptor 1 12.45989 -1 70.8759 1 49.3476 0 Descriptor 2 9.7228 -1 23.7459 0 93.8016 2 Descriptor 3 69.7904 1 69.0302 1 1.6486 -1 Descriptor 4 17.9573 -1 24.2259 0 38.1186 0 Descriptor 5 24.2259 0 147.2232 2 72.8551 1
[0121] The coding of the descriptors and therefore of the values obtained can be carried out on the basis of predetermined intervals. For example, in the context of descriptor 1 which can correspond to a maximum acceleration value on the Y axis, the method according to the invention can use predetermined rules according to which a value less than 20 will be transformed into a value “-1”, a value from 20 to 50 (50 inclusive) will be transformed into a value “0”, a value between 50 (50 excluded) and 80 (80 excluded) will be transformed into a value “1”, and a value greater than or equal to 80 will be transformed into a value “2”. Thus, descriptor 1 is transformed into data coded on at most eight bits, in this case 2 bits.
[0122] The predetermined intervals can be modified over time either automatically based on the values obtained or through updates.
[0123] The coding of the descriptors can also correspond to comparisons of values of different descriptors or of the same descriptor over several tem windows porelles 31. For example, if descriptor 3, which can correspond to the average of the acceleration on the x axis, does not vary by more than 10% between the FTO and FT1 windows then its value will be transformed into a value “1”. In the same way, descriptor 3 does not vary by more than 10% between the FT1 and FT2 windows then its value will be transformed into a value “1”. On the other hand, descriptor 3 varies by more than 80% downward between the FT2 and FT3 windows then its value will be transformed into a value “-1”.
[0124] Thus, the coding of the descriptors, that is to say the generation for each of the descriptors of data coded on at most eight bits, can advantageously be based on predetermined rules. In addition, it can advantageously take into account predetermined intervals, changes in values over time and / or results of mathematical operations between the descriptors.
[0125] A method according to the invention may lead, as will be described later, to the generation of a data tensor which could, according to the example above, be represented in the following form:
[0126] VI = {-1,-1,1,-1,0;1,0,1,0,2;0;2,-1,0,1}
[0127] Such a data tensor comprises several sets of data encoded in at most two bits. For example, it comprises a set of five values encoded in at most two bits corresponding to the encoding of descriptors for the time window denoted “FT1”; a set of five values encoded in at most two bits corresponding to the encoding of descriptors for the time window denoted “FT2”; and a set of five values encoded in at most two bits corresponding to the encoding of descriptors for the time window denoted “FT3”.
[0128] Thus, if the time windows FT1, FT2 and FT3 (table 2) correspond to a 32 time sequence and therefore to a unit of movement then said movement of one second can be described by fifteen values encoded on two bits, i.e. in the case illustrated by 30 bits. Furthermore, in the case of the use of a sliding 32 time sequence then, such a movement could be represented by five 32 time windows, i.e. 150 bits.
[0129] This compares to 14,400 bits of data from motion sensors such as 9-axis inertial measurement units sampling one second of motion at 100 Hz.
[0130] The present invention therefore allows, with a minimal reduction in information, a reduction in the necessary data flows of around one hundred, resulting in a reduction in energy consumption and the necessary storage capacities.
[0131] Thus, a method according to the invention may also comprise a step 500 of generating a data tensor comprising the coded data obtained for each of the time windows 31.
[0132] The data tensor comprising the coded data obtained for each of the time windows 31 noted “FT” in table 2 may correspond to a plurality of data tensors each comprising the coded data obtained for a time window 31.
[0133] Thus, the data tensor preferably comprises an encoding of a combination of several phonemes of movement of the user, preferably of a shoe and in particular of a sole. The data tensor advantageously comprises an encoding which is representative of a position or a movement of the shoe worn by a user.
[0134] The inventors have further shown that the use of such a data tensor, comprising data coded on at most eight bits, preferably at most three bits, more preferably at most two bits, makes it possible to improve the performance of a machine learning model based on such a data tensor. Indeed, a machine learning model based on an encoding according to the invention will, where appropriate, have a reduced neural network size and / or a reduced calculation time while being tolerant to the statistical variabilities of the representative populations of the data sets and flexible to the mechanical sensitivity of the inertial sensors.
[0135] Advantageously, the data tensor comprises a series coded on four bits at most, preferably three bits at most and more preferably two bits at most. However, it may comprise other data which will not necessarily be coded on at most two bits. For example, the data tensor may comprise uncoded data which may correspond to attributes calculated at the level of the sole, for example, and describing an aspect of the user's gait.
[0136] Alternatively, the data tensor 34 corresponds to a series coded on eight bits at most, preferably at most three bits, preferably two bits at most.
[0137] The generation step 500 may comprise the generation of several data tensors for a time window 31. This is for example the case if a data tensor is generated for each of the axes of an inertial platform. Alternatively, a data tensor may comprise the coded data relating to several axes of an inertial platform.
[0138] A method according to the invention may also comprise a step 600 of transmitting the data tensor.
[0139] Advantageously, the data being coded on at most eight bits, the transmission can advantageously be the subject of a concatenation within data frames allowing the transfer of data frames for example of 8 bits, 16 bits or 32 bits. Thus the number of messages is reduced by the optimization of the occupation of the frames. Thus the transmission frames are optimized reducing the energy cost and the cost of verification of the packets (eg Ack...). This is particularly significant in the context of low energy transmissions such as BLE (for Bluetooth Low Energy in Anglo-Saxon terminology) or Zigbee.
[0140] Although a step of determining a movement or more generally a user's gait can be carried out at the level of the device carrying out the step 200 of acquiring data generated by the movement sensor(s), it is also possible to divide the processing and to interpret the data tensor in an electronic device different from the electronic device having generated the data tensor.
[0141] The transmission may for example be carried out by wireless communication means such as radio communications. The transmission of the data tensor is then preferably made to an external mobile device. The transmission of the data tensor could also be made to a remote computing device such as a data processing server.
[0142] The transmission of the data tensor can also be done directly to a device for processing the data measured by one motion sensor to another. For example, from one sole to another sole of the same pair of shoes. The transmission of the data tensor can be carried out continuously with the transmission of one data tensor at a time. It can also correspond to the transmission of several data tensors. For example, a transmission can include several data tensors, each corresponding to a different time window, or to data tensors generated from variables of different motion sensors.
[0143] According to another aspect, the present invention relates to a method for determining 2 the gait of a user. Preferably, the gait of the user corresponds to a posture, a movement, or an activity of the user.
[0144] The determination method 2 will preferably be executed by one or more processors. Advantageously, the method 2 for determining the gait of a user is implemented at least in part at the level of an encoding device 20, preferably positioned on or in a shoe, for example in a sole or in a shoe. Alternatively, the method 2 for determining the gait of a user is implemented at least in part at the level of an external terminal 30 such as a mobile device, or of a computer infrastructure.
[0145] Such a determination will preferably be made from data encoded, or capable of being encoded, according to an encoding method 1 according to the present invention.
[0146] In particular, the method 2 for determining the gait of a user according to the invention will be based at least in part on encoded data characterized in that the encoded data take the form of a time series for which each time window 31 comprises at least four encoded data on at most eight bits, preferably at most three bits, more preferably at most two bits. Preferably, each time window 31 comprises between 4 and 50 data encoded on at most eight bits, preferably at most three bits, more preferably at most two bits. The time series may correspond for example to one or more movement unit(s) and each of the movement units being formed by several time windows 31.
[0147] In particular, the method 2 for determining the gait of a user according to the invention will be based at least in part on encoded data characterized in that the encoded data take the form of one or more data tensors comprising for each time window 31 at least four data encoded on at most eight bits, preferably at most three bits, more preferably at most two bits. Preferably, each time window 31 is represented by 4 to 50 data encoded on at most three bits, preferably at most two bits. The one or more data tensors corresponding for example to one or more movement(s) and each of the movements being formed represented by several time windows.
[0148] As illustrated in [Fig.4], the method 2 for determining a user's gait comprises: a step 710 of loading one, preferably several data tensors; and a step 720 of processing the data tensors preferably by a machine learning model trained so as to identify a posture or a movement.
[0149] The method 2 for determining a user's gait preferably corresponds to a real-time determination of one or more constituent elements of the user's gait. That is to say, after the generation of data by one or more motion sensors, preferably in less than 60 seconds, more preferably in less than 30 seconds, even more preferably in less than 10 seconds, the method according to the invention is able to determine an element of the user's gait such as a posture, a step, a jump, or any other movement constituting an individual's gait.
[0150] As illustrated in [Fig.4], the determination method 2 according to the invention may also comprise the following steps: a step 730 of selecting a posture or a movement for a time interval from the postures or movements identified for contiguous time sequences, a step 740 of comparing the posture and / or movement identified from data from different sensors and a step 750 of transmitting data to an external terminal.
[0151] As mentioned, the method 2 for determining a user's gait comprises a step 710 of loading one or more data tensors.
[0152] Loading the data tensor may correspond to loading several data tensors. This step may correspond in particular, within the framework of the invention, to the storage, preferably in a random access memory, of one or more data tensors. This acquisition may also correspond to a loading into the memory of one or more processors intended to process them.
[0153] As mentioned, the method 2 for determining a user's gait comprises a step 720 of processing the data tensor(s) by a machine learning model.
[0154] In particular, the step 720 of processing the data tensor by a machine learning model will comprise the identification of event classes from the data tensor(s). In particular, the processing step 720 makes it possible, from several data tensors, preferably each corresponding to a movement phoneme, to identify an event class. Preferably, the processing step 720 is carried out on data tensors of order 3 comprising several data tensors.
[0155] Figure 3C illustrates for example three data tensors 34 in the form of three second-order tensors each comprising five data tensors. Each data tensor here corresponds to data calculated from a time window 31 and a data tensor 34 can then correspond to a time sequence 32. A second-order tensor illustrated in Figure 3C comprises data coded on one bit (0 or 1) for seven descriptors and on five time windows 31.
[0156] In Figure 3C, the tensors 34 correspond to sliding and overlapping time windows 32. A time sequence 32 could also correspond to a window whose size varies as posture, movement or more broadly event class calculations progress. For example, a first time sequence 32 comprises three time windows 31, a second time sequence 32 comprises four time windows 31 and so on until a predetermined value of maximum dimension of time sequence 32 is reached. The time sequence 32 could then see its size reduced to a predetermined value of minimum dimension.
[0157] Thus, the data coded in bits associated with one or more time windows 31, preferably representative of a movement phoneme, will be processed by a machine learning model to infer an event class such as a movement or posture class. A movement class may correspond, in the sense of the invention, to a movement taking place over a very short duration, for example less than 500 ms, and may correspond to portions of longer movement. For example, it will be possible, in the context of the invention, from the bits associated with a time window 31 or a time sequence 32, to determine whether there is kneeling in progress, a step forward, a step backward, etc.
[0158] Advantageously, the solution according to the invention uses one or more algorithms derived from machine learning, preferably previously calibrated, for processing the data tensor(s). These algorithms may have been constructed from different learning models, supervised, unsupervised or by reinforcement. An unsupervised learning algorithm may for example be selected from an unsupervised Gaussian mixture model, a hierarchical ascending classification (Hierarchical clustering Agglomerative in English terminology), a hierarchical descending classification (Hierarchical clustering divisive in English terminology). Alternatively, the algorithm is based on a supervised statistical learning model configured so as to minimize a risk of the ordering rule and thus making it possible to obtain more efficient prediction rules.In this case, the determination and estimation calculation steps can be based on a model, trained on a dataset and configured to predict a label (e.g. gait similar or not to the recorded gait). For example, for the purposes of calibration, it is possible to use a dataset representative of a situation whose label is known. The dataset can also include multiple labels. The algorithm can be derived from the use of a supervised statistical learning model selected for example from kernel methods (e.g. Wide Margin Separators - Support Vector Machines SVM, Ridge Kernel Regression), ensemble methods (e.g. decision trees, Random Forests), hierarchical partitioning, k-means partitioning, logical regression or neural networks.
[0159] Preferably, the trained machine learning model is preferably selected from a neural network, for example a deep learning model, a recurrent neural network (eg an LSTM, for "Long Short-Term Memory" in English terminology), a multi-layer perceptron or other machine learning methods such as methods based on decision trees (eg random forest) or clustering methods (eg partitioning method or hierarchical methods).
[0160] Furthermore, advantageously, the method according to the invention may comprise a step 730 of selecting a posture or a movement from the postures or movements identified for contiguous time sequences 32. This selected posture or movement may in particular correspond to the posture or movement of a user during a particular time interval 33 having been the subject of several contiguous time sequences 32 as illustrated in FIG. 3B. The time interval 33 may correspond to a unit of movement.
[0161] Thus, the step 730 of selecting a posture or a movement can be considered as a validation of a movement or a posture from the processing multiple classes of motion or posture inferred during the data tensor processing step 720 by a machine learning model.
[0162] The selection step 730 could include a comparison of the posture or movement identified for contiguous time sequences 32. Indeed, in particular when a sliding window is used, or more generally when a movement is represented by several contiguous time sequences, as is the case in FIG. 3B, the method according to the invention can include an identification of a posture or movement for each of the several contiguous time sequences 32. Then, the method includes a selection of the posture or movement most represented within the contiguous time sequences 32 studied. This makes it possible to reduce the risks of false positives by taking advantage of real-time recognition of posture or movement within time series and multiple validation of the same posture or movement.For example, a posture or movement may be selected from an analysis of at least 3, preferably at least 4, and even more preferably at least five contiguous time sequences.
[0163] Preferably, the selection step 730 is carried out on data tensors of order 3 comprising several data tensors.
[0164] Furthermore, advantageously, the method according to the invention may also include a step 740 of comparing postures and / or movements identified, for example for a given instant, from data from different sensors.
[0165] Such a comparison 740 of identified postures and / or movements can be used to determine the user's gait by associating information from several sensors.
[0166] This comparison 740 is preferably made for synchronized time intervals so that the measured and interpreted data correspond to the same moment in the user's approach. Such synchronization is possible, for example, through the use of a synchronized clock or even inter-device communications.
[0167] In particular, the method 2 for determining a user's gait can be implemented from data tensors originating from several sources. For example, from several motion sensors positioned at different points of a user also generating data tensors according to the present invention.
[0168] Under these conditions, the method 2 for determining a user's gait can advantageously comprise, for a given time interval, a comparison 740 of the posture, and / or the movement, identified from the data of a first sensor with the posture, or the movement, identified from a second sensor. For example, the step 740 of comparing the posture, and / or the movement, identified can include a comparison of the posture, or movement, identified from a sensor positioned on the right foot with the posture or movement identified from a sensor positioned on the left foot.
[0169] The step 740 of comparing postures and / or movements identified for a given instant for different sensors may implement postures or movements identified from data from different sensors not implementing the data tensor according to the invention. For example, the posture, or the movement, identified from data encoded according to the invention may be compared with the posture, or the movement, identified with other methods.
[0170] In particular, the method 2 for determining a user's gait may advantageously include a comparison of the posture or movement identified for each of the feet with a reference system. Such a comparison will make it possible to strengthen the specificity of the determination performance and in certain cases to determine a type of movement or a complex posture. Indeed, certain movements or postures can only be determined accurately by taking into account the results of several movement sensors. The reference system may consist of a database, an algorithm transcribing expert rules, or even a machine learning model.
[0171] Preferably, the method may also comprise a step 750 of transmitting data to an external terminal 30. This transmission is preferably done punctually. In particular, this may correspond to the transmission of all the data generated and / or calculated to an external terminal 30.
[0172] Furthermore, the method may comprise the transmission of the data received by the external terminal 30 to one or more other external terminals which could be involved in the calculation of other parameters of the approach or in their display.
[0173] The transmitted data may, for example, be raw data as generated by the motion sensors, preprocessed data, transformed data or even gait data.
[0174] Furthermore, a determination method 2 according to the invention may also comprise a step 800 of identifying an activity associated with one or more identified movements and / or postures.
[0175] This identification of an activity preferably comprises the comparison of activity reference values to the values of one or more data tensors or to the values of one or more identified movements or postures.
[0176] It is then possible, preferably from several identified movements and / or postures, to determine what activity is practiced by the user.
[0177] Below, the various mandatory or optional steps of a method 2 for determining a user's approach are presented to summarize and put into perspective. the present invention.
[0178] During the processing 720 of the data tensor by a machine learning model, the present invention may for example study a plurality of time windows 31 so as to identify a class of movement, such as a step, for each of the time windows 31. More preferably, during the processing 720 of the data tensor, in particular a data tensor of order 3, by a machine learning model, the present invention may study a plurality of time windows 31 so as to identify a class of movement, such as a step, for each of the time sequences 32 comprising several time windows 31.
[0179] An optional additional step 730 of selecting a posture or a movement can be implemented so as to confirm, by analyzing the movement classes identified for several contiguous time windows 31 or for several time sequences 32, the presence of a step. For example, a time sequence 32 can indicate a step while the following three indicate a slip. In this case, at the end of the additional step 730 of selecting a posture or a movement, the present invention may indicate the occurrence of a slip.
[0180] Furthermore, more generally, the analysis of the movement classes identified for several contiguous time windows 31 or for several time sequences 32, can make it possible to indicate the occurrence of a “risky” movement. Indeed, when the identified movement is not associated with any inferred movement or posture class, it can be the consequence of an involuntary movement of the user and potentially the source of muscular and / or skeletal injury for the user. This can be particularly applicable in physically demanding professions that can lead to injuries, particularly in the lower limbs, such as construction and public works.
[0181] In an optional additional step 740 of comparing identified postures and / or movements, the present invention will study data generated by other sensors, preferably also in the form of a data tensor (e.g. a data tensor of order 3), for example a data tensor generated from the movements of the left foot. In the continuation of the example above, the analysis of the data tensor(s) coming from the left foot may indicate a flexion of the knee. In this case, the movement identified at the end of step 740 may for example be a mowing. Thus, the movement or posture identified at the end of step 740 may be considered as a more complex movement.
[0182] Finally, in a step of identifying the activity 800, the present invention will be able, from generated data, to determine several movements which will lead, for example, to identifying that the activity carried out is a judo-type sporting activity.
[0183] Preferably, the method may also comprise a step 900 of storing data. This storing is preferably done continuously. In particular, this may correspond to the storing of all the data generated and / or calculated within the framework of a method according to the invention. The stored data may, for example, be raw data as generated by the motion sensors, preprocessed data, transformed data or gait data. Preferably, the stored data correspond to transformed data or gait data.
[0184] Advantageously, as opposed to raw and / or pre-processed data which are saved for a short period (e.g. less than 5 minutes) for example on a cache memory, a data tensor or gait value is stored for a longer term, for example on a memory.
[0185] Advantageously, the calculation of a data tensor and / or of a determined gait is done in real time, that is to say less than 1 hour after the generation of the data by one or more of the movement sensors of a sole, preferably less than 10 minutes, more preferably less than one minute, even more preferably less than 10 seconds.
[0186] According to another aspect, the invention relates to a device 20 for encoding data generated by one or more motion sensors 21, preferably coupled to a shoe worn by a user.
[0187] Such a data encoding device 20 can in particular be used to identify a position or movement of a shoe over a time sequence 32, in particular when the shoe is worn by a user. More broadly, if it is applied to motion sensors 21 positioned on the body of a user, it can be used to identify a position or movement of the user. More specifically, such an encoding device 20 is particularly advantageous when it is used to identify a position or movement of a sole 11, 12 for a shoe when it is inserted into a shoe or integrated into a shoe.
[0188] Weighing only a few grams and being of reduced size, this encoding device 20 can be housed in a space-saving manner in any inner and / or outer sole. This small volume limits the impact on user comfort and has the advantage of optimizing production costs by making the integration of this technology into the sole during the industrial process less expensive and simpler.
[0189] An encoding device 20 according to the invention is detailed in [Fig. 5]. The various components of the encoding device 20 are preferably arranged on an electronic card 27 (or printed circuit), but the invention may provide various types of arrangement, for example a single module combining all of the functions described here. Similarly, these means can be divided into several electronic cards or gathered on a single electronic card. In addition, when an action is attributed to a device, a means or a module, this is in fact generally carried out by a microprocessor of the device or module controlled by instruction codes stored in a memory. Similarly, if an action is attributed to an application, this is in fact carried out by a microprocessor of the device in a memory of which the instruction codes corresponding to the application are stored. When a device or module transmits or receives a message, this message is transmitted or received by a communication interface.
[0190] Such an encoding device 20 according to the invention may comprise a motion sensor 21 such as an inertial platform configured to generate data on the gait of a user. During the use of the encoding device 20 by a user (the encoding device is advantageously positioned on a user) the inertial platform acquires signals representative of a movement parameter (acceleration and / or speed, for example angular speed) of the user, preferably of the user's foot, along the X, Y, Z axes. In addition, this data may then be processed to generate at least one acceleration signal. The inertial platform is for example made up of at least one accelerometer and one gyroscope. Preferably, it comprises several accelerometers and gyroscopes.The encoding device 20 may also comprise one or more magnetometers so as to acquire three additional raw signals corresponding to the magnetic field values in three dimensions.
[0191] Each encoding device 20 may also include other sensors, in particular an inclinometer, a barometer, a temperature sensor and an altimeter to benefit from increased precision.
[0192] Preferably, an encoding device 20 comprises one or more processors 22 configured to execute one of the methods according to the invention, preferably the encoding method, and their different embodiments, preferred, advantageous or not.
[0193] In particular, the one or more processors 22 are configured to acquire data generated by the motion sensor(s) 21, said data comprising acceleration and angular velocity values.
[0194] The one or more processors 22 may also be configured to calculate a plurality of descriptors from the data acquired over a time window 31, the calculation step being repeated for several consecutive time windows 31, the consecutive time windows 31 forming a time sequence 32;
[0195] Further, the one or more processors 22 may also be configured to transform the plurality of calculated descriptors, said step comprising encoding the descriptors each on at most two bits, said bits associated with a time window 31 being representative of a movement phoneme.
[0196] Also, the one or more processors 22 may be configured to generate a data tensor comprising the bits obtained for each of the time windows 31, such that the data tensor comprises an encoding of a combination of several motion phonemes that is representative of a posture or movement of the user.
[0197] Preferably, the one or more processors 22 will be configured so as to:
[0198] - acquire data generated by the motion sensor(s) 21, - calculate a plurality of descriptors from the data acquired on a time window 31, the calculation step being repeated for several consecutive time windows 31; and - transforming the plurality of calculated descriptors, said step comprising a coding of the descriptors each on at most two bits, said bits associated with a time window 31 being representative of a movement phoneme
[0199] Furthermore, the one or more processors 22 will be configured so as to generate a data tensor comprising the bits obtained for each of the time windows 31, so that the data tensor comprises an encoding, preferably of a combination of several movement phonemes, which is representative of a posture or a movement of the user.
[0200] In particular, the one or more processors 22 may be configured to perform the following actions:
[0201] - Loading data coded on at most eight bits associated with the time sequence 32, being representative of a position or a movement of the shoe on said time sequence 32, for example in the form of one or more data tensors; and - Process the data tensor(s) by a trained machine learning model to identify a user posture or movement.
[0202] Alternatively, the processor(s) may be configured to generate the data tensor and then send it to an external terminal 30 to determine a user's gait.
[0203] Thus, the encoding device 20 according to the invention advantageously comprises one or more communication means 25. The one or more communication means 25 are capable of receiving and transmitting the data over at least one communication network. Preferably, the communication is carried out via a wireless protocol such as Wi-Fi, 3G, 4G, 5G and / or Bluetooth. Preferably, the communication protocol is a BLE or ANT+ protocol. These communication protocols allow low energy consumption.
[0204] Preferably, the encoding device 20 according to the invention comprises a first communication means 25 configured so that the encoding device 20 is capable of transmitting at least a portion of the data tensor to an external terminal 30. This data can be transmitted in real time or delayed to an external terminal 30. The external terminal 30 can for example be a remote system such as a tablet, a mobile phone (“smartphone” in English terminology), a computer or a server. As mentioned, the encoding device 20 according to the invention can also implement a step of determining the gait of a user; thus, it can be configured to send posture or movement data identified for a user. This data can be transmitted in real time or delayed to an external terminal 30.
[0205] Advantageously, an encoding device 20 according to the invention further comprises a second communication means configured so that a first encoding device 20 is able to communicate with a second encoding device 20.
[0206] In particular, the encoding devices 20 may be configured to communicate with each other and to initiate the generation of data on the movement of a user only after receiving a message from the other encoding device 20. This contributes to the synchronization of the encoding devices 20. For example, the encoding devices 20 may be configured to associate and / or match timestamps of the samples from the inertial platforms.
[0207] Advantageously, if an encoding device 20 were to become disconnected or lose synchronization over time with respect to the other encoding device 20, a method according to the invention could comprise a step of synchronizing the encoding devices 20. Thus, a search signal is sent by the connected encoding device 20, the disconnected encoding device 20 receives the search signal and synchronizes with the connected encoding device 20.
[0208] The encoding device 20 according to the invention also comprises a data memory 23, configured to store at least a portion of the acquired data and / or data tensors. The encoding device 20 according to the invention is such that it can operate with a low-capacity data memory 23. It can be configured to store the raw data generated by the inertial platform. Advantageously, the data memory 23 is configured to store at least a portion of the transformed data, but not to store the generated raw data. Thus, its capacity is not burdened by generated raw data. The transformed data can correspond to data preprocessed by the processor(s) or to data tensors.
[0209] Furthermore, the encoding device 20 according to the invention may comprise a source 26. The energy source is preferably of the battery type, rechargeable or not. Preferably the energy source is a rechargeable battery. In addition, it can be associated with a system for recharging by movement or by external energy. The system for recharging by external energy can in particular be a system for recharging by wired connection, an induction or even photovoltaic recharging system.
[0210] The energy source 26 is preferably of the battery type, rechargeable or not. Preferably the energy source is a rechargeable battery. Recharging can be carried out using different technologies such as:
[0211] - by charger, with a connector flush with the sole; - with a mechanical recharging device integrated into the sole, for example a piezoelectric device capable of providing electrical energy from walking; - with a contactless device, for example by induction; and / or - with a photovoltaic device.
[0212] Furthermore, the encoding device according to the invention may comprise a wired connection port 28, preferably protected by a removable tab. This wired connection port may be, for example, a USB or firewire port. Advantageously, the USB port is also resistant to water or humidity. Furthermore, the USB port is advantageously surmounted by a polymer beam to give it greater resistance in use conditions. This wired connection port 28 may be used as mentioned above to recharge the battery, but also to exchange data and, for example, to update the firmware of the electronic card carrying the various components of the encoding device.
[0213] Preferably, the removable tab or USB cover protects the USB port from foreign bodies. For example, the removable tab protects the USB port from water or dust. Such a tab may preferably be made of an elastomer or polyurethane polymer.
[0214] Furthermore, according to another aspect, the present invention relates to a system 5 for determining the gait of a user.
[0215] Such a system 5 for determining a user's gait, described in connection with [Fig. 6], will advantageously comprise one or more encoding devices 20 according to the present invention. In particular, the system 5 for determining a user's gait according to the invention comprises at least two encoding devices 20 according to the present invention.
[0216] In particular, each of the encoding devices 20 is designed so as to be able to communicate independently with the other and / or directly with an external terminal 30 in order to be able to exchange its own information on the movement posture / the activity of the user and in particular of his foot.
[0217] In particular, the system 5 according to the invention may comprise a pair of soles 11, 12 each comprising an encoding device 20 according to the invention. The soles 11, 12 usable in the context of the system 5 according to the invention may, for example, correspond to outer soles or inner soles of shoes. These soles may be removable or be permanently integrated into the sole assembly of the shoes. Conventionally, the soles 11, 12 making up said pair of soles each comprise an encoding device 20 according to the invention. As shown in [Fig. 6], the encoding device 20 is preferably positioned at a median portion of the sole.
[0218] As mentioned, the system 5 according to the invention may comprise an external terminal 30 configured to receive data from the encoding device(s) 20.
[0219] Advantageously, a dedicated application is installed on this external terminal in order to process the information transmitted by the encoding device(s) 20 and allow the user to interact with the invention. It is then, for example, possible to access this remote server via a web interface. All communications with the remote server can be secured, for example, by HTTPS protocols (for “HyperText Transfer Protocol Secure” in English terminology) and AES encryption (for “Advanced Encryption Standard” in English terminology) 512. Thus, this can allow, via a client, access to the data by medical personnel responsible for monitoring the user.
[0220] The external terminal 30 is generally a tablet, a mobile phone (“smartphone” in English terminology), a gateway, a router, a computer or a server. It may be able to transfer this data to a remote server. It is then, for example, possible to access this remote server via a web interface. It may, for example, be configured to implement inference solutions specific to given applications from coded data, or from coded data tensors or in particular from order data tensors generated by the encoding device 20. It may also be configured to generate data on the gait from data resulting from the step 700 of determining the user's gait.
[0221] Thus, the user can access data related to his gait such as data relating to his daily physical activities and relating to several biomechanical parameters, such as posture, pronation / supination, impact force, step length, contact time, lameness, balance and several other parameters relating to the user and describing his movements, his gait, his postures and his movements and thus to follow their evolution.
Claims
Claims
1. Method (1) for encoding data generated by one or more motion sensors (21) coupled to a shoe, for identifying a position or movement of the shoe over a time sequence (32), said method (1), executed by one or more processors (22), comprising: - A step of acquiring (200) data generated by the motion sensor(s) (21) coupled to the shoe, said data comprising variable values in the form of time series, preferably said variable values comprising acceleration and angular velocity values; - A step of calculating (300), from the data acquired over a time window (31), a plurality of descriptors, the calculating step being repeated for several consecutive time windows (31), consecutive time windows (31) forming the time sequence (32);and - A transformation step (400) of the plurality of descriptors of each of the consecutive time windows (31) forming the time sequence (32), said transformation step (400) comprising a generation for each of the descriptors of data coded on at most eight bits, said data coded on at most eight bits associated with the time sequence (32) being representative of a position or a movement of the shoe on said time sequence (32).;
2. Method (1) for encoding data according to claim 1, characterized in that it further comprises a step of generating (500) a data tensor (34) comprising the data coded on at most eight bits obtained for several of the time windows (31), so that the data tensor (34) comprises an encoding representative of a position or a movement of the shoe.
3. Method (1) for encoding data according to any one of claims 1 or 2, characterized in that the descriptors are calculated at from mathematical functions applied to the data, pre-processed or not, generated by the motion sensor(s) (21) coupled to the shoe, and said applied mathematical functions comprising for example a calculation of average, statistical asymmetry, flattening coefficient, comparisons of values of the extremums or quartiles, positions of the extremums in a time series, comparisons of the edge values of the subsets of the motion and / or a calculation of a topology of the data.
4. Method (1) for encoding data according to any one of claims 1 to 3, characterized in that the time windows (31) of two consecutive time sequences (32) have different durations.
5. Method (1) for encoding data according to any one of claims 1 to 4, characterized in that, during the transformation step (400), at least four descriptors are used to each generate data coded on at least one bit, more preferably at least two bits.
6. Method (1) for encoding data according to any one of claims 1 to 5, characterized in that, during the transformation step (400), between four and twenty descriptors are used, each generating data coded on a number of bits selected from: one, two or three.
7. Method (1) for encoding data according to any one of claims 2 to 6, characterized in that it comprises a step of analyzing the data tensor as a function of user activity data, said analysis step comprising a comparison between the data tensor and a database comprising reference data for the given activity.
8. Method (1) for encoding data according to any one of claims 1 to 7, characterized in that, during the transformation step (400), one or more data coded on at most eight bits, preferably at most three bits, are calculated from values of composite descriptors calculated from data acquired over consecutive time windows, said composite descriptors being formed from several descriptors coming from different time windows.
9. Method for determining (2) the gait of a user of a shoe, from data likely to have been encoded according to an encoding method according to any one of claims 2 to 8, said determination method (2), executed by one or more processors, comprising: - A step of loading (710) one or more data tensors comprising an encoding representative of a position or a movement of the user's shoe, said encoding comprising descriptors calculated from data generated by one or more motion sensors coupled to the shoe, each of the descriptors being coded on at most eight bits; and - A step of processing (720) the data tensor(s) by a machine learning model trained so as to identify a posture or a movement of the user.
10. Method for determining (2) the gait of a user according to claim 9, characterized in that the step of processing (720) the data tensor(s) by a machine learning model comprises the identification of event classes from the data tensor(s), preferably a movement class per time sequence (32).
11. Method for determining (2) the gait of a user according to any one of claims 9 or 10, characterized in that it comprises a step of selecting (730), preferably by the trained machine learning model, a posture or movement of the user from a comparison of the postures or movements of the user identified from contiguous time windows (31) or contiguous time sequences (32).
12. Method for determining (2) the gait of a user according to any one of claims 10 or 11, characterized in that it comprises a step of comparing (740) postures and / or movements identified for a given time window (31) or time sequence (32) from data from different sensors so as to identify a posture or movement of the user.
13. Device (20) for encoding data generated by one or more motion sensors (21) coupled to a shoe, for identifying a position or movement of the shoe over a time sequence (32), said encoding device (20) comprising said one or more motion sensors (21) and one or more processors (22), said one or more processors (22) being configured to: - Acquire the data generated by the one or more motion sensors (21) coupled to the shoe, said data comprising variable values in the form of time series, preferably said variable values comprise acceleration and angular velocity values; - Calculate, from the data acquired over a time window (31), a plurality of descriptors, the calculation being repeated for several consecutive time windows (31), the consecutive time windows (31) forming the time sequence (32);- Transforming the plurality of descriptors of each of the consecutive time windows (31) forming the time sequence (32), said transformation comprising a generation for each of the descriptors of data coded on at most eight bits, said data coded on at most eight bits associated with the time sequence (32) being representative of a position or a movement of the shoe on said time sequence (32).;
14. Device (20) for encoding data generated by one or more motion sensors (21) according to the preceding claim, characterized in that said one or more processors (22) are configured to generate a data tensor (34) comprising the data coded on at most eight bits obtained for several of the time windows (31), so that the data tensor (34) comprises an encoding representative of a position or a movement of the shoe.
15. System (5) for determining the gait of a user, said determination system comprising at least one encoding device (20) according to claim 14 and an external terminal (30) configured to receive the generated data tensor and then to determine the gait of a user from the generated data tensor.
