System and method for counting, recognition, and evaluation in real time of repeated biomechanical activities from human and robotic limbs

Abstract

The present invention relates to a system and method capable of counting, classifying, and evaluating repeated biomechanical activities in real time. The basic configuration of the system consists of one or more devices, wearable and / or embedded, each consisting of a microcontroller, an IMU, and an LED, capable of processing inertial data, providing feedback on counting, and sending it to the computing node that classifies and evaluates motion through machine learning algorithms and data fusion techniques. The cloud service enables optimal management of training datasets and user profile information. The graphical user interface allows the user to interact with the system by accessing evaluation histories and customizing it based on their own goals. The method ology presented involves a central use of mathematical modelling, ensuring specialized biomechanical assessments and the construction of robust training datasets. This invention finds further application as monitoring aimed at controlling robotic limbs.

Description

[0001] SYSTEM AND METHOD FOR COUNTING, RECOGNITION, AND EVALUATION IN REAL TIME OF REPEATED BIOMECHANICAL ACTIVITIES FROM HUMAN AND ROBOTIC LIMBS

[0002] 1 . The field of the art

[0003] [1] The present invention relies to the field of monitoring systems and methods for counting, automatically recognizing and evaluating in real time biomechanical activities repeated by human limbs and further usable in monitoring for the control of robotic and industrial systems, using wearable devices and / or integrated into generalized intelligent instrumentation to multiple loT application contexts.

[0004] [2] From a physical point of view, the term "limb" generally refers to a lever system formed by rigid bodies provided with a certain number of degrees of freedom.

[0005] [3] The term “monitoring” refers to a systematic collection procedure, analysing and using information to track the progress of an activity toward achieving its objectives and to guide future management decisions. The objective of the monitoring is to collect information and data to return feedback of various types, acoustic-sound, visual-optical, mechanical and / or informative.

[0006] [4] In the robotic field, the term "control" refers to a procedure for collecting, analysing and using information to guide the mechanical action of a robotic limb toward its correct execution using actuators.

[0007] [5] Wearable devices are intelligent devices that can be worn and connected to central data reception and sorting nodes such as smartphones or other specially designed devices, through wireless communication technologies, allowing not only detection but also storage and exchange of data of different types, immediately and often without the need for human intervention.

[0008] [6] Specialized intelligent instrumentation is equipment that contains electronic devices that can be connected to central data reception and sorting nodes such as smartphones or other specially designed devices, through wireless communication technologies, allowing not only to detect but also to store and exchange data of different types, immediately and often without the need for human intervention. [7] The term loT, or the Internet of things, refers to the collective network of connected devices and technologies that facilitates communication between devices and the cloud, as well as between the devices themselves.

[0009] [8] The present invention provides for the use of hardware devices that can be integrated in the loT context for the intelligent collection of data, used to conduct analyses and elaborations aimed at biomechanical studies and monitoring of the performance relative to the repeated movement of human and / or robotic limbs dedicated to different application contexts.

[0010] [9] An loT system operates through real-time data collection and exchange and it typically consists of three components: Smart devices, loT applications and a graphical interface.

[0011]

[0010] An intelligent device is a computing device. The device collects data from its environment, including user input and usage modes, communicating such information over the Internet and to its loT application.

[0012]

[0011] An loT application is a collection of services and software that integrates data received from various loT devices. Through the use of machine learning or artificial intelligence (IA), it analyses this data by making decisions on its own and providing informative outputs useful for carrying out subsequent analysis with relevant views. The result of the processing is, on the one hand, retransmitted to the loT device, which responds to the inputs intelligently, on the other hand, sent to the graphical interface typically using cloud services.

[0013]

[0012] The user can manage one or more loT devices through the graphical interface. Common examples include a mobile application or website that can be used to manage smart devices and view results.

[0014]

[0013] An embedded system (literally "embedded" or "embedded") or integrated system generically identifies all those electronic microprocessor processing systems designed specifically for a particular purpose, or not re-programmable by the user for other purposes, often with an ad hoc hardware platform, capable of managing all or part of the required functionality, receiving and processing data from sensors and providing feedback of different types, for example through the use of LEDs.

[0014] A microcontroller is an electronic device integrated on a single electronic circuit, born as an alternative evolution to the microprocessor and generally used in embedded systems or for specific applications of digital control.

[0015]

[0015] The sensors are components that detect and measure inertial data, such as accelerations and angular velocities, magnetic fields, pressures, temperatures, biometric parameters and others. This data is sent to the microcontroller for processing.

[0016]

[0016] LEDs are lights that are used to provide visual feedback, such as indicating device status, alerting, or displaying measurement results.

[0017]

[0017] The present invention provides for the use of antennas for wireless communications which allow a device to send and receive data via radio frequency, enabling connection to wireless networks such as Wi-Fi, Bluetooth, NB-loT, RFID or Zigbee for transmitting collected data or receiving commands such communication types are typically employed in loT architectures.

[0018]

[0018] Wi-Fi antennas are used to connect devices to local networks and the Internet. The most common frequencies are 2.4 GHz and 5 GHz. These antennas are often integrated into wearable and embedded smart devices.

[0019]

[0019] Bluetooth and Bluetooth Low Energy (BLE) antennas are essential for short range communication between loT devices, such as wearable devices, smartphones and, in general, central reception nodes. BLE antennas are particularly appreciated for their low power consumption.

[0020]

[0020] Zigbee antennas, operating in the 2.4 GHz ISM band, are typically used in mesh networks for loT applications.

[0021]

[0021] NB-loT (Narrowband loT) antennas support low-power, low-cost cellular communications for loT devices, using existing mobile networks. They are suitable for applications such as industrial monitoring and smart meters.

[0022]

[0022] Radio-Frequency Identification (RFID) antennas are used for object identification, association and tracking, often in logistics. They operate at various frequencies, with the most common at 125 kHz, 13.56 MHz, and 900 MHz.

[0023]

[0023] The present invention provides for the use of specialized neural networks for recognition and classification.

[0024] Neural networks play a crucial role in Internet of things (loT) applications because of their ability to analyse, interpret and learn from data collected from connected devices. Among the various uses are data processing and analysis, learning and prediction, automation and intelligent control, recognition and classification, performance optimization, customization and adaptation, integration and interoperability as well as the detection of anomalies and safety.

[0024]

[0025] Neural networks amplify the potential of loT applications, enabling advanced data processing, improving automation, and making loT systems more intelligent, adaptable, and secure.

[0025]

[0026] Examples of artificial intelligence (IA) models mainly include neural network models, interference models, neural models, classifiers, decision tree models, neural network spiking models (SNN), convolutional neural network models (CNN), recurrent neural network (RNN), deep neural network models (DNN), generative network models.

[0026]

[0027] Examples of machine learning algorithms may include a support vector machine (SVM), a naive Bayes classification, a decision tree, supervised learning algorithm, or unsupervised learning algorithms for classification and regression. Machine learning algorithms can be trained using one or more training datasets.

[0027]

[0028] Edge computing is a key enabler for the effective use of neural networks in the loT, offering benefits such as reduced latency, scalability, increased privacy, bandwidth management efficiency, and operational resilience. It enables distributed Al solutions to be implemented directly on devices, opening the way for a wide range of smart and standalone applications.

[0028]

[0029] Cloud computing significantly supports neural network integration in loT contexts, providing the computing power needed, scalable resources, centralized management tools, and the ability to perform real-time analytics. It provides a flexible platform for processing and storing large amounts of data generated by loT devices, optimizing the life cycle of neural networks, from training to inference.

[0029]

[0030] Embedded neural networks represent a growing frontier in the field of artificial intelligence, offering the possibility of performing local inference on devices with limited resources, such as wearable intelligent devices or integrated into instrumentation. While there are challenges related to optimization and hardware constraints, these networks are enabling a wide range of intelligent applications, improving energy efficiency, privacy, and operational capabilities in real time.

[0030]

[0031] The present invention involves the use of a cloud service which allows the user to store data and access computer systems via mobile or web applications. By using it, companies and individuals with a Web site no longer need physical servers to store and access data. Cloud computing services typically include databases, software, networks, analytics, and other capabilities available through this type of infrastructure.

[0031]

[0032] Cloud users can browse files, programs, and data anywhere, without having to save information on dedicated hard disks or servers. Thanks to this service the user data are accessible remotely in the data centres, which transmit the various resources via the Internet eliminating the need to realize it infrastructures on their own hardware.

[0032] 1.1 State of the art

[0033]

[0033] The scope of application embraced by the system and / or method of the invention is rather vast even though it is well outlined within sports and rehabilitation contexts for the real-time monitoring of the performance and training status of the user. The interest toward this type of system is possible thanks to the technological growth and arises from the request for intelligent instrumentation interacting with the individual.

[0034]

[0034] One of the areas where technologies have had a more obvious impact is the monitoring of the performance of athletes. Wearable devices, such as fitness watches and motion sensors, allow athletes to collect detailed performance data during training and competitions. This data, such as heart rate, speed, power and more, is analysed to optimize training and prevent injury.

[0035]

[0035] In the field of fitness, wearable electronic devices, also called wearable devices, must be synchronized with the smartphone by means of a wireless connection, so as to be able to perform a wide range of functions, from simple notifications to a more specific use related to fitness or in rehabilitation. Other types of fitness systems reside in intelligent instrumentation, i.e. machines equipped with calculation power, sensors and actuators, capable of returning feedback to the user in real time.

[0036]

[0036] Among the most common wearable electronic devices in the world of fitness we find smart-watches, i.e. hi-tech watches with hundreds of features, smart bands or bracelets that communicate with the user through screens and / or light and / or vibration pulses, fitness trackers, these are specifically designed to monitor physical parameters during the workout, smart rings designed as authentication devices or for NFC technology.

[0037]

[0037] In the context of sports performance, machine learning algorithms use the computing power of computers to return feedback and process specialized performance metrics to monitor the performance of the athletes in an optimized and autonomous manner and always improved by collecting large amounts of data from suitable devices.

[0038]

[0038] Machine learning techniques are subsets of artificial intelligence in which algorithms are used to learn from data and from errors autonomously, without requiring explicit and precise instructions from users.

[0039]

[0039] The continuous availability of updated data is essential for analytical models to continue to provide adequate responses over time. Much of the success of machine learning is found in the continuous and automatic retraining to keep the analytical performance of predictive models high.

[0040]

[0040] In sports such as skiing, tennis, basketball and any other sport, not only are films analysed, but training is also supported by specific software and instruments that measure effort, movements, calories burned and other metrics, helping athletes improve their performance.

[0041]

[0041] The future of technologies in sport is promising. Artificial intelligence, combined with the growing development of intelligent instrumentation integrated into loT contexts, enables the creation of customizable services and functional areas useful for monitoring the performance of athletes, ensuring useful tools for coaches and personal trainers.

[0042]

[0042] Data collection, possible thanks to emerging technologies, is also useful for improving the viewer experience. Sensors in sports fields and equipment, along with real-time data analysis, provide detailed statistics during events. This information can be displayed in real time on the screen, enriching the audience's understanding and enthusiasm.

[0043]

[0043] The technology has introduced numerous innovations in the field of rehabilitation, significantly improving the results for patients and facilitating the work of therapists. The most important technological instruments in this field include devices for monitoring the health status of patients designed to help them recover their physical and motor functionality in the best possible way.

[0044]

[0044] Examples of applications in the rehabilitation field are rehabilitation robotics, in which the main interest is in the development of exoskeletons and robots for the rehabilitation of upper limbs, balance and force platforms, biofeedback devices, that is, EMG systems and / or motion sensors capable of returning feedback on the physiological activity of skeletal muscle groups and information related to the dynamics of movement, paying particular attention to the constraining reactions that could damage the patient, worsening the clinical situation.

[0045]

[0045] The Internet of things (loT) systems in fitness have created numerous patents, covering a wide range of technologies used to monitor, analyse and improve physical performance. These patents cover wearable devices, intelligent sensors, connected training platforms, and data analysis software.

[0046]

[0046] The patent application W02024064703A1 relates to a system and method capable of counting repetitions and classifying a movement using machine learning algorithms based on video data.

[0047]

[0047] The patent application US20230075058A1 relates to a mobile application capable of being associated with intelligent devices and communicating with it to process information and provide feedback to the user.

[0048]

[0048] The patent application US20220406453A1 relates to a system and method capable of detecting the level of fatigue by analysing the electrocardiographic data integrated in wearable devices.

[0049]

[0049] The patent application US20230343450A1 describes a system and method for creating custom exercise programs using optical sensors to capture user athletic gestures. The data can be processed to provide metrics regarding the execution of the exercise.

[0050]

[0050] The patent application US11051720B2 relates to a system capable of collecting data relating to human movement using fitness tracking devices to be worn on the arms of the user and of detecting when the arm is constrained by its action, estimate the position and orientation of the hand.

[0051]

[0051] The patent application US20210220702A1 relates to systems and methods for formulating performance metrics in the field of professional swimming. This tool is a wearable device capable of processing user activity assessments by automatically and adaptively calculating parameters such as applied force and direction.

[0052]

[0052] The patent application WO2023136848A1 relates to a method which detects muscle fatigue by analysing the time series of data coming from sensors, using them to provide predictions on subsequent repetitions. The method also allows to recognize incomplete repetitions, yielding, muscular fatigue and series not finished, updating on the basis of the user's performance.

[0053]

[0053] The patent application WO2022256652A1 relates to a system and method capable of improving the user experience being trained, offering an interactive experience and fun game challenges thanks to the dedicated platform.

[0054]

[0054] The patent application EP3287871A1 refers to a method to be used on wearables to provide feedback including detection of particular movements. The system converts the gestures into action instructions for associated peripheral devices.

[0055]

[0055] Another field of interest for the present invention relates to monitoring systems for controlling mechanical actions performed by industrial machinery and robotic systems.

[0056]

[0056] The state of the art in control systems for robotics and industrial processes is based on a combination of advanced predictive control techniques, artificial intelligence, loT and flexible automation. Technologies such as Model Predictive Control, collaborative robots, and adaptive control are at the heart of innovations that improve the accuracy, efficiency of work and consumption, and resilience of industrial and robotic systems. These developments are driven by the needs of Industry 4.0, with an increasing emphasis on flexibility, advanced automation and real-time analysis.

[0057]

[0057] Model Predictive Control (MPC) is one of the most advanced control techniques used in robotics and industrial processes. The MPC is based on a mathematical model of the system to predict future behaviour and optimize real-time control. Recent developments in the MPC include integration with machine learning algorithms to improve prediction and adaptation to nonlinear or complex conditions, extending their effectiveness in highly dynamic scenarios.

[0058]

[0058] Adaptive control systems can modify their control parameters in response to changes in the system or operating environment. These approaches are particularly useful in mobile robotic systems and collaborative robots (cobots), which must continuously adapt to new scenarios and tasks.

[0059]

[0059] Decentralized control systems are an emerging trend in industrial automation, especially in distributed facilities and large infrastructure. This type of architecture is ideal for the management of smart factories and modular production systems within industry 4.0.

[0060]

[0060] The integration between loT and industrial control systems allows to connect machines, sensors and actuators within a connected and intelligent network. Industrial loT (HoT) is a key component of smart factories, with distributed control capabilities to improve process management and event response while ensuring optimization of effort and consumption.

[0061]

[0061] The patent application US20240181635A1 relates to a computer- implementable method capable of determining the movement of a robot in relation to external loads, using information from different sensors. The system determines a set of parameters that allow to give instructions on the movements of the robot to respect the desired trajectory.

[0062]

[0062] The patent application US1 1554482B2 relates to a method using simulations to train an artificial intelligence model to optimize control strategies in the industrial field.

[0063]

[0063] The patent application US2022379468A1 relates to a control system and method for a robotic arm configured to be maintained in a static position when only the force of gravity acts thereon and to enable adaptive modification of the position of the robotic arm when an external force other than gravity is applied.

[0064] 2. Summary of the invention

[0065]

[0064] The problem faced by this invention is to provide a system and method which allows counting, automatic classification and personalized evaluation of a biomechanical movement repeated by limbs, i.e. systems which, on the one hand, process inertial data from IMU to give real-time feedback to the user, on the other hand, support the completion of personal objectives through personalized evaluations based on the user's information and objectives.

[0066]

[0065] The present invention solves the above problem by the system and / or method of the invention, as well as by the use of such a system and / or method through the use of computer systems; it can be implemented as a wearable device and / or integrated into dedicated instrumentation, capable of recording acceleration and gyroscope data to understand in real time the biomechanical movement repeated by the user by the use of artificial intelligence techniques. At the end of the exercise, the system sends the output to a data collection and management node, thus making it viewable with a web-based or app-cloud user interface.

[0067]

[0066] The invention therefore allows to have an absolutely versatile system and / or method which can be used for counting, classifying and evaluating repeated biomechanical movements, based on real-time signal processing techniques. automatic classification of movement through machine learning algorithms and biomechanical modelling capable of giving a quantitative evaluation on the quality of the gesture and the performance of the user.

[0068]

[0067] The present invention consists of a wearable or embedded device capable of providing real-time counting and recognition, possibly by using edge architectures which provide an external working environment with great computational power capable of processing the data with artificial intelligence techniques. Motion evaluation is performed by data post processing and fusion algorithms which further process the information collected over time at the end of the repetitions.

[0068] The system and / or method of the present invention allows to manage and associate different intelligent devices, through specific hardware architectures and software programming, providing a personalized service to the user for monitoring their performance and / or rehabilitations, aimed at the biomechanical study of the movement of the human body, and in particular one or more limbs, to count, recognize and evaluate the activity performed by the user by providing performance metrics viewable through a GUI.

[0069]

[0069] Further features and advantages of the system and / or method of the invention and its applications will be apparent from the description and the embodiments of the invention itself provided as an indication thereof.

[0070]

[0070] Unlike traditional approaches, the system and / or method of the invention aims to allow complete freedom in biomechanical gesture, not binding the user to the use of optical sensors, to remain in a given area or to specify activity with an input, using inertial data from an IMU as the main processing signals. The use of parametric biomechanical modelling makes it possible to generate a large amount of data that are used for training the neural network (Data Augmentation). This methodology combined with the use of cloud services allows to build structured databases to efficiently and reliably train machine learning algorithms.

[0071]

[0071] Unlike traditional approaches, the system and / or method of the invention uses parametric biomechanical modelling to convert the kinematics of the object into dynamics for the human body. For example, by increasing the weight to be lifted the force required by the user and the constraining reactions on the joints will increase consistently according to the physical model.

[0072]

[0072] Unlike traditional approaches, the system and / or method of the invention uses parametric biomechanical modelling to manage the complexity of the signals, variable both in time and in user-to-user space, and the development of different configurations, by combining multiple equal peripherals or with other integrated sensors to be installed at certain user locations, increasing the flow of information and making the system architecture more complex. The modelling therefore allows to create an artificial data flow to develop data fusion algo- rithms and to renew the architecture of the recognition node and in the evaluation node.

[0073] 3. Detailed description of the invention

[0074]

[0073] For the purposes of the description of this document the term "and / or", if used in a list of two or more articles, means that any one of the listed articles can be used alone or in any combination of two or more of the listed articles.

[0075]

[0074] For example, if a combination is described as containing components A, B and / or C or A and / or B and / or C, the composition may contain only A, only B, only C, A and B in combination, A and C in combination, B and C in combination or A, B and C in combination.

[0076]

[0075] The terms "comprise", "comprising" or any other variation of them are intended to cover non-exclusive inclusion such that a system, a method, a use, etc. which include a list of items, not only include those items but may include others not expressly listed or inherent in such a system, method, use, etc.

[0077]

[0076] An element followed by "includes..." it does not prevent, without further constraints, the existence of further identical elements in the system, method or use which comprises the element.

[0078]

[0077] It Is known that artificial intelligence applications provide for the execution, by digital electronic processing devices, of suitable algorithms designed to process data and make decisions associated with the behaviour and intelligence typical of the human being.

[0079]

[0078] Such processing and therefore the artificial intelligence algorithms are based on the acquisition of data from transducers and / or sensors and / or other acquisition devices. The data are the representation of the characteristics of a physical phenomenon that is observed, based on which the artificial intelligence algorithms are called to make decisions and / or to take certain actions and / or to cause state changes in the system, for example by means of commands.

[0080]

[0079] An artificial intelligence application is typically performed by a specific electronic device, which is configured as a different logical entity from the data acquisition device representative of the physical phenomenon.

[0080] The invention relates to a system and / or method based on signal processing and artificial intelligence techniques for performing counting, recognizing and evaluating the quality of the exercise in real time.

[0081]

[0081] The invention relates to a system based on Internet of things (loT) architectures that allows constant monitoring representing a continuous service for the device, machine and / or user.

[0082]

[0082] The system and / or method allows the combination of multiple electronic units that process and send data to the processing node to obtain advanced and specialized functionalities and / or metrics based on the configuration used and the application context.

[0083]

[0083] The invention is based on the reception of data processed from inertial signals comprising accelerations and angular velocities coming from an Inertial Measurement Unit (IMU). Such data are processed by a calculation node, internal and / or external to the peripheral itself, on which artificial intelligence techniques are based, able to receive and process the input in order to classify by means of a label the type of biomechanical movement performed by the user and / or machine. During the training phase, it is possible to collect signals, acquired and simulated by biomechanical modelling, which are coded and labelled in order to catalogue them in an incremental database. This will serve as a basis for Al algorithms to increase the ability to identify and manage the data itself through ongoing training in the cloud. At the same time, an additional database will have the purpose of storing parameters and weights of the architectures in use.

[0084]

[0084] The electronic unit in its basic configuration is a wearable and / or integrated device in instrumentation capable of reading the inertial data comprising accelerations and angular velocities coming from IMU in order to segment and process the signal to transmit an encoded output to the computing node for performing the various functionalities.

[0085]

[0085] The electronic unit guarantees the possibility of managing data coming from further hardware components for the measurement of additional physical information which can be encoded together with the inertial data in order to increase the level of information.

[0086] Depending on the complexity of the composition of the device and / or their combination, it is possible to design a system that is scalable and adaptable according to the scope of application and the degree of detail.

[0086]

[0087] The invention consists of a calculation node, internal and / or external to the peripheral itself, on which artificial intelligence techniques run to receive and process the input in order to classify by a label the type of biomechanical movement performed by the user and / or machine. In addition, the calculation node contains data fusion and post-processing techniques which allow to obtain performance metrics.

[0087]

[0088] The neural network training method provides for the use of complete and effective datasets obtained through the processing and coding of real signals acquired by the devices and of synthetic signals generated by means of advanced mathematical models. This approach guarantees the availability of a large amount of reliable data to increase the solidity and consistency of the training, since the latter often involves criticality of generalization in many fields of application and existing technologies, caused by the lack and insufficiency of data.

[0088]

[0089] The invention provides a function of acquisition and storage of signals relating to the movements and / or exercises carried out by the user due to the loT architecture that allows the sending of data to the cloud service in order to be saved on suitable storage spaces.

[0089]

[0090] The invention comprises the use of a parametric physical-mathematical model which describes the biomechanics of a limb and generates simulated signals from IMU and / or other sensors. The use of the model also allows to give a consistency in the biomechanical evaluations of the recognized movements, starting from the nominal trends and from the user and / or machine information.

[0090]

[0091] The invention relates to a system and / or method provided with a parametric physical-mathematical model which, according to the invented method, is used to generate synthetic data so as to train neural networks, to provide the user with physically coherent and personalized feedback (performance, quality) calculated ad-hoc through modelling parameters set by the user (height, weight, etc.).

[0092] The possibility of using models to solve criticalities of previous inventions such as network training and method scalability is a fundamental characteristic element, both of the system and of the methodology object of the invention. The model allows to simulate synthetic data to generate a database and therefore allows to have reference values, including possible execution defects (errors and uncertainties), for the biomechanical movements that it is intended to include in the application.

[0091]

[0093] A further application of the invention is the state machine present in the electronic device. This solution allows the real-time counting of repeated movements, the automatic recognition of the initial positioning, the detection of the rise / fall times, the coding and the sending of the data to the calculation node. This software mechanism allows the electronic device to manage the complete flow of execution of the functionalities, starting from the association with the user until it reaches the encoding and sending of the data to the calculation node.

[0092]

[0094] This control solution carried out by the state machine solves the need to provide manual inputs for selecting and carrying out movements and it is able to adapt in real time to the gesture being made. This mechanism also allows to realize a self-contained system that does not require manual input from the user.

[0093]

[0095] The invention includes a graphical interface in communication with the cloud, accessible via mobile device and / or computer, which allows the user to view the history of his activities. A reserved area will be provided in which to generate a user profile useful for personalizing the service offered to the user.

[0094]

[0096] A further functionality of the invention is the customization of the system based on the biometric and user information. The inputs are required during profile configuration (user height, user weight, etc.), and / or during subsequent offline customization sessions (creation of goals, training plans, etc.), favouring an immersive experience and supporting the achievements of personal goals.

[0095]

[0097] The user can visualise through the graphical interface the reports and the metrics for evaluating his activities, both to follow the progress during their execution and a posteriori by accessing the history.

[0098] The present invention is located in the field of systems and methods of monitoring, evaluation and / or control of biomechanical movements repeated over time relative to human and / or robotic limbs, through the use of wearable devices and / or integrated into intelligent instrumentation specialized in an application context and enhanced by loT architectures.

[0096]

[0099] The recognition of repeated biomechanical movements guarantees an evolution of the functionalities and services offered with respect to the classical software and hardware applications, allowing as a main aspect the monitoring of activities of the user / machinery, which has utility as a support tool for professional figures in the sports, rehabilitation and / or industrial field, with the further addition of secondary but equally important aspects of data collection, evaluation of quantitative performance metrics, automatic and intelligent interaction with the user, digital graphic reconstruction of biomechanical movements (such as the creation of personalized avatars for entertainment and educational purposes) and development of dedicated functional areas containing multiple interconnected technological instruments. The described system allows new fields of research relating to biomechanics and to the design of optimized robotic systems, due to the analysis carried out.

[0097] 3.1 Hardware architectures

[0098]

[0100] The object of the present invention is a system that performs the counting, recognition and evaluation in real time of biomechanical activities repeated by the user and further usable in the monitoring aimed at the control of robotic and industrial systems, comprising or alternatively consisting of: a) a reference device wearable by an individual, integrated in equipment and / or machinery comprising: i. a microcontroller for managing and processing data from the other components; ii. an IMU 6 dof for measuring the inertial signal related to the movement; iii. any other sensors, such as magnetometers, thermometers, pressure sensors, ECG and / or distance sensors; iv. one or more RGB LEDs to provide real-time visual feedback to the user; v. vibration and / or sound components to provide real-time alternative feedback to the user; vi. one or more antennas for wireless communication and association between the device and the user profile; vii. a power supply battery. b) a combination of devices a) may be associated with the same user so as to increase the processing capabilities of the system. Such configurations allow to improve the quality and functionality of the system by adding new performance metrics; c) one or more calculation nodes and / or a sink node in communication with the reference device a) and / or combination b) capable of: i) in the case of configuration b), synchronizing all the devices associated with the user and / or machine through a broadcast signal in the preliminary acquisition and processing step; ii) receiving the encoded data from the device a) or from the configuration b) performing a further control on the input datum; iii) processing the information collected by artificial intelligence to classify the type of biomechanical movement; iv) carrying out data fusion and post-processing techniques which allow to obtain performance metrics; v) sending a return message to the device a) or to the configuration b) containing the result of the recognition; vi) sending the output of the neural network and the signal recorded over time to a cloud node d) which will communicate to the platform e) the final evaluation of the activity through performance metrics. d) a cloud node for managing and / or controlling the information from the processing node c), updating the information of the user profile and providing the data for displaying the performance by communicating with the graphical interface e). In addition, the cloud service will serve for the training and management of neural architectures with the relative storage in suitable databases of the weights of the architectures and of the various training datasets. e) a visualization node designed as web app, desktop app and / or mobile app, which is in communication with the cloud d) and manages the information allowing the visualization of statistics and metrics of evaluation and performance of the activities.

[0099]

[0101] The possibility of wearing at the same time and using one or more wearable and / or integrated devices in instrumentation, ensuring the possibility of acquiring and managing at the same time different signals coming from several devices are some of the fundamental elements characteristic both of the system and the methodology object of the invention.

[0100]

[0102] According to an embodiment of the invention, the term repeated biomechanical movement means an athletic movement and / or gesture performed regularly by human limbs, for example hand, wrist and / or elbow, or robotic, as mechanical arms for industrial processes and / or humanoids, i.e. movements of such limbs in space and time.

[0101]

[0103] The system of the present invention comprises or, alternatively, consists of a reference device a) which can be worn or integrated inside mobile or fixed instrumentation and can be associated with the user and / or machinery by means of suitable sensors and association methodologies.

[0102]

[0104] By the term device-user association is meant the action of the reference device a) following an input from the user by means of cards or proximity chips, using communication technologies such as Radio Frequency Identification (RFID), or by scanning an identification code with the smartphone, such as QR codes.

[0103]

[0105] The reference device a) and / or the configuration of devices b) allows an asynchronous communication, performed by the component v. of a), with the data processing module c).

[0104]

[0106] Reference device a) and / or device configuration b) communicate a set of processed and encoded data to processing module c). This data relates to information extrapolated by processing the signals from the various sensors, such as for example the translations, the rotations and the attitudes related to various phases of movement performed by the user.

[0107] The reference device a) and / or the device configuration b) determine and subsequently provide to the data processing module c) the starting state and its evolution, organizing the information extracted from the signals over different repeated time ranges, i.e. by detecting the information relating to the single repetition in the 6 degrees of freedom related to the sensor ii. contained in a), adding other information coming from further sensors as indicated in point iii. of a).

[0105]

[0108] The expression "6 degrees of freedom" refers to the number of directions in which the reference device a) can translate and / or rotate with respect to the origin of the axes. The origin and orientation of the axes depend on the specific device a) and / or the combination of devices b) and are chosen with respect to the specific application field based on the configuration most suitable for processing.

[0106]

[0109] The reference device a) comprises or, alternatively, consists of a system adapted to determine and / or transmit the initial position of the movement.

[0107]

[0110] According to a preferred embodiment of the invention, the reference device a) comprises or, alternatively, consists of a system able to determine and / or transmit the initial position of the movement by using one or more sensors able to perceive the physical information measurable by them. Such sensors may comprise accelerometers, gyroscopes, magnetometers and / or position sensors.

[0108]

[0111] According to one embodiment of the invention, the reference device a) and / or combination b) may be selected from the group comprising watches, cuffs, rings, straps for various body parts and / or additional wearable devices.

[0109]

[0112] According to a further embodiment of the invention, the reference device a) can be integrated into technical instrumentation such as handlebars, kettleball, barbell, gym and / or physiotherapeutic equipment, such as fixed machinery, cables, etc. and / or within robotic instrumentation such as industrial and / or humanoid mechanical arms.

[0110]

[0113] According to a preferred embodiment of the invention, the reference device a) is provided with a state machine which allows the electronic device to manage the complete flow of execution of the functionalities, starting from the association with the user until the data is encoded and sent to the calculation node.

[0111]

[0114] The term "state machine" refers to a temporal software program for controlling and / or managing the flow of operations to be performed.

[0112]

[0115] According to an embodiment of the invention, repeated biomechanical movements are meant to be movements and / or gestures repeated with regularity over time which are functional to the correct mechanical activity by the user and / or machinery.

[0113]

[0116] The reference device a) wearable from an individual, after the user has performed the positioning, can segment the signal and processing an output to be communicated to the data processing module c).

[0114]

[0117] The term positioning refers to the process by which the device, starting from a known reference configuration, changes its attitude and position to reach the initial position of the biomechanical activity.

[0115]

[0118] By the term data segmentation and coding refers to that step in which the buffer filled in the acquisition step is processed by the device a) to extract kinematic data such as rise and fall time, device attitude during motion, angles, displacements, etc.

[0116]

[0119] According to a preferred embodiment of the invention, the sensors of the reference device a) and / or the device configuration b) constitutes an inertial platform.

[0117]

[0120] An inertial platform is an aggregation of mobile and static sensors, which evaluate the movements. These sensors are a set of accelerometers, suitable to measure the accelerations in the three directions (longitudinal, transverse and vertical), and gyroscopes which measure the axes of rotation with respect to the reference system positioned integrally with the origin of the sensor. These sensors record variations in translational and rotational inertia with respect to the position of the wearable device (i.e. with respect to a point of the human and / or robotic body) or integrated in tools and / or machinery (i.e. with respect to a point of the instrumentation itself), further determining the vibrations and noise associated with the movement.

[0121] An inertial platform consists of a sensor of the MEMS (Micro ElectroMechanical Systems) type. Such micrometric sensors allow to transform a mechanical input such as pressure, acceleration, rotation, earth magnetic field, temperature or other physical quantities into an electrical pulse, quantifiable and measurable, capable of being processed by the reference device a).

[0118]

[0122] The IMU is a set of different sensors such as, for example, the accelerometer, the gyroscope and possibly the magnetometer which allow to obtain the absolute inclination of the device itself.

[0119]

[0123] The IMU can provide the accelerations along the axes, hence the three accelerations X, Y and Z, from which it is possible to obtain the instantaneous speed with a mathematical formula, and the angular velocities from which it is possible to obtain the three angles of rotation with respect to said axes, providing rolling, pitch and yaw.

[0120]

[0124] The sensors contained in the inertial platform and / or wearable devices are those which allow to describe the motion of a rigid body in space, having 6 degrees of freedom. Its movement can be simplified by breaking it into three translations along a set of three orthogonal axes (X,Y,Z) and three rotations about the same axes. This instantaneous information is provided at a certain frequency allowing to study the evolution of the movement and to reconstruct its trajectory through the aid of real-time processing techniques.

[0121]

[0125] The reference system of each sensor is determined by the orientation of the axes X, Y and Z of the sensor itself which establishes the sign and components of the measurements made by it. By knowing the orientation of the sensor, it is possible to remap the signals acquired into a preferred reference system.

[0122]

[0126] According to a preferred embodiment of the invention, the X axis is the axis transverse with the "+" on the right, the axis Z is the vertical axis facing upwards, while the axis Y of the right-hand triad is the longitudinal axis with the "+" forward.

[0123]

[0127] According to an alternative embodiment of the invention, the inertial platform also comprises a magnetometer, besides the accelerometer and the gyroscope, capable of determining an absolute orientation in the world (such as the reference systems North-East-down or East-North-Up or others) typically used in various application contexts. This configuration is thus a 9 degree of freedom inertial unit (accelerometer, gyroscope and magnetometer).

[0124]

[0128] The accelerometer is a sensor capable of measuring acceleration by calculating the relationship that binds the moving mass with respect to the force that determines its movement (this is therefore a force measurement per unit mass). The accelerometer sensor is therefore the result of the application of laws of mechanical physics using the state-of-the-art technologies.

[0125]

[0129] Gyroscopes are inertial sensors for measuring angular movement (velocity, angular). The physical reference principle is the Coriolis effect. There are many types of gyroscopes related to the implementation mode: tuning-fork, wineglass, Focault pendulum, oscillating wheels. The gyroscope provides the angular velocity measurement by a proportional voltage measured in millivolts per degree per second (mV I (deg s)).

[0126]

[0130] The sensors and / or the inertial platform included in the device a) are integrated in an electronic circuit, powered by a battery which can be recharged by a charging system, such as the USB port and / or induction system.

[0127]

[0131] According to one embodiment of the invention, each device a) of the configuration b) may further comprise magnetometers, thermometers, pressure sensors, ECGs and / or distance sensors.

[0128]

[0132] According to a preferred embodiment of the invention, the device further comprises RGB LED, vibration and / or sound components, i.e. devices capable of returning respectively light, tactile and / or sound feedback to the user, as indicated in points iv. and v. contained in a).

[0129]

[0133] The magnetometer is the instrument for measuring the magnetic field. The measurement of the components of the field along the three independent directions allows to define only the magnetic field vector at the point where the measurement is done.

[0130]

[0134] By means of the inertial platform and / or the further sensors it is possible to recognize the starting position, to count the repetitions and to process the segmented signal of the repeated biomechanical movement, received and analysed by the computation node c) in the form of encoded data for the classification.

[0135] The signals representing the movements can be encoded in signals that can be processed by each of the reference devices a) through signal processing algorithms, such as for example Kalman filters, sliding windows and / or real-time interpolation techniques.

[0131]

[0136] Subsequently, the data processed by the reference device a) is packaged in data structures containing information characterizing physical aspects of the movement, both spatial and temporal, so that the data detected by the sensors can be transmitted both in raw and structured form to the data processing module c) which can interpret and classify them, giving feedback to device a) or combination b).

[0132]

[0137] The raw data from the accelerometer and gyroscope are made available for example via SPI or I2C interfaces.

[0133]

[0138] According to a preferred embodiment, it is possible to define the sampling frequency for data acquisition from the sensor, which is possibly at least 100 Hz. ensuring the possibility of performing solid interpolations and applying noise removal techniques on time intervals consistent with the application context of the devices (indicatively less than 0.1 seconds).

[0134]

[0139] The maximum measuring range of the physical quantities can be set up to ±4G for the accelerometer and ±2000dps for the gyroscope so as to avoid saturation of the data from the sensor in the case of particularly fast gestures.

[0135]

[0140] According to an alternative embodiment of the invention, the reference device a) contains further sensors in addition to the accelerometer and gyroscope, which can provide sampled data at different frequencies and have measurement intervals according to the need.

[0136]

[0141] According to an embodiment of the invention, the wearable or integrated device a) can communicate with other sensors in addition to the IMU, as indicated by iii., allowing to obtain additional information on the reconstruction of the exercise (WBAN tag, pressure sensors, temperature sensors, etc.). The data encoded and sent to the calculation node c) can therefore depend on the specific hardware configuration.

[0137]

[0142] According to a preferred embodiment of the invention, the device a) consists of a printed circuit (PCB boards), or a support used to interconnect the var- ious electronic components of the circuit, i.e. those described from i. to vi. through conductive tracks etched on a non-conductive material.

[0138]

[0143] According to an alternative embodiment of the invention, the device a) is a PCB board provided with CPU, volatile and non-volatile storage devices, microcontrollers, etc.

[0139]

[0144] According to a preferred embodiment of the invention, the device a) is a system on modules (SOM) or computer on modules (COM), i.e. a single-board computer type (SBC) considered an embedded system. An embedded system (embedded system), generically identifies all those microprocessor electronic processing systems specially designed for a particular use often with a dedicated hardware platform, integrated in the system that controls and manages all or part of the required functionality.

[0140]

[0145] According to a preferred embodiment of the invention, the device a) contains a real-time operating system for embedded systems such as FreeRTOS, Zephyr, VxWorks, ThreadX, etc.

[0141]

[0146] According to one embodiment of the invention, the device a) is provided with connection systems, which allows sending and / or receiving of data and / or commands between the device itself and the data processing module c), which can be executed by a local and / or remote and / or cloud server.

[0142]

[0147] The system of the present invention may comprise or, alternatively, consist of a network connection module for connecting the device a) to a server unit and / or a cloud.

[0143] 3.2 Device Configuration

[0144]

[0148] Configuration b) is a combination of one or more devices a) integrable within tools / machinery and / or wearable on one or more limbs of the individual (wearable device) which is / are in communication with data processing module c).

[0145]

[0149] According to an embodiment of the invention, the device composition b) comprises, or alternatively consists of a centralized synchronization mechanism contained in the processing module c), being part of an edge node, such as a local server or by means of an external sink node, able to temporally align the data and to coordinate and manage the correct flow of execution of the functions carried out by the various devices. This mechanism allows, for example, to start the acquisition procedure in a controlled and synchronized manner with consequent processing of the signals.

[0146]

[0150] According to a preferred embodiment of the invention, the devices a) included in combination b) can communicate independently and directly with the calculation node c).

[0147]

[0151] According to an alternative embodiment of the invention, the devices a) included in the combination b) can communicate in an independent and indirect way with the calculation node c) through a device included in the combination itself (Multi-Hop Relay), which can perform both the calculation and transmission functions.

[0148]

[0152] According to an alternative embodiment of the invention, the devices a) included in the combination b) can communicate in an independent and indirect way with the calculation node c) through a device external to the combination and intermediate between the devices that compose it and the calculation node, dedicated to the reception and sorting of data (sink node).

[0149]

[0153] According to a preferred embodiment of the invention, the devices a) which make up the combination b) may be different from each other in terms of hardware components and specialized in different monitoring according to the purpose to be performed.

[0150]

[0154] The combination of devices b) allows to improve the accuracy of the classification and / or increase the number of available performance metrics and functionalities.

[0151]

[0155] According to a preferred embodiment of the invention, the communication between the devices forming the configuration b) and / or the individual device a) and the data processing module c) is carried out by wireless systems, i.e. a wireless connection through a Wi-Fi or Bluetooth system equipped with an internal or external antenna (with reference to low power protocols such as Bluetooth low energy, BLE). It may also include short range transmission, association and / or synchronization systems such as near field communication (NFC) systems.

[0156] According to an alternative embodiment of the invention, the communication between the devices making up the configuration b) and / or the individual device a) and the data processing module c) can be carried out by cable systems, such as a serial connection through USB, UART, I2C and / or SPI ports. In addition, wireless communications may be provided.

[0152]

[0157] The reference device a) and / or the configuration of devices b) is comprising, or alternatively, consists of one or more sensors suitable to perceive and convert the movements of the limbs of the user and / or machinery into signals.

[0153]

[0158] According to an embodiment of the invention, the gesture recognition comprises computer technologies which have the objective of interpreting the activity of the user and / or machine according to the use of mathematical and probabilistic algorithms also in the form of machine learning and / or deep learning techniques to process sequentially or in parallel the information coming from the devices a) which make up a configuration b). Subsequently, the various classification results are post-processed through advanced data fusion techniques so as to obtain a further check on the consistency of the results obtained.

[0154]

[0159] According to a preferred embodiment of the invention, the gestures must originate from a stable starting position relative to the movement that the user intends to perform, which is recognized by the device a) or by the configuration b) at an initial stage. Having known the starting state of the configuration, the users can carry out the activity freely without giving further input, respecting, on the one hand, the capabilities of the system for how it has been designed on the other hand, the real and coherent dynamics of the predetermined types of movement. In particular, the movement performed by the user commonly comes from the limbs, but it can also comprise information relating to other parts of the body in relation to the cardinality of the types of activity that is to be recognized and evaluated.

[0155] 4. Calculation node

[0156]

[0160] The system of the present invention, comprising or, alternatively, consists of one or more calculation nodes and / or a sink node in communication with the reference device a) and / or combination b) capable of: i) in the case of configuration b), synchronizing all the devices associated with the user and / or machine through a broadcast signal in the preliminary acquisition and processing step; ii) receiving the encoded data from the device a) or from the configuration b) performing a further control on the input datum; iii) processing the information collected by artificial intelligence to classify the type of biomechanical movement; iv) carrying out data fusion and post-processing techniques which allow to obtain performance metrics; v) sending a return message to the device a) or to the configuration b) containing the result of the recognition; vi) sending the output of the neural network and the signal recorded over time to a cloud node d) which will communicate to the platform e) the final evaluation of the activity through performance metrics.

[0157]

[0161] According to one embodiment of the invention, the possible architectural configurations of the computing node c) are represented by embedded, edge and cloud computing solutions. The steps that constitute its functionalities can be executed in alternative ways according to the preferred configuration and the application context.

[0158]

[0162] According to one embodiment of the invention, in step i) the synchronization of the devices a) included in the combination b) takes place, which can be perfromed by means of the continuous communication between each wearable device and / or tool / machine and the sink node, which is able to temporally align the received data and to coordinate and manage the correct flow of execution of the functions carried out by the various devices.

[0159]

[0163] According to an embodiment of the invention, step i) is executed by means of a sink node placed in a dedicated device, in the edge computing node and / or in one of the devices a).

[0160]

[0164] According to a preferred embodiment of the invention, the reception of the data encoded by the devices a) included in the combination b), possibly composed of the single device a), described in step ii), can take place by sending, from the wearable devices and / or tools / machinery, signals or electromagnetic pulses, radio frequencies (radio waves, Wi-Fi) which represent the data structures encoded by each reference device starting from the information measured by the sensors and / or by the inertial platform, i.e. characteristics of the movements performed by the limbs.

[0161]

[0165] According to an alternative embodiment of the invention, following the reception of the data encoded by the device a) included in the combination b), possibly composed of the single device a), described in step ii), the conversion and the saving in memory of the data structures received by means of JSON format can take place (JavaScript object notation) or through the comma- separated values format (abbreviated as CSV), or other compatible format.

[0162]

[0166] According to an alternative embodiment of the invention, following the reception of the data encoded by the device a) included in the combination b), possibly composed of the single device a), described in step ii), the conversion and the saving in memory of the received data structures can take place, to be subsequently sent to the cloud node d) in step vi).

[0163]

[0167] According to a preferred embodiment of the invention, in step iii), the user and / or machine recognizes the biomechanical movement through the use of dedicated machine learning techniques. The neural network input is a vector containing the information encoded in the structure(s) received in step ii), while the processing output, containing information relating to the recognition of the movement recorded by each device, is subsequently controlled and processed by advanced data fusion techniques.

[0164]

[0168] According to an embodiment of the invention, step iii), in which the user and / or machine recognition of biomechanical movement takes place using dedicated machine learning techniques, can take place in the devices a) that make up the configuration b), in the edge computing node and / or in the cloud node.

[0165]

[0169] According to a preferred embodiment of the invention, in step iv), the processing module performs post-processing and data fusion techniques for the processing and evaluation of the joint data.

[0166]

[0170] According to an embodiment of the invention, data labelling can be performed in step iv), i.e. labels are assigned which identify a datum to a specific signal corresponding to information on a perceived movement and / or variation by the sensors and / or inertial platform relating to the single recorded repetition. Labelling allows you to create part of the database, which is a set of data used to train the neural network.

[0167]

[0171] The possibility of making a database composed of both simulated signals and data acquired by the physical system, and of supporting the training of the neural network by providing it with a large database (contained in a database on the cloud or on a remote server provided with large memory) are some of the fundamental elements characteristic both of the system and of the methodology object of the invention.

[0168]

[0172] According to a preferred embodiment of the invention, in step v), the processing module provides feedback to the reference device a) or to the configuration b) to notify whether or not the movement performed by the user has been recognized.

[0169]

[0173] According to an alternative embodiment of the invention, the choice of performing step iii) on a remote node or cloud may limit the feedback to devices a) of configuration b) due to the communication latency.

[0170]

[0174] According to one embodiment of the invention, the commands generated in the processing module c) in step iii) and sent during step v), cause each device composing the configuration b) to perform an action and / or a change in its state.

[0171]

[0175] The sending of the coded commands to each device a) of the combination b) is carried out by means of wireless systems, such as Wi-Fi, Bluetooth equipped with an internal or external antenna (with reference to low power protocols, such as Bluetooth low energy, BLE), ZigBee, etc., typically used in loT architectures.

[0172]

[0176] With commands which cause the device(s) a) to change its state and / or perform an action, it is intended, for example, controls suitable for the synchronization of the devices a) forming part of the device(s) b) and therefore to the correct operating flow of the system, which includes electronic components able to provide a feedback of luminous, tactile and / or sound type to the user, and / or quantitative controls in real time capable of operating actuation mechanisms for the correct operation of an automatic robotic machine.

[0177] According to a preferred embodiment of the invention, a change in the state of the system provides for the return of a feedback of luminous, tactile and / or sound type, which can occur following the association of the devices, the control of the rest position, the starting position of the repeated biomechanical movement, the count, recognition and / or end of the activity.

[0173]

[0178] For example, a user may associate several devices a) to form a predicted configuration b). After installing the wearable devices, the user can take the associated tools and place in the rest configuration. The system, after a control on the stability of the position adopted by the user, will provide feedback that will allow him to start the activity he intends to perform. After this step, the user can proceed to the exercise, allowing the system to perform the counting and recognition in real time.

[0174]

[0179] For example, a machine equipped with several devices a) which make up a configuration b) and suitable for executing a repeated mechanical activity is monitored and controlled by the calculation module c), this can send control commands generating a mechanical action to the robotic machinery to allow its correct operation. The actions envisaged are, for example, the actuation of mechanical mechanisms operated by actuators, the cooling and / or the heating, the suspension of the activity and / or the change of the activity.

[0175]

[0180] These are examples of possible interactions, but it is intuitive that through such a system and / or method it is possible to manage activities related to fitness, professional sport or rehabilitation and / or to provide monitoring systems for control and design methods for automated robotic arms in industrial and / or humanoid fields.

[0176]

[0181] According to a preferred embodiment of the invention, in step vi), the processing module c) sends to the cloud node d) the data received in step ii) and labelled in step iv) and the outcome of the processing performed in steps iii) and / or iv). The data is then decoded and stored on a remote and / or cloud server, and made available to the user by suitable displays contained within dedicated application programs (desktop app, web app or mobile app).

[0177]

[0182] According to one embodiment of the invention, step vi), which provides for sending the data received in step ii) and labelled in step iv) and the outcome of the processing performed in steps iii) and / or iv) can be performed by the sink node, by the edge node, by one of the devices a) and / or from each of them on the basis of the architectural choice.

[0178]

[0183] According to an embodiment of the invention, it is possible to acquire simultaneously signals coming from each device a) included in configuration b), placed on different parts of the body (limbs and / or biomechanical body segments) and / or integrated into tools / machinery, counting and recognizing the user's activity and / or machinery in real time, and quantitative evaluation, both spatial and temporal, providing performance metrics to the user viewable by application, saving them on servers.

[0179]

[0184] According to an embodiment of the invention, it is possible to acquire simultaneously signals coming from each device a) included in the configuration b), placed on different parts of the body (limbs and / or biomechanical body segments) and / or integrated into tools / machinery, and the combination of such multiple signals is converted by the processing module c) into a single command and / or a plurality of simultaneous commands and coded data that is collected, saved and managed by the cloud node d).

[0180]

[0185] The system and / or method of the invention counts, recognizes and evaluates the repetitions of biomechanical movements repeated in real time. The counting is performed by each device a) by segmenting the signal, the recognition is performed by means of machine learning algorithms. The evaluation of the performance metrics is based on the acquisition and processing of the data collected over time and it is reinforced by parametric biomechanical modelling which allows to have nominal case studies relative to the specific activities making notes both kinematics information (such as the complete knowledge of the trajectory of the rigid body) and those dynamics (forces, constraining reactions, energy consumed etc.). The performance metrics, in addition to those already described, include, by way of example, the detection of asymmetries and failures, the evaluation of the rhythm and the achievement of specific objectives. Recognition therefore represents a central functionality of the system, whose precision depends on the success rate in recognizing repeated biomechanical movements expected, rejecting any other gesture that has not previously been included in the database used to train the neural network. The response time, depending on the architectural choice, is also an important feature for this type of realization.

[0181]

[0186] According to one embodiment of the invention, the processing module c) may comprise at least one electronic card, or even not, for processing the signal from the sensors, for filtering such signals and for converting analogue signals into digital signals to be transmitted to the devices of the configuration b) and to the cloud node d).

[0182]

[0187] According to one embodiment of the invention, the processing module c), in communication with the devices a) of the configuration b) and with the cloud node d), can be a software platform configured to perform the steps of the invention.

[0183]

[0188] The platform, constituting the module c), comprises one or more artificial intelligence modules.

[0184]

[0189] According to a preferred embodiment of the invention, the processing module c), in communication with the devices a) of the configuration b) and with the cloud node d), may be software configured to perform the steps of the invention.

[0185]

[0190] A platform is a hardware and / or software base on which programs or applications are developed and / or executed; it may also indicate a running environment which includes hardware and operating system and possibly specific middleware elements, server applications and other tools to support program execution.

[0186]

[0191] The platform is the hardware composition on which a certain operating system and a certain set of application programs are executed (typically the processor architecture).

[0187]

[0192] According to one embodiment of the invention, a platform may be software, or a type of framework or the operating system on which the programs and applications that are developed and / or executed run.

[0188]

[0193] In some embodiments, the platform and / or system and / or method described herein include at least one computer program, or use thereof.

[0194] A computer program is an algorithmic method applied to a given problem to be automated and typically encoded in a series of code lines written in a certain programming language during programming making a software program. The application can be executed by a computer, receiving as input certain data and returning in output the possible results obtained following the execu- tion / processing of its instructions.

[0189]

[0195] The term "application" identifies an installed software or a series of software being executed on a computer with the purpose and result of making one or more functionality possible.

[0190]

[0196] A server is a computer component or subsystem for processing and managing information traffic (data) that provides, at a logical and physical level, any type of service to other components (typically called clients) that request it through a computer network, within a computer system or even directly locally on a computer.

[0191] 4.1 Neural network

[0192]

[0197] According to a preferred embodiment, of the invention, the data processing module c) contains an artificial intelligence model.

[0193]

[0198] The term artificial intelligence model (i.e. model IA) may be used herein to refer to a wide variety of information structures that can be used by a data processing module to perform a calculation or evaluate a specific condition, characteristic, factor, dataset or behaviour. Examples of IA models mainly include neural network models, interference models, neural models, classifiers, decision tree models, spiking neural network models (SNN), convolutional neural network models (CNN), recurrent neural network (RNN), deep neural network models (DNN), generative network models. In some embodiments, an IA model may include an architectural definition (e.g. neural network architecture, etc.) and one or more weights (e.g. neural network weights, etc.).

[0194]

[0199] The term "neural network" may be used herein to refer to an interconnected group of processing nodes (or neural models) that collectively operate as a software application or process that controls a function of a computing device and / or generates an overall output interface result. Single nodes in a neural network can attempt to emulate biological neurones by receiving data, perform- ing simple operations to generate output data, and passing them to the next node in the network.

[0195]

[0200] Automatic learning (i.e. machine learning) is a branch of artificial intelligence that collects methods developed under different names such as: computational statistics, pattern recognition, artificial neural networks, adaptive filtering, dynamic systems theory, image processing, data mining, adaptive algorithms, etc., using statistical methods to improve the performance of an algorithm in identifying patterns in the data. In the field of computer science, machine learning is a variant to traditional programming in which a machine is prepared to learn something from the data independently, without explicit instructions.

[0196]

[0201] The availability of biomechanical movements stored in the database implies that, for each movement, coded data have been obtained by means of suitable acquisition and processing methods starting both from really acquired data and from those simulated by the modelling. Each type of movement has the same data structure.

[0197]

[0202] According to a preferred embodiment of the invention, the processing module c) is an automatic learning model using an algorithm.

[0198]

[0203] According to a preferred embodiment of the invention, machine learning is developed using artificial neural networks and / or adaptive algorithms and / or automatic learning algorithms and / or machine learning algorithms. This approach is strengthened by the generation of synthetic signals that allow to realize a complete and meaningful database for a wide variety of activities.

[0199]

[0204] Machine learning is closely related to pattern recognition and computational theory of learning and explores the study and construction of algorithms that can learn from a set of data and make predictions thereon, inductively constructing a sample-based model.

[0200]

[0205] Examples of machine learning algorithms may include a support vector machine (SVM), a naive Bayes classification, a decision tree, supervised learning algorithm, or unsupervised learning algorithms for classification and regression. Machine learning algorithms can be trained using one or more training datasets.

[0206] In some embodiments, the machine learning algorithm uses regression models, in which relationships between prediction variables and dependant variables are determined and weighted.

[0201]

[0207] According to a preferred embodiment of the invention, the platform and / or system and / or method allows to generate and / or use neural networks and / or algorithms for automatic learning (i.e. machine learning algorithms) which allow to identify anomalies with comparable accuracy. The choice between neural networks and machine learning algorithms is given by the compromise between optimization duration (processing duration) and available hardware resources.

[0202]

[0208] According to an alternative embodiment of the invention, an automatic learning algorithm employs backpropagation convolutional neural network (BCNN) algorithms, or convolutional neural network algorithms.

[0203]

[0209] According to a preferred embodiment of the invention, the machine learning algorithms used can comprise algorithms of the decision tree type, SVM, K- NN, etc.

[0204]

[0210] According to an alternative embodiment of the invention, an algorithm for recognizing biomechanical movements and their repetitions can be DTW (Dynamic Time Warping) and HMM (Hidden Markov Model).

[0205]

[0211] According to one embodiment of the invention, the platform and / or system and / or method include one or more databases. In various embodiments, suitable databases include, by way of non-limiting examples, relational databases, non-relational databases, oriented databases, entity databases, associative databases, XML databases. Additional non-limiting examples include SQL, MySQL and Oracle. In some embodiments, a database is based on a remote server connected to the Internet. In further embodiments a database is cloud computing. In other embodiments, a database is based on one or more storage devices such as local computers and / or servers.

[0206]

[0212] The term database, database or database (sometimes abbreviated as DB), refers to a data store, or a set of well-structured archives, in which the information contained therein is structured and connected to each other according to a particular logic model (relational, hierarchical, reticular or object-like) and in such a way as to allow the efficient management / organization of the data and the interfacing with the user's requests through the so-called query language (query or query, insertion, cancellation, updating etc.). Thanks to dedicated software applications (Database Management Systems or DBMS), based on a client-server architecture.

[0207] 4.2 Hardware

[0208]

[0213] According to an embodiment of the invention, the data processing module c) comprising or, alternatively, is a digital processing device.

[0209]

[0214] In further embodiments, the digital processing device comprises one or more hardware central processing units (CPUs) and graphics processing units (GPUs) that perform the functions of the platform and / or system and / or method of the invention. In still further embodiments, the digital device comprises an operating system configured to execute dedicated instructions. In some embodiments, the digital processing device is connected to a computer network which may be a local server. In further embodiments, the digital processing device is connected to the Internet to allow interaction with remote servers and / or cloud servers. In still further embodiments, the digital processing device is connected to a cloud computing infrastructure. In other embodiments, the digital processing device is optionally connected to an intranet.

[0210]

[0215] According to a preferred embodiment of the invention, the digital processing device has a CPU which allows parallel calculation and execution of multi-threading processes.

[0211]

[0216] In accordance with the description given herein, suitable digital processing devices include, by way of non-limiting examples, server computers, desktop computers, cloud computers, smartphones and tablets and / or custom hardware.

[0212]

[0217] In some embodiments of the invention, the processing device comprises a dedicated operating system. The operating system is for example a software, including programs and data, which manages the hardware of the device and provides services for running applications. As non-limiting examples we have FreeBSD, OpenBSD, Linux, Apple MacOS server®, Oracle®, Windows server®. Suitable commercial computer operating systems include Microsoft ®, Windows®, Apple MacOS®, UNIX®, and UNIX operating systems such as GNU / Linux®. In some embodiments, a real-time operating system (RTOS) is installed on the digital processing device, which may be, as non-limiting examples, Xenomai, Preempt-RT, Litmus-RT, RTLinux, RTAI and TimeSys Linux.

[0213]

[0218] In some embodiments of the invention, the processing device comprises a storage and / or memory device, which are one or more physical apparatuses used to store data or programs on a temporary basis (volatile memory) and / or permanent (non-volatile memory). In some embodiments, the non-volatile memory comprises a flash memory, ROM, EEPROM, SSD and / or others. In some embodiments, the volatile memory comprises a random-access memory (DRAM, SRAM, and / or other forms of RAM). In further embodiments, the storage device may be cloud-based. In further embodiments, the storage and / or memory device is a combination of devices such as those described herein.

[0214]

[0219] According to an embodiment of the invention, the activities for which the module c) is configured are carried out by one or more computer programs. Specifically, such programs can be installed on different hardware architectures constituting embedded, edge and / or cloud computing systems.

[0215]

[0220] According to an embodiment of the invention, the activities for which the module c) is configured are executed by means of a software platform and / or software.

[0216]

[0221] According to an embodiment of the invention, the data processing module c) can be included, inserted and / or stored in an embedded, edge and / or cloud unit.

[0217] 4.3 Wireless data exchange and communication

[0218]

[0222] The system of the present invention may comprise or, alternatively, consists of a network connection module configured to connect the devices a) of the configuration b) and the processing module c) to the Internet and / or local intranet.

[0219]

[0223] The web connection module may comprise or, alternatively, consists of a set of software and hardware infrastructures suitable to allow the devices a) of the configuration b) and the processing module c) to connect and then interact with the cloud node d).

[0224] According to one embodiment of the invention, the devices a) that make up the configuration b) and / or the processing module c) and / or the cloud node d) can be in communication with each other.

[0220]

[0225] According to one embodiment of the invention, communication between the devices a) that make up the configuration b) and / or the processing module c) and / or the cloud node d) include wireless systems such as Wi-Fi and / or Bluetooth equipped with an internal or external antenna (with reference to low power protocols, as Bluetooth Low Energy, BLE). It may also include short range information transmission systems such as near field communication (NFC) systems.

[0221]

[0226] According to an alternative embodiment of the invention, the communication between the devices a) that make up the configuration b) and / or the processing module c) and / or the cloud node d) can be carried out through cellular telecommunication network (i.e. cellular network or mobile network).

[0222]

[0227] The cellular telecommunications network is a network that allows telecommunications at all points of a territory divided into non-large areas, called "cells" (from which the definition) for cellular mobile radio telephony, each served by a different base radio station. Mobile telephony, using radio waves in the form of a radio communication, is able to serve entire geographical areas continuously with the advantage of user mobility.

[0223]

[0228] According to one embodiment of the invention, communication between the devices a) that make up the configuration b) and / or the processing module c) and / or the cloud node d) can be carried out by means of a fifth generation or higher mobile network (5G and / or 6G).

[0224]

[0229] The term "5G" identifies any system using the 5G NR (5G New Radio) software.

[0225]

[0230] The term 5G indicates new generation technologies and standards for mobile communication. This "fifth generation", which follows the previous 2G, 3G and 4G, is therefore the connection technology that will use our smartphones, but also and above all the many connected objects (loT, Internet of things) around us, destined to be more and more numerous (wearable devic- es, equipment, technical machinery, etc.). One of the main characteristics of this network is, indeed, that it allows many more connections simultaneously, with high speed and very fast response times.

[0226]

[0231] Like its predecessors, the 5G network is a digital cellular network, in which the area covered by the service is divided into small geographical areas called cells. All devices 5G within a cell receive and transmit the signal via radio to the local antenna, which in turn is connected to the telephone network and the internet via high-capacity optical fiber or via radio link through the backhaul network.

[0227]

[0232] According to one embodiment of the invention, communication between the devices a) that make up the configuration b) and / or the processing module c) and / or the cloud node d) can be performed by radio frequency, more preferably radio waves with a very high frequency up to

[0228] 300 GHz.

[0229]

[0233] According to an embodiment of the invention, the communication between the devices a) that make up the configuration b) and / or the processing module c) and / or the cloud node d) can be carried out by radio frequency with transmission rates of from about 50 Mbit / s to over 1 Gbit / s.

[0230]

[0234] According to an embodiment of the invention, the devices a) that make up the configuration b) and / or the processing module c) and / or the cloud node d) can comprise or, alternatively, be constituted by suitable systems and / or equipment and / or software capable of supporting the network 5G or subsequent generations.

[0231]

[0235] According to one embodiment of the invention, the devices a) that make up the configuration b) and / or the processing module c) and / or the cloud node d) are adapted to support the network 5G or subsequent generations.

[0232] 4.4 Cloud service

[0233]

[0236] According to a preferred embodiment of the invention, the system consists of a cloud node d).

[0234]

[0237] According to a preferred embodiment of the invention, the system consists of a cloud node d) for managing and / or controlling the information prevented by the processing node c).

[0238] The cloud unit may comprise or, alternatively, consist of a set of software and hardware infrastructures allowing the system of the invention to save and / or implement and / or perform the steps of the data processing module c) and / or send updated data to the graphic interface e).

[0235]

[0239] According to an alternative embodiment of the invention, the cloud node d) can be used to host part of the data processing described in the processing module c), possibly providing direct communication between the devices a) of the configuration b) and the cloud node itself (cloud computing solution).

[0236]

[0240] According to a preferred embodiment of the invention, the cloud node d) is in communication with the graphical interface e) for updating the user profile information and providing the performance display data.

[0237]

[0241] According to one embodiment of the invention, communication between the cloud node d) and the graphic interface e) is carried out by means of Internet network protocols.

[0238]

[0242] According to a preferred embodiment of the invention, the cloud node d) is used for training and management of neural architectures with the relative storage in suitable architecture weights databases.

[0239]

[0243] According to a preferred embodiment of the invention, the cloud node d) is used for collecting big data used as neural network training datasets. Such datasets include both data encoded starting from both real signals and simulated data provided by modelling, in accordance with the methodology of the present invention.

[0240]

[0244] According to one embodiment of the invention, the cloud node d) is used for storing user data in the form of structured databases.

[0241]

[0245] According to an embodiment of the invention, the data processing module c) can be included and / or inserted and / or stored in the cloud node.

[0242] 4.5 User Application

[0243]

[0246] The system of the present invention comprises a display node e) thought of as web app, desktop app and / or mobile app.

[0244]

[0247] The system of the present invention comprises a graphical interface node e) in communication with the cloud d) which manages the information allowing the display of performance metrics and statistics.

[0248] The system of the present invention comprises a display and / or print unit e) adapted to be connected to the cloud node d) and adapted to display and / or print the obtained outputs.

[0245]

[0249] The system of the present invention comprises a display and / or print unit e) which reads the data to be displayed and / or printed from a database contained in the cloud node d).

[0246]

[0250] According to a preferred embodiment, the graphical interface e) allows the user to access the output of the system once it is finished the repeated biomechanical activity and / or movement. This output is comprised of performance metrics processed by the system itself.

[0247]

[0251] According to a preferred embodiment, the graphical interface e) allows the user to access the personal history, making accessible to it the performance metrics related to activities previously performed.

[0248]

[0252] According to a preferred embodiment, the graphical interface e) allows the user to manually enter his own data and / or customize the evaluation service according to his own objectives.

[0249]

[0253] According to a preferred embodiment, the graphical interface e) allows the user to enter and / or plan a path for achieving his personal goals, allowing him to monitor progress.

[0250] 4.6 Description method of the invention

[0251]

[0254] The system of the invention is combined with a methodology, which is an integral part of the present invention, which includes on the one hand the use of a state machine integrated in each device a) to regulate and manage the correct flow of information and to focus the acquisition phase only at the moments of actual user activity, on the other hand, the possibility of generating synthetic acceleration and angular velocity signals relating to exercises for training the neural network to the recognition of movement, associating with it a biomechanical evaluation obtained from the modelling used to generate the signals themselves. This methodology serves to give solidity and completeness to the network learning database, which will therefore be formed both from really acquired signals and from simulated signals, and to reconstruct relevant and hidden information relating to the biome- chanics of the movement with an arbitrarily deep and specialized degree of detail according to the product request. For this purpose, this methodology can be implemented in software libraries that can be integrated in ad-hoc developed applications or in external applications.

[0252]

[0255] An object of the present invention is a method for counting, recognizing and evaluating in real time biomechanical activities repeated by the user and further usable in monitoring for the control of robotic and industrial systems, comprising the following steps: a. associating and synchronizing one or more devices a) which, together with the use of data fusion techniques, allows the personalized processing and management of predefined configurations b); b. adjustment and management of the system workflow by means of a state machine that allows the recognition of the initial positioning and the signal segmentation and provides the real-time count of the repetitions; c. restitution of real time feedback to the user on the status of the system by means of light, sound and / or tactile signals, such as counting, device status, recognition, etc. d. creation of the training database of the neural network formed both by really acquired signals starting from the activity of one or more users and by those generated by means of a parametric physical-mathematical model; e. customisation of neural architectures for the analysis of the features extracted from the calculation node c) analysing the signal segmented by the device a). f. specialization of recognition dependant on the sensoristic composition of devices a) and their combination b); g. restitution of the performance metrics customized on the basis of the user profile and obtained by comparing the mathematical modelling of the biomechanical movement and the real acquired signal during the activity.

[0253]

[0256] According to an embodiment of the invention, the method of the invention is performed by means of the system of the invention.

[0257] According to a preferred embodiment of the invention, the method can be performed by one or more software configured to perform the steps of the method.

[0254]

[0258] According to a preferred embodiment of the invention, the method can be performed by means of one or more software running on the hardware on the devices a) making up the configuration b), on the calculation node c) and on the cloud node d).

[0255]

[0259] Step a. is performed by a sink node which can be placed in a dedicated device, in the edge computing node and / or in one of the devices a).

[0256]

[0260] Steps from b. to c. are performed by devices a) in real time.

[0257]

[0261] Steps from d. to f. are performed offline during system development using cloud service d) and may be subject to software updates.

[0258]

[0262] Step g. is performed by the computing node c) and / or on the cloud service d) to render the system outputs displayable via the graphical interface e).

[0259]

[0263] Using the method of the present invention, a system and / or method for counting, recognizing and evaluating repeated biomechanical movements performed by the user / machinery is available.

[0260]

[0264] Using the method of the present invention, a system and / or method for counting, recognizing and evaluating repeated biomechanical movements performed by the user / machinery is available using physical-mathematical modelling, big data analysis and artificial intelligence algorithms.

[0261]

[0265] By using the method of the present invention, a system and / or method for counting, recognizing and evaluating repeated biomechanical movements performed by the user / machinery is available using physical-mathematical modelling and big data analysis and artificial intelligence algorithms, received by suitable sensors (for example an inertial platform) and processed based on the state of the devices a), thus allowing the signal to be segmented and the realtime count to be obtained.

[0262]

[0266] By software is meant the set of programs used in a data processing system that manages the operation of a computer.

[0267] In addition to providing a subject system and / or method, embodiments also include computer programs capable of performing the operations of the method of the invention.

[0263]

[0268] Computer systems and / or programs, according to some embodiments of the present invention, may be inserted into computer readable storage media and / or defined with computer readable program code and incorporated into the storage medium.

[0264]

[0269] It will be understood that the method of the invention can be implemented by computer program instructions. These computer program instructions may be loaded onto a computer, or other programmable apparatus, to produce the method steps such that instructions that are executed correctly and implement the specific functions of the method of the invention.

[0265]

[0270] These instructions of the computer program may also be stored in a computer readable memory, which may direct a computer or other programmable apparatus to perform the steps of the method, such that the instructions stored in the computer readable memory produce the method of the invention.

[0266]

[0271] The embodiments of the method of the invention may be implemented in hardware, firmware, software or any combination thereof.

[0267]

[0272] The embodiments may also be implemented as instructions stored on a machine-readable medium, which may be read and executed by one or more processor circuits.

[0268]

[0273] A machine-readable medium may include any mechanism for storing and / or transmitting information in a machine-readable form (such as a processing device). A machine-readable medium may include, for example, readonly memory (ROM), random access memory (RAM), removable storage media, and / or cloud services.

[0269]

[0274] The method of the invention and / or its component steps may be implemented as computer readable code (i.e. machine-readable computer program instructions) which is executed by one or more processors to perform the operations of the described embodiments.

[0270]

[0275] The method of the invention and / or its component steps may include components which can be implemented in the computer system using hard- ware, software, firmware, computer readable media (i.e. machine-readable) on which computer program instructions are stored, or a combination thereof, and may be implemented in one or more computers or computer systems or other processing systems.

[0271]

[0276] If programmable logic is used, it may be performed on a commercially available processing platform or on a device for special purposes. One skilled in the art may understand that embodiments of the disclosed object may be practiced with various configurations of computer systems, including multiprocessor systems, multi-core systems, minicomputers, computers, mainframe, connected or grouped computers with distributed features or miniature computers that can be incorporated into virtually any device.

[0272]

[0277] After reading this description, a person of ordinary skill in the subject technical field will know how to implement the embodiments described using other computer systems and / or computer architectures. Although the operations may be described as a sequential process, some of the operations may be performed in parallel, simultaneously and / or in a distributed environment and with the program code, stored locally or remotely, for access by single or multiprocessor machines. Furthermore, in some embodiments, the order of the operations can be reorganized without departing from the spirit of the object described.

[0273]

[0278] Computer programs, when executed, allow the computer system to implement the method of the invention and / or its component phases. Specifically, computer programs, when executed, allow the processor to implement the processes of the method of the invention. When an embodiment is implemented using the software, it can be stored in a peripheral storage device or in the cloud.

[0274]

[0279] An apparatus, device and / or data processing system may include any suitable machine readable (i.e. computer-readable) storage device on which computer program instructions describing the method of the invention are stored.

[0280] The method of the invention can be implemented using software, hardware and / or operating systems implementations other than those described herein.

[0275]

[0281] Any implementation of suitable programs, software, hardware and operating system, to carry out the steps of the method of the invention it can be used.

[0276]

[0282] The embodiments are applicable to both a client and a server or a combination of both.

[0277]

[0283] The description sets out exemplary embodiments and, as such, does not intend to limit in any way the scope of the embodiments of the description itself and of the appended claims.

[0278]

[0284] The method of the invention and its component steps can be implemented through computer systems that allow its use and / or execution via local computer networks or computer networks.

[0279]

[0285] A computer network is a packet-switched telecommunications network type characterized by a set of hardware devices with suitable switching software, i.e. switching nodes connected to each other by suitable communication channels which allow the exchange and sharing of data and the communication between the distributed devices or terminals (hosts): the data is transmitted and transferred in the form of data packets (PDUs), which are provided by a header (which contains the data for message delivery) and a body (which contains the body of the message), all regulated by precise network protocols.

[0280]

[0286] The network provides a data transfer service, through common transmission and reception capabilities, to a population of devices distributed over a larger or smaller area. Examples of the computer network are the LAN, WLAN, WAN and GAN networks whose global interconnection gives rise to the Internet network.

[0281]

[0287] A further object of the present invention is an apparatus, device and / or data processing system comprising a processor suitable for carrying out the steps of the method of the present invention.

[0282]

[0288] A further object of the present invention is the design and development of loT technology instrumentation dedicated to the functions described by the present invention.

[0289] A further object of the present invention is a parametric physical- mathematical modelling for obtaining nominal signals representing repeated biomechanical movements capable of enhancing the learning capabilities of neural networks.

[0283]

[0290] The present invention further provides a computer program comprising instructions which, when the program is executed by a computer, cause it to perform the steps of the method of the present invention.

[0284]

[0291] Object of the invention is also the use of the system of the invention for counting, recognizing and evaluating repeated biomechanical movements in real time.

[0285]

[0292] according to a preferred embodiment, the use of the system of the invention for counting, recognizing and evaluating repeated biomechanical movements is preferred.

[0286]

[0293] All of the preferred features and embodiments of the above-mentioned system and / or method of the present invention may be combined in any possible combination to describe the system and / or perform the claimed method.

[0287]

[0294] From the above description, the advantages of the method of the present invention are evident, which solves at the same time many problems present in the prior art using the main currently available technical instruments.

[0288]

[0295] Many modifications and / or embodiments of the invention set out herein will be in mind to one skilled in the art to which these inventions relate, having these benefit from the teachings presented in the foregoing descriptions and accompanying drawings. Therefore, such modifications and / or other embodiments are to be understood as being included within the scope of the appended claims.

[0289]

[0296] It should therefore be considered that the subject invention should not be limited to the specific embodiments described and that modifications and / or other embodiments are to be understood as included within the scope of the appended claims.

[0290]

[0297] As will be apparent from the description of the invention herein, the invention finds application in various fields of the art and not only, therefore it is of interest to the application: in the counting, automatic recognition and evaluation of repeated biomechanical movements for fitness, competitive sports and / or rehabilitation, in the field of wearable accessories, in the monitoring aimed at realtime control for industrial robotic systems and / or development and design of mechanical arms.

[0291]

[0298] Although specific terms are used herein, they are used only in a generic, exemplary and descriptive sense and not for the purpose of the invention.

[0292]

[0299] The terms "collected data", "acquired data", "detected data", and "measured data" can all be used in the present invention to refer to data acquired from an individual device (e.g., using its sensors, etc.), and / or data acquired from an inertial platform and / or data generated by biomechanical simulations.

[0293] 5. Description of the drawings

[0294]

[0300] The accompanying drawings, which are incorporated and form an integral part of the present specification, illustrate the embodiments and, together with the description, explain the principles of methods and systems.

[0295]

[0301] FIG. 1 shows in a schematic and simplified manner the block diagram of the system in question containing the main components of the invention: an electronic device which can be worn or integrated in instrumentation a), a multidevice combination b), a calculation node c), a cloud node d) and a graphic interface e). The communications and location of the components of the computing node c) may vary depending on the architectural choice (embedded, edge or cloud computing.

[0296]

[0302] FIG. 2 shows a possible implementation comprising a multi-device application strategy, offering a specialized service for monitoring and evaluating repeated biomechanical movements performed by the user for sports and / or rehabilitation purposes. The present illustration provides for the use, not strictly necessary according to the chosen architectural solution, of further devices dedicated to data synchronization which can be represented by a minicomputer, by the user's smartphone or by other dedicated devices.

[0297]

[0303] FIG. 3 shows a possible alternative multi-device implementation applicable in the robotic and industrial field for monitoring for limb control.

[0298]

[0304] FIG. 4 shows an illustrative workflow which describes at a high level the entire system, starting from the intelligent instrumentation, which is intended for the extraction of characteristic information and for the return of feedback in real time, up to the graphical display of the performance and the quantitative and qualitative evaluations that can be customized through a dedicated graphical interface.

[0299]

[0305] FIG. 5 illustrates a possible prototype of the printed electronic circuit that makes up the device a). In this example the components are a microcontroller ESP32, an IMU sensor 9 dof (BNO086), two RGB LEDs and an antenna dedicated to short-range communication.

[0300]

[0306] FIG. 6 illustrates in a figurative manner the architectural choice implementing the edge computing approach using, for example, a dedicated local server in which the computing node c) is entirely contained, and using the cloud service for database management and training of neural networks with consequent saving of weights. This architectural choice is well suited in use environments having dedicated technological areas in which it is possible to implement a robust IT infrastructure.

[0301]

[0307] FIG. 7 illustrates in a figurative manner the architectural choice which implements the embedded computing approach using, for example, neural networks adapted based on the available computing power and / or advanced hardware components. This implementation choice reduces architectural complexity, adapting well to individual and domestic use contexts.

[0302]

[0308] FIG. 8 illustrates in a figurative way the architectural choice that implements the cloud computing approach, which, by managing the entirety of the processing chain, results to have an advanced computing power and an optimal data management, to the detriment of technical limitations such as the need to have an adequate internet connection and the introduction of latency in the communication between the computing node c) and the devices a) for the return of feedback on the recognition in real time.

[0303]

[0309] FIG. 9 schematically shows an example of a state machine implemented in each device a), which allows to manage the complete flow of execution of its functions in real time, allowing the counting of repeated movements, the automatic recognition of the initial positioning, the segmentation of the signal, the encoding and sending of the data to the calculation node c). The states repre- sent the working steps of the device, starting from the process of association between device and user, until the extraction of the information useful for recognition and final evaluation is reached.

[0304]

[0310] FIG. 10 shows in an illustrative manner the processing of the signal performed by the device a), which provides for example noise filtering and signal interpolation techniques to segment the signal relating to the repetition and extrapolate the characteristic information necessary for the subsequent processing.

[0305]

[0311] FIG. 11 schematically shows a graph describing the line of implementation of the system. Based on the economic and technological availability and complexity of the desired service, it is possible to make an evaluation of the architectural choices of implementation.

[0306]

[0312] FIG. 12 illustrates at logic level the flow chart relating to the operation of the entire system, which maintains the generality regardless of the architecture chosen.

[0307]

[0313] FIG. 13 shows a comparison between signals recorded by an IMU 6 dof with signals generated by mathematical modelling relating to the specific type of movement performed. In particular, the consistency of the results is noted both in form and in amplitude. Any dislocation is due, in addition to the noise acting on the sensor, also to imperfections in the execution of the gesture, which are managed, on the one hand, by the state machine, on the other hand, by the solidity of the same modelling.

[0308] 6. Example of embodiment of the invention

[0309]

[0314] The present invention, described at a high level from FIG. 1 , allows numerous embodiments in different application contexts, such as for example that of fitness, rehabilitation and professional sport (see FIG. 2), also leading to monitoring areas for control for industrial and robotic systems (see FIG. 3).

[0310]

[0315] In the field of fitness, sports and rehabilitation, forms of development of the present invention are, as non-limiting examples, specialized infrastructural IT services for technological areas dedicated to training (Wireless Body Area Network, or WBAN), wearable devices for monitoring biomechanical activities that provide for the repetition of movements performed with human limbs, smart gym equipment dedicated to fitness such as handlebars, rockers, gym machines etc., specialized rehabilitation tools, support tools for professionals in the field (personal trainers, coaches, physiotherapists, etc.) these allow for personalized and intelligent follow-up with the graphical interface.

[0311]

[0316] In the industrial field it is possible to provide computer services that connect to the machinery to monitor their regular operating state through evaluation metrics based on data from a network of sensors. The feedback can be returned in coded data to the actuators which allow to perform a determined action, ensuring a greater precision, autonomy, efficiency and resilience of industrial systems.

[0312]

[0317] In the robotic field, specialized monitoring methods can be implemented to control humanoid limbs or mechanical arms. The data from the sensors can be encoded and processed by a central calculation node which converts the raw information derived therefrom to control the movements in a predictive and adaptive manner.

[0313]

[0318] The system and method of the present invention can be used to produce technological gym equipment, containing electronic devices such as that shown in FIG. 5, specialized in the automatic monitoring of exercises. The functions envisaged, constituting the operating fulcrum of the invention itself, are the counting, automatic recognition and real-time evaluation of the exercises carried out by the user, described by the flow chart in FIG. 12.

[0314]

[0319] The exercises performed by the user by gripping gym equipment can be, in accordance with the methodology presented, described using mathematical models and methods applied to the biomechanics of the limbs. The formalism used allows to start from a general formulation of the arm, understood as a set of levers and joints characterized by degrees of freedom, to specialize the description on movements characterizing exercises which serve to train a given muscular group, which is specialized to perform a variation of its configuration by optionally activating muscle groups which stabilize the movement itself. Any biomechanical gesture, carried out with overload or without, is in fact provided with relative constraining reactions on the joints and / or degrees of freedom not directly involved in the specific movement.

[0320] Starting from these considerations it is possible to generate inertial signals simulating the acquisition of sensors placed inside the equipment relating to a set of exercises that is intended to cover in the offered service, compatible with those acquired by the sensors placed in the tool itself (see FIG. 13). These signals are associated with biomechanical evaluations which provide information on stress, constraint reactions, musculoskeletal stabilization, based on raw information such as rise and fall times and range of motion, including execution defects and generalization in motion variants.

[0315]

[0321] The signal acquired by the sensors placed inside the equipment allows to recognize repeated signals in time having similar shape to each other. The mechanism performed by the state machine described in FIG. 9 allows the signal to be segmented by obtaining rise and fall times of each repetition, extracting characteristic information relating to the shape and amplitude of the acquired signal (see FIG. 10), used for subsequent processing, which provides for the complete recognition and evaluation of the quality of the exercise, providing a set of performance metrics. The segmentation of the repeated gesture, which takes place by recognizing the rise and fall pattern, is followed by sending such data to the calculation node and enables the count with consequent real-time feedback to the user.

[0316]

[0322] The recognition of the exercise allows to specialize the evaluation based on nominal samples of exercises, the time series of information extracted from the sensors and the signal itself. This workflow, described in an illustrative and general manner from FIG. 4 and from FIG. 12, allows to formulate specialized and personalized performance metrics thanks to post-processing and data fusion techniques.

[0317]

[0323] Performance metrics form a complete picture of the biomechanics of the movement (energy, muscle strength, stabilization forces, joint reactions, etc.), and the quality of execution of the exercise by evaluating, over time, information such as rhythm, muscle failures, asymmetries, pattern patterns, and more.

[0318]

[0324] The cloud service, as set out in the disclosure, is used on the one hand as a support tool for big data management useful for training neural networks and saving generated weights, on the other hand, for managing user profiles, which can access the personal area using login credentials.

[0319]

[0325] The graphical interface is a tool to be understood, on the one hand, as a user service to customize workouts, display results in real time and monitor progress, on the other hand, as a technological support for personal trainers and coaches, it is possible to manage the training plan and the progress for a greater number of users in an optimized way.

[0320]

[0326] Based on type of service to be provided, it is possible to make a specialized architectural choice by adapting it and optimizing the operation of the system according to the given context (see FIG. 6, FIG. 7 and FIG. 8). Such choices are also dictated by the economic and technological availability and complexity of the desired service as schematically and illustratively described in FIG. 11.

[0321]

[0327] The edge computing architecture (see FIG. 6) it is preferable for implementations intended for dedicated technological areas, offering a real IT service to specialized technical structures, such as gyms.

[0322]

[0328] Cloud computing architecture (see FIG. 8) is preferable for services intended for the domestic use of technological tools, or in general for environments provided with a good Internet connection (Wi-Fi, optical fiber) which allows to exchange regularly, safely and without packet loss, managing large data flows with limited latency.

[0323]

[0329] The embedded computing architecture (see FIG. 7) is preferable for the manufacture of compact products which can also be used offline, without having limitations on the place of use thereof, logic typically used by wearable devices. This device must be able to store in a buffer the processing carried out by it and then connect a sorting node which, by putting it in communication with the cloud, allows to provide the training evaluations through the graphical interface.

Claims

C L A I M S1. System for counting, recognition and evaluation in real time of biomechanics activities repeated by the user, or furthermore usable for the monitoring finalised to robotic systems control or industrial systems control, comprising or alternatively consisting of: a) a reference device wearable by an individual, integrated in equipment and / or machinery comprising: i. a microcontroller for managing and processing data from the other components; ii. an IMU 6 dof for measuring the inertial signal related to the movement; iii. any other sensors, such as magnetometers, thermometers, pressure sensors, ECG and / or distance sensors; iv. one or more RGB LEDs to provide real-time visual feedback to the user; v. vibration and / or sound components to provide real-time alternative feedback to the user; vi. one or more antennas for wireless communication and association between the device and the user profile; vii. a power supply battery. b) a combination of devices a) may be associated with the same user so as to increase the processing capabilities of the system. Such configurations allow to improve the quality and functionality of the system by adding new performance metrics; c) one or more calculation nodes and / or a sink node in communication with the reference device a) and / or combination b) capable of: i) in the case of configuration b), synchronizing all the devices associated with the user and / or machine through a broadcast signal in the preliminary acquisition and processing step; ii) receiving the encoded data from the device a) or from the configuration b) performing a further control on the input datum; iii) processing the information collected by artificial intelligence to classify the type of biomechanical movement;iv) carrying out data fusion and post-processing techniques which allow to obtain performance metrics; v) sending a return message to the device a) or to the configuration b) containing the result of the recognition; vi) sending the output of the neural network and the signal recorded over time to a cloud node d) which will communicate to the platform e) the final evaluation of the activity through performance metrics. d) a cloud node for managing and / or controlling the information from the processing node c), updating the information of the user profile and providing the data for displaying the performance by communicating with the graphical interface e). In addition, the cloud service will serve for the training and management of neural architectures with the relative storage in suitable databases of the weights of the architectures and of the various training datasets. e) a visualization node designed as web app, desktop app and / or mobile app, which is in communication with the cloud d) and manages the information allowing the visualization of statistics and metrics of evaluation and performance of the activities.

2. System according to claim 1 ), wherein strictly necessary sensors for the device a) are the accelerometer and gyroscope.

3. System according to claims from 1 ) to 2), wherein device a) contains a state flow machine configured to handle the complete execution flow of its functionalities in real time, allowing repeated movements counting, initial positioning automatic recognition, signal segmentation, encoding and sending of data to elaboration node c).

4. System according to the claims from 1 ) to 3), wherein configurations b), eventually composed by a single device a), are generalised to multiple application contexts with additional specifications depending on needs and / or usage requirements.

5. System according to the claims from 1 ) to 4), wherein elaboration node c) contains one or more Neural Networks trained to recognise a determined ensemble of biomechanical movements previously established and specialised.

6. System according to the claims from 1 ) to 5), wherein elaboration node c) is able to send a feedback back to single devices a) and to cloud node d) with consequent action and / or state variation of those devices.

7. System according to the claims from 1 ) to 6), wherein cloud node d) allows big data storage composed by signals received from elaboration node c), and also the Neural Network architectures training and handling, with their relative storage into appropriate databases of both architecture weights and of the various training datasets.

8. System according to the claims from 1 ) to 7), wherein graphic interface e) allows visualisation of statistic metrics and evaluation metrics about personalised performances computed by the systems following activities ensuring monitoring of relative progresses about physical activities plans or even rehabilitative plans, making available and visualisable the history of activities associated to each user.

9. Method for real time counting, recognition and evaluation about biomechanical activities repeated by the user or even usable for monitoring finalised to robotic systems and industrial systems controls, including following phases: a. associating and synchronizing one or more devices a) which, together with the use of data fusion techniques, allows the personalized processing and management of predefined configurations b); b. adjustment and management of the system workflow by means of a state machine that allows the recognition of the initial positioning and the signal segmentation and provides the real-time count of the repetitions; c. restitution of real time feedback to the user on the status of the system by means of light, sound and / or tactile signals, such as counting, device status, recognition, etc. d. creation of the training database of the neural network formed both by really acquired signals starting from the activity of one or more users and by those generated by means of a parametric physical-mathematical model;e. customisation of neural architectures for the analysis of the features extracted from the calculation node c) analysing the signal segmented by the device a). f. specialization of recognition dependant on the sensoristic composition of devices a) and their combination b); g. restitution of the performance metrics customized on the basis of the user profile and obtained by comparing the mathematical modelling of the biomechanical movement and the real acquired signal during the activity.

10. Method according to claim 9), wherein said method is executed through the system according to any of the claims from 1 ) to 8).

11. Method according to claims from 9) to 10), wherein said method involves the use of a physical-mathematical model capable to generate synthetic data of predefined biomechanical movements and perform evaluation metrics and performances associated to those movements.

12. Method according to claims from 9) to 11 ), wherein said method provides the capability to save large amount of data acquired from devices a) employed in the composition of datasets for efficiently and incrementally neural networks training and also reusable in alternative research and development contexts.

13. Method according to claims from 9) to 12), wherein said method provides the capability to create complete and efficient datasets composed of both acquired signals from devices a) and data simulated by the mathematical model guaranteeing robustness in discrimination by having a sufficient variety and quantity of data available from the outset to overcome the limitation of the typical overfitting problem present in the application contexts covered by the invention.

14. A parametric physical and mathematical model that allows generality in the description of biomechanical movements, useful for the autonomous and intensive generation of simulated reference inertial signals as if they were acquired while performing the activity, ensuring a complete and ideal representation of a potentially unlimited number of movements with relative biomechanicalevaluation of the efforts, constraint reactions, stabilization forces and quantities derived from them.

15. A computer software including instructions such that, when executed by a computer, that make that computer execute those phases of the method of any of the claims from 8) to 11 ).

16. System usage according to any of the claims from 1 ) to 8) for the counting, recognition and evaluation of biomechanical movements repeated over time of human limbs and / or for the monitoring finalised to the real time control of robotic systems and industrial systems.

Images