Method for extracting motion feature data

Abstract

The invention provides a method for extracting motion characteristic data. The method comprises the following steps: acquiring kinematics data of a muscle to be detected; establishing a finite element model of the to-be-measured muscle according to the electrical impedance imaging differential voltage signal value of the to-be-measured muscle and the anatomical prior information of the section of the to-be-measured muscle; extracting respective time domain information and frequency domain information in the conductivity data of the plurality of signal nodes from the finite element model; correlation calculation is carried out based on the time domain information, the frequency domain information and the kinematics data, the motion participation degree of each signal node is obtained, and the motion participation degree is used for representing the muscle motion participation degree of each signal node; according to the motion participation degree of each signal node, the finite element model and the conductivity data, motion feature data are obtained, the motion feature data are used for representing spatial position distribution of multiple key muscle units in the to-be-detected muscle, and therefore the morphological features of the to-be-detected muscle can be determined based on the spatial position distribution.

Description

Technical Field

[0001] The present invention relates to the fields of computer technology and wearable human-machine interface technology, and in particular to a method and device for extracting motion feature data, an electronic device and a storage medium. Background Art

[0002] The core function of wearable human-machine interface technology is to provide accurate, timely and robust motion intention recognition decision information for wearable robot control systems. Wearable human-machine interface includes measurement front end, signal measurement system, signal processing module and subsequent intention recognition algorithm module.

[0003] In the human-machine interface technology based on muscle signals, bioimpedance tomography (also known as electrical impedance tomography, EIT) can measure the changes in the conductivity distribution within the anatomical cross-section of the limbs, and then obtain the muscle morphological characteristics, and combine with the subsequent processing algorithm to realize the decoding of the movement intention. Compared with the human-machine interface technology based on surface electromyography signals, the human-machine interface based on bioimpedance tomography can obtain the conductivity distribution of biological tissues below the skin through the combination of current signal excitation-voltage signal measurement, and then extract deep muscle characteristics, broadening the information dimension of the existing technology. However, in the current electrical impedance imaging technology, the solution algorithm is an ill-posed problem, that is, the input data dimension is much smaller than the output data dimension, resulting in low spatial resolution, so that the EIT reconstruction technology cannot identify the deep muscle area and the corresponding signal activity, and the subsequent algorithm of the wearable human-machine interface cannot use the EIT-driven musculoskeletal model method for intention recognition.

[0004] The information richness of the signal for muscle feature extraction determines the overall performance of subsequent intention recognition and human-computer interaction systems. Therefore, a method that can more accurately extract motion feature data is needed. Summary of the invention

[0005] In view of the above problems, the present invention provides a method for extracting motion characteristic data, comprising: obtaining kinematic data of a muscle to be tested, the kinematic data comprising data on changes in the degree of joint activity corresponding to the muscle to be tested over time;

[0006] A finite element model of the muscle to be tested is established according to the electrical impedance imaging differential voltage signal value of the muscle to be tested and the anatomical prior information of the cross section of the muscle to be tested, wherein the finite element model includes a plurality of signal nodes, each of the plurality of signal nodes having conductivity data of conductivity changing with time obtained by solving and reconstructing the electrical impedance imaging differential voltage signal;

[0007] Extracting time domain information and frequency domain information of each of the conductivity data of multiple signal nodes from the finite element model;

[0008] Based on the correlation calculation of time domain information, frequency domain information and kinematic data, the movement participation degree of each signal node is obtained, where the movement participation degree is used to characterize the degree to which each signal node participates in muscle movement;

[0009] The motion characteristic data is obtained according to the motion participation, finite element model and conductivity data of each signal node. The motion characteristic data is used to characterize the spatial position distribution of multiple key muscle units in the muscle to be tested, so as to determine the morphological characteristics of the muscle to be tested based on the spatial position distribution.

[0010] According to an embodiment of the present invention, the concept of movement participation is introduced by combining the actually measured joint activity motion data and the conductivity data obtained by EIT measurement based on a finite element model constructed through anatomical prior information. Then, through the calculation and feature extraction of movement participation, characteristic data related to muscle activity can be obtained, which can realize the identification of the spatial position of deep muscles and improve the spatial resolution of existing bioelectrical impedance imaging technology in muscle identification tasks, so that the user's movement intention can be more accurately identified, and the accuracy and robustness of movement intention recognition can be improved, thereby improving the naturalness and efficiency of human-computer interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0012] Figure 1 A diagram schematically shows an application scenario of a method for extracting motion feature data according to an embodiment of the present invention;

[0013] Figure 2 A flowchart of a method for extracting motion feature data according to an embodiment of the present invention is schematically shown;

[0014] Figure 3 A flowchart of a method for calculating sports participation according to an embodiment of the present invention is schematically shown;

[0015] Figure 4 Schematically shows a flow chart of a method for calculating motion feature data according to an embodiment of the present invention;

[0016] Figure 5 A flowchart of a specific embodiment of the method for extracting motion feature data according to an embodiment of the present invention;

[0017] Figure 6 A schematic diagram of a specific embodiment of extracting motion feature data according to an embodiment of the present invention

[0018] Figure 7 A schematic diagram shows a structural block diagram of a device for extracting motion feature data according to an embodiment of the present invention;

[0019] Figure 8 A block diagram of an electronic device suitable for implementing a method for extracting motion feature data according to an embodiment of the present invention is schematically shown. DETAILED DESCRIPTION

[0020] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0021] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The term "comprising" used herein indicates the existence of features, steps, operations, but does not exclude the existence or addition of one or more other features.

[0022] In the case of using expressions such as "at least one of A, B, and C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.). In the case of using expressions such as "at least one of A, B, or C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, or C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0023] In the process of implementing the present invention, it was found that the basic principle of bioimpedance tomography is to obtain a series of current and voltage signals through a circle of metal electrodes surrounding the surface of the medium by current signal injection and voltage signal measurement, and reconstruct the conductivity distribution inside the medium through a solution algorithm, and the conductivity distribution signal is in the form of an image.

[0024] In general, bioimpedance tomography (or bioelectrical impedance imaging, hereinafter referred to as EIT) technology used for human biological tissue measurement reconstructs the conductivity distribution within the cross section of human anatomy, and the spatial resolution of the conductivity image determines the subsequent effect. However, in the electrical impedance imaging technology, the solution algorithm is an ill-posed problem, that is, the input data dimension is much smaller than the output data dimension, resulting in low spatial resolution. In the current EIT technology, the spatial resolution is generally within 10% of the diameter of the medium to be measured. Related research uses methods based on prior knowledge, deep learning and sparse Bayesian learning to improve the resolution of EIT images and obtain high-resolution reconstruction in specific high-conductivity contrast imaging tasks.

[0025] However, the existing technical solutions still cannot obtain sufficiently accurate spatial resolution. First, from a physiological mechanism point of view, the thickness of muscle fascia is much smaller than the thickness of muscle cross section. For example, the thickness of forearm muscle fascia is only about 0.4 mm, and the spatial scale exceeds the spatial resolution range of EIT. Second, from the perspective of electrode configuration, due to the requirements of system portability, time domain response capability, and computational burden, the number of electrodes of the wearable human-machine interface cannot be too large. Generally, it is an EIT acquisition system with 16 electrodes. Therefore, the independent sampling data dimension does not exceed 16*16=256, which limits the data dimension information of the reconstruction algorithm. Third, due to the requirements of time domain response capability and computational burden of the wearable human-machine interface, the reconstruction algorithm cannot use iterative optimization methods to solve pathological problems. Generally, time-difference EIT (tdEIT) is used. tdEIT reflects the changes of the internal medium over time and is not sensitive to stable disturbances. Therefore, it is more stable than the EIT image of single-frame data imaging. However, most of these algorithms use a single-step solution method, which further limits the spatial resolution of the reconstruction algorithm. These factors together result in the inability of existing EIT reconstruction technology to identify deep muscle areas and corresponding signal activities, and subsequent algorithms of wearable human-computer interfaces are unable to use the EIT-driven musculoskeletal model method for intent recognition.

[0026] Figure 1 An exemplary application scenario 100 to which the method for extracting motion feature data according to an embodiment of the present invention can be applied is schematically shown. It should be noted that: Figure 1 The examples shown are merely examples of application scenarios to which the embodiments of the present invention can be applied, to help those skilled in the art understand the technical content of the present invention, but do not mean that the embodiments of the present invention cannot be used in other devices, systems, environments or scenarios.

[0027] like Figure 1As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0028] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients and / or social platform software, etc. (only as examples).

[0029] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0030] The server 105 may be a server that provides various services, such as a background management server (only an example) that provides support for websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0031] It should be noted that the method for extracting motion feature data provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the method system for extracting motion feature data provided in the embodiment of the present invention can generally be set in the server 105. The method for extracting motion feature data provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the method system for extracting motion feature data provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0032] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0033] The following will be based on Figure 1 The scene described by Figure 2 to Figure 6The method for extracting motion feature data in the disclosed embodiment is described in detail.

[0034] Figure 2 The flowchart of the method for extracting motion feature data according to an embodiment of the present invention is schematically shown.

[0035] like Figure 2 As shown, the method includes operations S201 to S205.

[0036] In operation S201, kinematic data of the muscle to be tested is obtained, wherein the kinematic data includes data on changes in the degree of joint activity corresponding to the muscle to be tested over time.

[0037] According to an embodiment of the present invention, the muscle to be tested is a muscle for which motion feature data is extracted, and may be a muscle with high kinematic correlation. The kinematic data may be obtained by an inertial measurement unit (IMU), a motion capture instrument, infrared optical motion capture technology, artificial intelligence motion analysis software, etc.

[0038] For example, when the kinematic data being monitored is the range of motion of the wrist joint, the muscle to be tested is the forearm muscle. Furthermore, the thickest part of the front of the arm can be selected as the muscle to be tested. Joint mobility refers to the range of motion of the joint during movement, usually measured in angles. Joint mobility can be unidirectional, such as flexion or extension, or multi-directional, such as rotation with multiple degrees of freedom. Data on changes in joint mobility over time means that the joint mobility data is collected dynamically, that is, the joint activity is continuously recorded over a period of time.

[0039] For example, when the monitored motion data is the activity level of the ankle joint, the muscle to be tested may be the calf muscle; when the monitored motion data is the activity level of a finger joint, the muscle to be tested may be the muscle on the finger that controls the shutdown of the finger, etc.

[0040] In some specific embodiments of the present invention, kinematic data of the cross section of the muscle to be measured is collected by an inertial measurement unit (IMU).

[0041] In operation S202, a finite element model of the muscle to be tested is established based on the electrical impedance imaging differential voltage signal value of the muscle to be tested and the anatomical prior information of the cross section of the muscle to be tested, wherein the finite element model includes a plurality of signal nodes, each of the plurality of signal nodes having conductivity data of conductivity changing with time obtained by reconstructing the electrical impedance imaging differential voltage signal.

[0042] According to an embodiment of the present invention, electrical impedance imaging is a non-invasive imaging technique that reconstructs the distribution of electrical impedance in the body by applying a small current on the surface of the body and measuring the voltage change. The differential voltage signal refers to the voltage difference measured between different electrodes when the current is applied, and these signals are used to calculate the conductivity distribution of the tissue. In biological tissues, conductivity is related to factors such as the water content, ion concentration and cell structure of the tissue, so the physiological and pathological state of the tissue can be inferred by measuring conductivity. Finite element model (FEM) refers to a numerical calculation method that divides the cross section of a muscle to be measured into a finite number of small elements and obtains an approximate solution to the entire structure by assembling the solutions of all elements. In the finite element model, a signal node refers to a virtual point used to measure or calculate conductivity data. These nodes are distributed in different locations of the model, and each node is associated with corresponding conductivity data, which can be static or time-varying. Anatomical prior information includes the shape, size, fiber orientation and relative position of the muscle in the body. Anatomical prior information helps to accurately reproduce the geometric characteristics of the muscle in the model. For example, anatomical prior information of the muscle to be tested can be obtained through magnetic resonance imaging and CT imaging technology.

[0043] For example, the forearm finite element model required for EIT image reconstruction can be constructed by referring to the magnetic resonance imaging (MRI) images of the forearm in anatomical theoretical research and the muscle and bone contour data annotated by experts on the MRI images.

[0044] The finite element model includes a plurality of signal nodes, and each signal node has a set of data on the conductivity of the signal node changing with time, which can be specifically expressed as a specific waveform.

[0045] In some specific embodiments of the present invention, the change of conductivity data over time obtained by electrical impedance imaging technology within a certain period of time can be used to dynamically monitor physiological changes in biological tissues, such as muscle activity, blood flow changes, etc., because the change of conductivity can reflect the change of tissue state. Each signal node in the finite element model has a conductivity change over time data, which can also correspond to the movement data of the muscle to be tested.

[0046] In some specific embodiments of the present invention, motion data and conductivity data have the same sampling frequency. The synchronous collection of kinematic data and conductivity data can ensure that the time correspondence between the two is accurate during analysis, and can more intuitively reflect the dynamic relationship between muscle activity and conductivity changes. At the same time, synchronous sampling can improve the accuracy of data analysis, and can compare and analyze the kinematic behavior and electrophysiological characteristics of muscles at the same time point or time period, thereby more accurately identifying muscle activity patterns and physiological changes.

[0047] In operation S203, time domain information and frequency domain information of each of the conductivity data of a plurality of signal nodes are extracted from the finite element model.

[0048] According to an embodiment of the present invention, in the process of constructing a finite element model, time domain information refers to the characteristics of a signal in the time dimension, including the change of a signal over time, specifically refers to the data of the change of conductivity data over time, that is, the curve of the change of conductivity over time. Frequency domain information is the characteristic of a signal in the frequency dimension. By converting a time domain signal to a frequency domain, for example, using Fourier transform, the spectrum of the signal can be obtained, that is, the frequency components contained in the signal and their corresponding amplitude and phase information, specifically refers to the frequency components extracted from the conductivity data and their characteristics. Each signal node has data of the change of the conductivity data of the node over time, from which time domain information can be extracted and frequency domain information can be further obtained.

[0049] In operation S204, correlation calculation is performed based on the time domain information, the frequency domain information and the kinematic data to obtain the movement participation degree of each signal node, wherein the movement participation degree is used to characterize the degree to which each signal node participates in muscle movement.

[0050] According to an embodiment of the present invention, sports participation is a custom parameter used to characterize the degree of participation of a signal node in muscle movement, which can be understood as the activity or contribution of a signal node to muscle movement. By analyzing the time domain and frequency domain information of each signal node, the role and importance of each signal node in muscle activity can be quantified, thereby evaluating its contribution to overall muscle function, which can be specifically expressed as a numerical value of sports participation.

[0051] Specifically, the sports participation can be calculated from two aspects, time domain and frequency domain, so as to finally obtain a sports participation value. For example, in the time domain, the temporal correlation between the node signal and the signal of the sports data is considered, and in the frequency domain, the amplitude response of the node signal on the frequency of the sports signal is considered to obtain the sports participation.

[0052] In operation S205, motion feature data is obtained according to the motion participation, finite element model, and conductivity data of each signal node. The motion feature data is used to characterize the spatial position distribution of multiple key muscle units in the muscle to be tested, so as to determine the morphological characteristics of the muscle to be tested based on the spatial position distribution.

[0053] According to an embodiment of the present invention, motion feature data refers to input data that can be used in wearable human-machine interface technology to perform intention recognition on the EIT-driven musculoskeletal model method, specifically refers to a data set that can characterize muscle motion characteristics, which can be used in wearable human-machine interface technology. In the EIT-driven musculoskeletal model, it can be used to identify the user's motion intention, thereby controlling external devices such as robotic arms or other auxiliary devices.

[0054] The calculation of motion feature data can be based on the distribution of motion participation, wherein the higher the motion participation value is, the greater the response of the node to the motion data. The motion participation can be reflected in the form of a distribution graph on the EIT scanning section or the section of the muscle to be tested. Feature extraction in the distribution graph can obtain feature points, which are used as motion feature data.

[0055] For example, the motion feature data can be expressed as specific coordinates in the constructed finite element model. The motion feature data is specifically expressed as a series of coordinate values, which can be used as input information for the subsequent motion intention recognition algorithm.

[0056] According to an embodiment of the present invention, the concept of movement participation is introduced by combining the actually measured joint activity motion data and the conductivity data obtained by EIT measurement based on a finite element model constructed through anatomical prior information. Then, through the calculation and feature extraction of movement participation, characteristic data related to muscle activity can be obtained, which can realize the identification of the spatial position of deep muscles and improve the spatial resolution of existing bioelectrical impedance imaging technology in muscle identification tasks, so that the user's movement intention can be more accurately identified, and the accuracy and robustness of movement intention recognition can be improved, thereby improving the naturalness and efficiency of human-computer interaction.

[0057] Figure 3 The flowchart of the method for calculating the sports participation degree according to an embodiment of the present invention is schematically shown.

[0058] like Figure 3 As shown, correlation calculation is performed based on time domain information, frequency domain information and kinematic data to obtain the sports participation of each signal node, including operations S301 to S303:

[0059] In operation S301 , a time series correlation coefficient of each signal node is obtained according to the time domain information and kinematic data of the signal node. The time series correlation coefficient is used to characterize the time series correlation between each signal node and the kinematic data.

[0060] According to an embodiment of the present invention, the time series correlation coefficient is a statistic used to measure the correlation or similarity between two time series data, and is used to characterize the degree of correlation between the change in conductivity of the time domain information of each signal node and the degree of joint activity of kinematic data in the time series. Correlation refers to the strength and direction of the relationship or connection between two variables, specifically refers to the correlation between the change in conductivity of the signal node and the degree of joint activity of muscle movement in the time series.

[0061] Specifically, the time series correlation coefficient can be calculated by a predetermined algorithm, such as the Pearson Correlation Coefficient, the Autocorrelation Coefficient, the Windowed Time-Lagged Cross-Correlation (WTLCC), etc.

[0062] In some specific embodiments of the present invention, the time series correlation coefficient can be calculated by acquiring kinematic data, which includes kinematic signal data of multiple consecutive time windows: m = [m1, m2, m3, ..., m T ];

[0063] Get the time domain information of each signal node, including the conductivity signal data of multiple consecutive time windows n = [n1, n2, n3, ..., n T ];

[0064] The timing correlation coefficients of the signal node obtained according to the kinematic data n and the time domain information m include:

[0065] The correlation coefficient is calculated by formula (1);

[0066]

[0067] Among them, in formula (1), ρ is the correlation coefficient of the signal node;

[0068] According to the coefficient ρ, the time series correlation coefficient is obtained through formula (2);

[0069] R=|ρ| (2)

[0070] Wherein, in formula (2), R is the timing correlation coefficient of the signal node.

[0071] Specifically, first obtain a continuous motion signal within a given time T as m = [m1, m2, m3, ..., m T ], synchronously obtain the time domain signal of a node n corresponding to the T time window as n=[n1,n2,n3,...,n T ], and the correlation coefficient ρ is calculated by formula (1).

[0072] The value of ρ calculated by formula (1) ranges from -1 to +1, where -1 indicates that the two sequences are negatively correlated, and +1 indicates that the two sequences are positively correlated. The positive and negative correlations between the motion signal m and the EIT node signal n are both strong correlations, so the correlation between the two sequences is defined as R, which represents the absolute value of ρ. The value of R calculated by formula (2) is used as the time series correlation coefficient.

[0073] Furthermore, a threshold th may be set, for example, th=0.5, and a group of node signals that are strongly correlated with the motion signal may be screened out according to the condition of R>th.

[0074] Operation S302 : obtaining a signal power spectrum of each signal node according to the frequency domain information and kinematic data of the signal node, where the signal power spectrum is used to characterize the amplitude response of each signal node at the frequency of the kinematic data.

[0075] According to an embodiment of the present invention, the signal power spectrum refers to a function that describes the power distribution of the signal at different frequencies, which is the square of the modulus of the signal spectrum, that is, the power of each frequency component. The power spectrum can be used to characterize the amplitude response of the signal of each node at each frequency, that is, the intensity or energy distribution of the signal at different frequencies. The amplitude response refers to the response amplitude of the kinematic data to input signals of different frequencies. The EIT signal is considered to be a power signal, and the power spectrum of each node signal in the EIT image or finite element model is calculated to obtain the amplitude response of the EIT node signal at the frequency of the motion data, which is then used in the next step to calculate the motion participation, that is, to calculate the influence or response degree between each signal node and the motion data from the frequency domain perspective.

[0076] In some specific embodiments of the present invention, obtaining the signal power spectrum of each signal node according to the frequency domain information and kinematic data of the signal node includes calculating using the following formula (3):

[0077]

[0078] Wherein, in formula (3), T represents the signal length of a signal node n, fft(n) represents Fourier transform of a signal of a signal node n, and P(f) is a function of the power spectrum of a signal node n.

[0079] According to the embodiment of the present invention, the power spectrum is the signal power within a unit frequency band, and its derivation formula is relatively complex. The power spectrum of the EIT signal n can be obtained by a direct method, that is, formula (3).

[0080] Operation S303: obtaining the motion participation degree of each signal node according to the time series correlation coefficient and the signal power spectrum of each signal node.

[0081] According to an embodiment of the present invention, the sports participation degree can be used to characterize the degree to which each signal node participates in muscle movement, combining the time series correlation coefficient and the signal power spectrum.

[0082] For example, the weighted average method can be used to take the time series correlation coefficient as the weight to measure the correlation between the signal node and the kinematic data, and the signal power spectrum as an indicator to measure the activity level of the signal node at different frequencies, and the motion participation of each signal node can be calculated by weighted average.

[0083] For example, a comprehensive scoring method can be used to set scoring criteria based on the timing correlation coefficient and signal power spectrum, score each signal node's timing correlation and frequency activity, and then combine these scores to obtain the degree of sports participation.

[0084] In some specific embodiments of the present invention, obtaining the sports participation degree of each signal node according to the time series correlation coefficient and the signal power spectrum of each signal node includes calculating using the following formula (4);

[0085] p=R·P(f motion ) (4)

[0086] In formula (4), R is the time series correlation coefficient of the signal node, P is the function of the power spectrum of the signal node, and f motion is the frequency of the motion data, which is obtained by performing Fourier transform on the motion data and calculating the index corresponding to its extreme point.

[0087] According to an embodiment of the present invention, by calculating the sports participation of each signal node based on time domain information, frequency domain information and kinematic data, the characteristics of the signal nodes in time and frequency can be fully captured. The combined use of the timing correlation coefficient and the signal power spectrum can improve the accurate measurement of the relationship between the signal node and the kinematic data. Frequency domain analysis helps to identify and filter noise and improve the signal-to-noise ratio of the signal, making the assessment of sports participation more reliable, thereby more accurately assessing muscle activity.

[0088] Figure 4 The flowchart of the motion feature data calculation method according to an embodiment of the present invention is schematically shown.

[0089] like Figure 4 As shown, obtaining motion feature data according to the motion participation degree of each signal node includes operations S401 to S405:

[0090] In operation S401 , a conductivity distribution map is obtained by reconstructing the conductivity data through calculation.

[0091] In operation S402 , a sports participation characteristic map is obtained based on the sports participation, the finite element model, and the conductivity distribution map of each signal node.

[0092] Operation S403: down-sample the sports participation feature map to obtain down-sampled image data.

[0093] In operation S404, feature extraction is performed on the motion participation information of each signal node in the down-sampled image data to obtain intermediate feature data.

[0094] Operation S405 : obtaining motion feature data based on the intermediate feature data and the finite element model.

[0095] In operation S401, the conductivity data can be obtained by measuring the differential voltage signal through EIT, and a forward model is established according to physical principles, which describes the relationship between the conductivity distribution and the measured data. For example, the Maxwell equations of the electromagnetic field can be used to describe the relationship between the conductivity and the electromagnetic field. Thus, a conductivity distribution map, i.e., an EIT imaging image, is obtained.

[0096] The reconstruction algorithm adopts the time-difference electrical conductivity imaging technology (tdEIT). The reconstruction algorithm adopts a single-step calculation method based on maximum a posteriori probability. In tdEIT, the reconstruction algorithm is used to reconstruct the conductivity image inside the object based on the measured boundary voltage data.

[0097] In operation S402, a sports participation characteristic graph is generated by analyzing the sports participation of each signal node. This characteristic graph can reflect the sports status of different areas within a specific time. With the conductivity distribution map as a reference, the values ​​calculated from the sports participation of each signal node and the established finite element model, a sports participation characteristic graph based on the sports participation value can be obtained.

[0098] In operation S403, downsampling the motion participation feature map to obtain downsampled image data can be achieved by using simple averaging, pooling, strided convolution, etc. The downsampled image meets the definition of a general two-dimensional image. In the subsequent feature extraction, downsampling can reduce the amount of calculation during model training and inference, improve efficiency, and can help reduce overfitting and improve the generalization ability of the model.

[0099] For example, the EIT forward model uses FEM node coordinates for calculation, and the downsampling module functions as coordinate conversion and downsampling. Given the coordinates of node n as (x, y), its corresponding motion participation can be expressed as p or p(x, y), the EIT scanning section is Ω, (x, y)∈Ω, and the downsampling calculation process is:

[0100]

[0101] Among them, in formula (5), S i Yes(x i ,y i ) is the image value after downsampling,Ω iRepresents (x i ,y i ) in different block areas in the scanning plane centered at i Represents Ω i The number of internal EIT nodes.

[0102] In operation S404, the intermediate feature data may represent a set of values ​​or coordinates or positions that represent a high degree of participation in the movement. For example, feature extraction may be performed by principal component analysis (PCA), linear discriminant analysis (LDA), Gaussian mixture model (GMM), extraction of wave crest points using a second-order difference discriminant method, etc., to obtain the intermediate feature data.

[0103] In some specific embodiments of the present invention, after the motion participation feature map is downsampled, the image can be preprocessed using a compatible general image filtering algorithm. This embodiment uses median filtering and mean filtering methods to denoise the downsampled image. The motion participation in the denoised downsampled image is peak-shaped in the spatial domain. The peak indicates that the point has the greatest response to the motion and the highest degree of participation. Therefore, the peak center of the image needs to be extracted and the peak is described using Gaussian components.

[0104] For example, the second-order difference discriminant method is used to extract the peak points. According to the properties of the peaks on the two-dimensional plane, the center of the peak should satisfy the first-order derivative of the image equal to 0, the second-order derivative of the image is less than 0, and the set of peak centers in the downsampled image Ω peaks It is expressed as shown in the following formula (6):

[0105]

[0106] The image second-order difference peak discrimination algorithm is to obtain the peak set of each row of the image (horizontal peak set), and the peak set of each column of the image (vertical peak set), and then take the intersection of the two to obtain the peak of the image. Its pseudo code is as shown in Algorithm 1.

[0107]

[0108]

[0109] Calculate the peak value Ω of a row or column in the downsampled image of EIT pxi and Ω pyi The method is as follows: for the time series T = [t1, t2, t3, ..., t N ], first solve the first-order forward difference of the sequence, the calculation method is:

[0110] TD i =t i+1 -t i ; i=1,...,N (7)

[0111] Since we only care about the positive and negative of the first-order difference signal, we use -1, 0, and 1 to represent the positive, zero (extreme value), and negative of the TD sequence respectively. Therefore, the signed first-order difference sequence is expressed as:

[0112]

[0113] Traverse the TDS sequence from index N to 1, and set the 0 element to the value of the next element, that is, when TDS i =0, TDS i =TDS i +1. The second-level difference calculation of the TDS sequence is:

[0114] TDD=TDS i+1 -TDS i (9)

[0115] Then the peak position set Ω of sequence T is p For TDD i =-2Next element index:

[0116] Ω p ={i+1|TDD i =-2} (10)

[0117] According to an embodiment of the present invention, the motion feature data is obtained by a nearest neighbor algorithm based on a finite element model with the intermediate feature data as the center and a preset signal node radius r.

[0118] According to an embodiment of the present invention, motion feature data can be understood as a series of coordinates, and the values ​​of this series of coordinates can be used as representative data of the corresponding muscle movement and the corresponding joint activity degree, and used as input values ​​for subsequent algorithms such as motion intention recognition. The peak position is the position in the downsampled image, and the peak position indicates that the movement participation corresponding to the node is the largest, and the nodes around the peak also contain motion-related information. Therefore, the preset signal node radius r can be set based on experience, such as setting it to 3, 4 or 5 signal points, etc., and extracting the node signal in the neighborhood of each peak center with a radius of r as the final feature signal, and then obtaining the motion feature data based on the previously established finite element model and the nearest neighbor algorithm.

[0119] Figure 5 A flowchart of a specific embodiment of the method for extracting motion feature data according to an embodiment of the present invention; Figure 6 The figure is a schematic diagram of a specific embodiment of extracting motion feature data according to an embodiment of the present invention.

[0120] like Figure 5 and Figure 6As shown, the user wears the EIT measurement front end and the inertial measurement unit (IMU) to measure and repeat the forearm wrist joint movements, including wrist extension, wrist neutral and wrist neutral, and synchronously records the wrist joint kinematic data to obtain a waveform data, where the horizontal axis is time and the vertical axis is the degree of joint activity. At the same time, according to the EIT measurement and solution reconstruction technology, a waveform diagram of the conductivity of multiple signal nodes changing with time can be obtained, and the corresponding EIT image obtained by cross-sectional reconstruction of the measured muscle can also be obtained.

[0121] In this embodiment, the EIT measurement system used is 16 channels, and the measurement plane of the EIT electrode is located at the thickest part of the user's forearm; the EIT measurement system uses a four-pole method for current signal excitation (injection) and voltage signal measurement, that is, the current signal injection and voltage signal measurement are completed through adjacent electrodes, so each frame of the EIT signal contains 16 current signal injections (each current signal injection is called a mode), each mode contains 13 pairs of differential voltage signals of adjacent electrodes, so each frame of the EIT signal contains 208 voltage signal values, and the EIT signal frame rate in this embodiment is 100fps. The motion signal measurement uses an inertial measurement unit IMU with a sampling frequency of 100Hz.

[0122] The reconstruction algorithm adopts tdEIT, and the reconstruction algorithm adopts a single-step calculation method based on maximum a posteriori probability. The forward model uses the finite element (FEM) method to discretize the cross-sectional area. This embodiment refers to the CT image of the forearm in the anatomical theory research and the muscle and bone contour data annotated by experts on the CT image to construct the forearm finite element model required for EIT image reconstruction. The number of finite element nodes in this embodiment is 2368.

[0123] Furthermore, the temporal correlation between the node signal and the motion signal is considered in the time domain, and the amplitude response of the node signal at the motion frequency is considered in the frequency domain. After obtaining the motion participation, the motion participation feature map is obtained by combining the EIT image obtained previously. After further downsampling processing, the downsampled image is obtained, and the node signal in the neighborhood with a radius of r around the center of each peak is further extracted as the final feature signal, which can be specifically expressed as Figure 6 Peak-detected EIT image in .

[0124] Based on the above method for extracting motion feature data, the present invention also provides a device for extracting motion feature data. Figure 7 The device is described in detail.

[0125] Figure 7 The structure block diagram of the device for extracting motion feature data according to an embodiment of the present invention is schematically shown.

[0126] like Figure 7 As shown, the device 700 for extracting motion feature data of this embodiment includes an acquisition module 710 , a construction module 720 , an extraction module 730 , a first processing module 740 and a second processing module 750 .

[0127] The acquisition module 710 is used to acquire kinematic data of the muscle to be tested, and the kinematic data includes data on the change of the joint activity degree corresponding to the muscle to be tested over time. In one embodiment, the acquisition module 710 can be used to perform the operation S201 described above, which will not be repeated here.

[0128] The construction module 720 is used to establish a finite element model of the muscle to be tested according to the electrical impedance imaging differential voltage signal value of the muscle to be tested and the anatomical prior information of the cross section of the muscle to be tested, wherein the finite element model includes a plurality of signal nodes, and each of the plurality of signal nodes has conductivity data of conductivity varying with time obtained by solving and reconstructing the electrical impedance imaging differential voltage signal. In one embodiment, the construction module 720 can be used to perform the operation S202 described above, which will not be repeated here.

[0129] The extraction module 730 is used to perform correlation calculation based on the time domain information, the frequency domain information and the kinematic data to obtain the sports participation degree of each signal node, wherein the sports participation degree is used to characterize the degree to which each signal node participates in the muscle movement. In one embodiment, the extraction module 730 can be used to perform the operation S203 described above, which will not be repeated here.

[0130] The first processing module 740 is used to perform correlation calculation based on the time domain information, the frequency domain information and the kinematic data to obtain the sports participation degree of each signal node, wherein the sports participation degree is used to characterize the degree to which each signal node participates in the muscle movement. In one embodiment, the first processing module 740 can be used to perform the operation S204 described above, which will not be repeated here.

[0131] The second processing module 750 is used to obtain motion feature data according to the motion participation, finite element model, and conductivity data of each signal node, and the motion feature data is used to characterize the spatial position distribution of multiple key muscle units in the muscle to be tested, so as to determine the morphological characteristics of the muscle to be tested based on the spatial position distribution. In one embodiment, the second processing module 750 can be used to perform the operation S205 described above, which will not be repeated here.

[0132] According to an embodiment of the present invention, the concept of movement participation is introduced by combining the actually measured joint activity motion data and the conductivity data obtained by EIT measurement based on a finite element model constructed through anatomical prior information. Then, through the calculation and feature extraction of movement participation, characteristic data related to muscle activity can be obtained, which can realize the identification of the spatial position of deep muscles and improve the spatial resolution of existing bioelectrical impedance imaging technology in muscle identification tasks, so that the user's movement intention can be more accurately identified, and the accuracy and robustness of movement intention recognition can be improved, thereby improving the naturalness and efficiency of human-computer interaction.

[0133] According to an embodiment of the present invention, any multiple modules among the acquisition module 710, the construction module 720, the extraction module 730, the first processing module 740 and the second processing module 750 can be combined in one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the acquisition module 710, the construction module 720, the extraction module 730, the first processing module 740 and the second processing module 750 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation modes of software, hardware and firmware or in a suitable combination of any of them. Alternatively, at least one of the acquisition module 710, the construction module 720, the extraction module 730, the first processing module 740 and the second processing module 750 may be at least partially implemented as a computer program module, and when the computer program module is executed, a corresponding function may be performed.

[0134] Figure 8 A block diagram of an electronic device suitable for implementing a method for extracting motion feature data according to an embodiment of the present invention is schematically shown.

[0135] Figure 8 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present invention is schematically shown. Figure 8 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0136] like Figure 8As shown, the electronic device 800 according to an embodiment of the present invention includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 into a random access memory (RAM) 803. The processor 801 may, for example, include a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include an onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0137] In RAM 803, various programs and data required for the operation of electronic device 800 are stored. Processor 801, ROM 802 and RAM 803 are connected to each other via bus 804. Processor 801 performs various operations of the method flow according to the embodiment of the present invention by executing the programs in ROM 802 and / or RAM 803. It should be noted that the program can also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 can also perform various operations of the method flow according to the embodiment of the present invention by executing the programs stored in the one or more memories.

[0138] According to an embodiment of the present invention, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to the bus 804. The electronic device 800 may further include one or more of the following components connected to the input / output (I / O) interface 805: an input portion 806 including a keyboard, a mouse, etc.; an output portion 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 808 including a hard disk, etc.; and a communication portion 809 including a network interface card such as a LAN card, a modem, etc. The communication portion 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the input / output (I / O) interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that the computer program read therefrom is installed into the storage portion 808 as needed.

[0139] According to an embodiment of the present invention, the method flow according to an embodiment of the present invention can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the processor 801, the above-mentioned functions defined in the system of the embodiment of the present invention are executed. According to an embodiment of the present invention, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0140] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0141] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device.

[0142] For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 802 and / or the RAM 803 described above and / or one or more memories other than the ROM 802 and the RAM 803 .

[0143] An embodiment of the present invention also includes a computer program product, which includes a computer program, which contains program code for executing the method provided by the embodiment of the present invention. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the method for extracting motion feature data provided by the embodiment of the present invention.

[0144] When the computer program is executed by the processor 801, the above functions defined in the system / device of the embodiment of the present invention are executed. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0145] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 809, and / or installed from a removable medium 811. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0146] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages, specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, such as Java, C++, python, "C" language or similar programming languages. The program code can be executed completely on the user computing device, partially on the user device, partially on the remote computing device, or completely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0147] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways, even if such a combination or combination is not explicitly recorded in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways. All these combinations and / or combinations fall within the scope of the present invention.

[0148] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A method for extracting motion feature data, wherein: include: Acquiring kinematic data of the muscle to be tested, wherein the kinematic data includes data on changes in the degree of joint activity corresponding to the muscle to be tested over time; A finite element model of the muscle to be tested is established according to the electrical impedance imaging differential voltage signal value of the muscle to be tested and the anatomical prior information of the cross section of the muscle to be tested, wherein the finite element model includes a plurality of signal nodes, each of the plurality of signal nodes has conductivity data of conductivity changing with time obtained by solving and reconstructing the electrical impedance imaging differential voltage signal; Extracting time domain information and frequency domain information of each of the conductivity data of multiple signal nodes from the finite element model; Performing correlation calculation based on the time domain information, the frequency domain information and the kinematic data to obtain the movement participation degree of each signal node, wherein the movement participation degree is used to characterize the degree to which each signal node participates in muscle movement; Motion characteristic data are obtained according to the motion participation degree of each signal node, the finite element model, and the conductivity data. The motion characteristic data are used to characterize the spatial position distribution of multiple key muscle units in the muscle to be tested, so as to determine the morphological characteristics of the muscle to be tested based on the spatial position distribution.

2. The method according to claim 1, wherein: The correlation calculation is performed based on the time domain information, the frequency domain information and the kinematic data to obtain the motion participation degree of each signal node, including: Obtaining a time series correlation coefficient of each signal node according to the time domain information and kinematic data of the signal node, wherein the time series correlation coefficient is used to characterize the time series correlation between each signal node and the kinematic data; Obtaining a signal power spectrum of each signal node according to the frequency domain information and kinematic data of the signal node, wherein the signal power spectrum is used to characterize the amplitude response of each signal node at the frequency of the kinematic data; The motion participation degree of each signal node is obtained according to the time series correlation coefficient and signal power spectrum of each signal node.

3. The method according to claim 1, wherein: The obtaining of motion feature data according to the motion participation degree of each signal node comprises: Obtaining a conductivity distribution map by reconstructing the conductivity data; Based on the sports participation of each signal node, the finite element model, and the conductivity distribution map, a sports participation characteristic map is obtained; Downsampling the sports participation feature map to obtain downsampled image data; Extracting features of the motion participation information of each signal node in the downsampled image data to obtain intermediate feature data; Motion characteristic data are obtained based on the intermediate characteristic data and the finite element model.

4. The method according to claim 3, wherein: The downsampling process of the sports participation feature map to obtain downsampled image data includes: Downsampling the sports participation feature map to obtain pre-processed downsampled image data; The pre-processed down-sampled image data is subjected to noise reduction processing to obtain the down-sampled image data.

5. The method according to claim 3 or 4, wherein: The feature extraction of the motion participation information of each signal node in the downsampled image data to obtain the intermediate feature data includes: The motion participation information of each signal node is subjected to peak extraction through the second-order difference discrimination method to obtain intermediate feature data.

6. The method according to claim 3, wherein: The obtaining of motion characteristic data based on the intermediate characteristic data and the finite element model comprises: Taking the intermediate characteristic data as the center and the preset signal node radius r, the motion characteristic data is obtained through the nearest neighbor algorithm based on the finite element model.

7. The method according to claim 1, wherein: The obtaining of kinematic data of the muscle to be tested comprises: The kinematic data of the muscle cross section to be measured are collected by an inertial measurement unit; The step of establishing a finite element model of the muscle to be tested according to the electrical impedance imaging differential voltage signal value of the muscle to be tested and the anatomical priori information of the cross section of the muscle to be tested comprises: The finite element model is obtained by using a time differential electrical impedance imaging algorithm and a finite element algorithm according to the electrical impedance imaging differential voltage signal value of the muscle to be measured and the anatomical prior information of the cross section of the muscle to be measured.

8. The method according to claim 2, wherein: The kinematic data includes a plurality of continuous time window kinematic signal data: m = [m1, m2, m3, ..., m T ]; The time domain information of each signal node includes a plurality of consecutive time window conductivity signal data n=[n1, n2, n3, ..., n T ]; The timing correlation coefficient of the signal node obtained according to the kinematic data n and the time domain information m includes: The correlation coefficient is calculated by formula (1); Among them, in formula (1), ρ is the correlation coefficient of the signal node; According to the coefficient ρ, the time series correlation coefficient is obtained through formula (2); R=|ρ (2) Wherein, in formula (2), R is the timing correlation coefficient of the signal node.

9. The method according to claim 2 or 8, wherein: The obtaining of the signal power spectrum of each signal node according to the frequency domain information and kinematic data of the signal node includes calculating using the following formula (3); Wherein, T represents the signal length of a signal node n, fft(n) represents Fourier transform of a signal of a signal node n, and P(f) is a function of the power spectrum of a signal node n.

10. The method according to claim 9, wherein: The obtaining of the motion participation degree of each signal node according to the time series correlation coefficient and the signal power spectrum of each signal node includes calculating using the following formula (4); p=R·P(f motion ) (4) Among them, R is the time series correlation coefficient of the signal node, P is the function of the power spectrum of the signal node, and f motion is the frequency of the motion data, which is obtained by performing Fourier transform on the motion data and calculating the index corresponding to its extreme point.

Images