Method for extracting motion feature data

Abstract

The present invention provides a method for extracting motion characteristic data, comprising: obtaining kinematic data of a muscle to be measured; establishing a finite element model of the muscle to be measured based on a differential voltage signal value of electrical impedance imaging of the muscle to be measured and anatomical prior information of a cross-section of the muscle to be measured; extracting time domain information and frequency domain information of each of the conductivity data of multiple signal nodes from the finite element model; performing correlation calculation based on the time domain information, the frequency domain information, and the kinematic data to obtain a motion participation degree of each signal node, wherein the motion participation degree is used to characterize the degree to which each signal node participates in muscle movement; obtaining motion characteristic data based on the motion participation degree of each signal node, the finite element model, and the conductivity data, wherein the motion characteristic data is used to characterize the spatial position distribution of multiple key muscle units in the muscle to be measured, so as to determine the morphological characteristics of the muscle to be measured based on the spatial position distribution.

Description

Technical Field

[0001] The present invention relates to the fields of computer technology and wearable human-machine interface technology, and in particular to a method and device for extracting motion feature data, an electronic device, and a storage medium. Background Art

[0002] The core function of wearable human-machine interface technology is to provide accurate, timely, and robust motion intention recognition decision information for wearable robot control systems. Wearable human-machine interface technology includes a measurement front end, a signal measurement system, a signal processing module, and a subsequent intention recognition algorithm module.

[0003] Among muscle signal-based human-machine interface technologies, bioimpedance tomography (EIT) can measure changes in electrical conductivity distribution within anatomical cross-sections of the limb, thereby obtaining muscle morphological characteristics. This information, combined with subsequent processing algorithms, can be used to decode motor intent. Compared to surface electromyography-based human-machine interface technologies, bioimpedance tomography-based human-machine interfaces can obtain the conductivity distribution of biological tissue below the skin through a combination of current signal excitation and voltage signal measurement. This allows for the extraction of deep muscle features, broadening the information dimension of existing technologies. However, current EIT solution algorithms suffer from an ill-posed problem: the input data dimension is significantly smaller than the output data dimension, resulting in low spatial resolution. Consequently, EIT reconstruction techniques are unable to identify deep muscle regions and corresponding signal activity. Subsequent wearable human-machine interface algorithms are also unable to use EIT-driven musculoskeletal modeling for intent recognition.

[0004] The information richness of the signal for muscle feature extraction determines the overall performance of subsequent intention recognition and human-computer interaction systems. Therefore, a method that can more accurately extract motion feature data is needed. Summary of the Invention

[0005] In view of the above problems, the present invention provides a method for extracting motion feature data, comprising: obtaining kinematic data of a muscle to be measured, the kinematic data including data on changes in joint activity levels corresponding to the muscle to be measured over time;

[0006] A finite element model of the muscle to be tested is established based on the electrical impedance imaging differential voltage signal value of the muscle to be tested and anatomical prior information of the cross section of the muscle to be tested, wherein the finite element model includes a plurality of signal nodes, each of the plurality of signal nodes having conductivity data of conductivity changing with time obtained by reconstructing the electrical impedance imaging differential voltage signal;

[0007] Extracting time domain information and frequency domain information of conductivity data of multiple signal nodes from the finite element model;

[0008] Correlation calculations are performed based on time domain information, frequency domain information, and kinematic data to obtain the degree of motor participation of each signal node, where the degree of motor participation is used to characterize the degree to which each signal node participates in muscle movement.

[0009] Motion feature data is obtained based on the motion participation, finite element model, and conductivity data of each signal node. The motion feature data is used to characterize the spatial position distribution of multiple key muscle units in the muscle to be tested, so as to determine the morphological characteristics of the muscle to be tested based on the spatial position distribution.

[0010] According to an embodiment of the present invention, the concept of movement participation is introduced by combining the actually measured joint movement motion data and the conductivity data obtained by EIT measurement based on a finite element model constructed by anatomical prior information. Then, through the calculation and feature extraction of movement participation, feature data related to muscle activity can be obtained, which can realize the identification of the spatial position of deep muscles and improve the spatial resolution of existing bioelectrical impedance imaging technology in muscle identification tasks, so that the user's movement intention can be identified more accurately, and the accuracy and robustness of movement intention recognition can be improved, thereby improving the naturalness and efficiency of human-computer interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0012] Figure 1 A diagram schematically illustrates an application scenario of a method for extracting motion feature data according to an embodiment of the present invention;

[0013] Figure 2 A flowchart of a method for extracting motion feature data according to an embodiment of the present invention is schematically shown;

[0014] Figure 3 A flowchart of a method for calculating sports participation according to an embodiment of the present invention is schematically shown;

[0015] Figure 4 A flowchart of a method for calculating motion feature data according to an embodiment of the present invention is schematically shown;

[0016] Figure 5 A flowchart of a specific embodiment of the method for extracting motion feature data according to an embodiment of the present invention;

[0017] Figure 6 A schematic diagram of a specific embodiment of extracting motion feature data according to an embodiment of the present invention

[0018] Figure 7 A schematic block diagram of a device for extracting motion feature data according to an embodiment of the present invention is shown;

[0019] Figure 8 The block diagram schematically shows an electronic device suitable for implementing the method for extracting motion feature data according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the detailed description below, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0021] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The term "comprising" used herein indicates the presence of features, steps, operations, but does not exclude the presence or addition of one or more other features.

[0022] When expressions such as “at least one of A, B, and C, etc.” are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (e.g., “a system having at least one of A, B, and C” should include but is not limited to systems having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, and C, etc.). When expressions such as “at least one of A, B, or C, etc.” are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (e.g., “a system having at least one of A, B, or C” should include but is not limited to systems having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, and C, etc.).

[0023] In the process of implementing the present invention, it was found that the basic principle of bioimpedance tomography is to obtain a series of current and voltage signals through a circle of metal electrodes surrounding the surface of the medium by current signal injection and voltage signal measurement, and reconstruct the conductivity distribution inside the medium through a solution algorithm. The conductivity distribution signal is in the form of an image.

[0024] Bioimpedance tomography (EIT), a technique used for measuring human tissue, reconstructs the conductivity distribution within a cross-section of human anatomy. The spatial resolution of the conductivity image determines subsequent performance. However, the solution algorithm for EIT is an ill-posed problem, where the input data dimension is much smaller than the output data dimension, resulting in low spatial resolution. Currently, EIT typically has a spatial resolution within 10% of the diameter of the medium being measured. Research is currently using methods based on prior knowledge, deep learning, and sparse Bayesian learning to improve the resolution of EIT images, achieving high-resolution reconstruction in specific high-contrast conductivity imaging tasks.

[0025] However, existing technical solutions still cannot achieve sufficiently accurate spatial resolution. First, from a physiological perspective, the thickness of muscle fascia is much smaller than the thickness of the muscle cross-section. For example, the thickness of forearm muscle fascia is only about 0.4 mm, which exceeds the spatial resolution range of EIT. Second, due to the requirements of system portability, temporal response capability, and computational burden, the number of electrodes in wearable human-machine interfaces cannot be too large. Generally, an EIT acquisition system with 16 electrodes is used. Therefore, the independent sampling data dimension does not exceed 16*16=256, which limits the data dimensionality of the reconstruction algorithm. Third, due to the temporal response capability and computational burden requirements of wearable human-machine interfaces, the reconstruction algorithm cannot use iterative optimization methods to solve ill-conditioned problems. In general, time-difference EIT (tdEIT) is used. tdEIT reflects the changes in the internal medium over time and is not sensitive to stable perturbations. Therefore, it is more stable than EIT images based on single-frame data imaging. However, most such algorithms use a single-step solution method, which further limits the spatial resolution of the reconstruction algorithm. These factors together lead to the inability of existing EIT reconstruction technology to identify deep muscle areas and corresponding signal activities, and subsequent algorithms of wearable human-computer interfaces are unable to use EIT-driven musculoskeletal model methods for intent recognition.

[0026] Figure 1 The following schematically illustrates an exemplary application scenario 100 in which the method for extracting motion feature data according to an embodiment of the present invention can be applied. Figure 1 The examples shown are merely examples of application scenarios in which the embodiments of the present invention can be applied, to help those skilled in the art understand the technical content of the present invention, but do not mean that the embodiments of the present invention cannot be used in other devices, systems, environments or scenarios.

[0027] like Figure 1As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used as a medium for providing a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0028] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software (for example only).

[0029] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0030] The server 105 may be a server that provides various services, such as a background management server (for example only) that supports websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process received data such as user requests, and feed back processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0031] It should be noted that the method for extracting motion feature data provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the method system for extracting motion feature data provided in the embodiment of the present invention can generally be set in the server 105. The method for extracting motion feature data provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the method system for extracting motion feature data provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0032] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0033] The following will be based on Figure 1 The scene described by Figures 2 to 6The method for extracting motion feature data according to the disclosed embodiment is described in detail.

[0034] Figure 2 The flowchart of the method for extracting motion feature data according to an embodiment of the present invention is schematically shown.

[0035] like Figure 2 As shown, the method includes operations S201 to S205.

[0036] In operation S201, kinematic data of the muscle to be measured is obtained, wherein the kinematic data includes data on changes in the degree of joint activity corresponding to the muscle to be measured over time.

[0037] According to an embodiment of the present invention, the muscle to be measured is a muscle for which motion characteristic data is extracted, and can be a muscle with high kinematic relevance. Kinematic data can be obtained by using an inertial measurement unit (IMU), a motion capture instrument, infrared optical motion capture technology, artificial intelligence motion analysis software, etc.

[0038] For example, when the kinematic data being monitored is the range of motion of the wrist joint, the muscle to be measured is the forearm muscle. Furthermore, the thickest part of the front of the arm can be selected as the muscle to be measured. The range of motion of a joint refers to the range of motion of the joint during movement, usually measured in angles. The range of motion of a joint can be unidirectional, such as flexion or extension, or multi-directional, such as rotation with multiple degrees of freedom. The data on the change in the degree of joint motion over time means that the data on the degree of joint motion is collected dynamically, that is, the joint activity is continuously recorded over a period of time.

[0039] For example, when the monitored motion data is the degree of ankle joint activity, the muscle to be tested may be the calf muscle; when the monitored motion data is the degree of activity of a finger joint, the muscle to be tested may be the muscle on the finger that controls the shutdown of the finger, etc.

[0040] In some specific embodiments of the present invention, kinematic data of the muscle cross section to be measured is collected by an inertial measurement unit (IMU).

[0041] In operation S202, a finite element model of the muscle to be measured is established based on the electrical impedance imaging differential voltage signal value of the muscle to be measured and the anatomical prior information of the cross section of the muscle to be measured, wherein the finite element model includes multiple signal nodes, and each of the multiple signal nodes has conductivity data of conductivity changing with time obtained by reconstructing the electrical impedance imaging differential voltage signal.

[0042] According to an embodiment of the present invention, electrical impedance imaging is a non-invasive imaging technique that reconstructs the distribution of electrical impedance in the body by applying a small current on the surface of the body and measuring the voltage change. The differential voltage signal refers to the voltage difference measured between different electrodes when the current is applied, and these signals are used to calculate the conductivity distribution of the tissue. In biological tissue, conductivity is related to factors such as the water content, ion concentration and cell structure of the tissue, so the physiological and pathological state of the tissue can be inferred by measuring conductivity. Finite element model (FEM) refers to a numerical calculation method that divides the cross section of a muscle to be measured into a finite number of small elements and obtains an approximate solution to the entire structure by assembling the solutions of all elements. In the finite element model, signal nodes refer to virtual points used to measure or calculate conductivity data. These nodes are distributed at different locations in the model, and each node is associated with corresponding conductivity data, which can be static or time-varying. Anatomical prior information includes the shape, size, fiber orientation, and relative position of the muscle in the body. Anatomical prior information helps to accurately reproduce the geometric characteristics of the muscle in the model. For example, anatomical prior information of the muscle to be tested can be obtained through magnetic resonance imaging and CT imaging technology.

[0043] For example, the forearm finite element model required for EIT image reconstruction can be constructed by referring to the magnetic resonance imaging (MRI) images of the forearm in anatomical theoretical research and the muscle and bone contour data annotated by experts on the MRI images.

[0044] The finite element model includes multiple signal nodes, and each signal node has a set of data on the conductivity of the signal node changing with time, which can be specifically expressed as a specific waveform.

[0045] In some embodiments of the present invention, the temporal variation of electrical conductivity data obtained through electrical impedance tomography (EIT) can be used to dynamically monitor physiological changes in biological tissues, such as muscle activity and blood flow, because changes in conductivity can reflect changes in tissue state. Each signal node in the finite element model has a temporal variation in conductivity data, which can also correspond to the motion data of the muscle being measured.

[0046] In some specific embodiments of the present invention, motion data and conductivity data have the same sampling frequency. The synchronous collection of kinematic data and conductivity data can ensure that the time correspondence between the two is accurate during analysis, and can more intuitively reflect the dynamic relationship between muscle activity and conductivity changes. At the same time, synchronous sampling can improve the accuracy of data analysis, and can compare and analyze the kinematic behavior and electrophysiological characteristics of muscles at the same time point or time period, thereby more accurately identifying muscle activity patterns and physiological changes.

[0047] In operation S203 , time domain information and frequency domain information of each of the plurality of signal node conductivity data are extracted from the finite element model.

[0048] According to an embodiment of the present invention, during the construction of a finite element model, time domain information refers to the characteristics of the signal in the time dimension, including how the signal changes over time. Specifically, it refers to the conductivity data changing over time, i.e., the conductivity changing over time curve. Frequency domain information refers to the characteristics of the signal in the frequency dimension. By converting the time domain signal to the frequency domain, for example, using a Fourier transform, the signal's spectrum can be obtained, i.e., the frequency components contained in the signal and their corresponding amplitude and phase information. Specifically, it refers to the frequency components extracted from the conductivity data and their characteristics. Each signal node has data on the conductivity data changing over time for that node, from which time domain information can be extracted and further frequency domain information can be obtained.

[0049] In operation S204, correlation calculation is performed based on the time domain information, the frequency domain information, and the kinematic data to obtain the movement participation degree of each signal node, wherein the movement participation degree is used to characterize the degree to which each signal node participates in muscle movement.

[0050] According to an embodiment of the present invention, exercise participation is a custom parameter used to characterize the degree to which a signal node participates in muscle movement. It can be understood as the activity level or contribution of a signal node to muscle movement. By analyzing the time domain and frequency domain information of each signal node, the role and importance of each signal node in muscle activity can be quantified, thereby evaluating its contribution to overall muscle function, which can be specifically expressed as a numerical value of exercise participation.

[0051] Specifically, exercise participation can be calculated from both the time domain and the frequency domain, ultimately yielding a numerical value for exercise participation. For example, in the time domain, the temporal correlation between the node signal and the motion data signal is considered, while in the frequency domain, the amplitude response of the node signal to the frequency of the motion signal is considered to obtain exercise participation.

[0052] In operation S205, motion feature data is obtained based on the motion participation, finite element model, and conductivity data of each signal node. The motion feature data is used to characterize the spatial position distribution of multiple key muscle units in the muscle to be tested, so as to determine the morphological characteristics of the muscle to be tested based on the spatial position distribution.

[0053] According to an embodiment of the present invention, motion feature data refers to input data that can be used in wearable human-machine interface technology to perform intention recognition on the EIT-driven musculoskeletal model method, specifically refers to a data set that can characterize muscle motion characteristics, which can be used in wearable human-machine interface technology. In the EIT-driven musculoskeletal model, it can be used to identify the user's motion intention, thereby controlling external devices such as robotic arms or other auxiliary devices.

[0054] The calculation of motion feature data can be based on the distribution of motion participation, where the higher the motion participation value is, the greater the response of the node to the motion data. The motion participation can be reflected in the form of a distribution graph on the EIT scanning section or the section of the muscle to be tested. Feature extraction in the distribution graph can obtain feature points, which are used as motion feature data.

[0055] For example, the motion feature data can be expressed as specific coordinates in the constructed finite element model. The motion feature data is specifically expressed as a series of coordinate values, which can be used as input information for the subsequent motion intention recognition algorithm.

[0056] According to an embodiment of the present invention, the concept of movement participation is introduced by combining the actually measured joint movement motion data and the conductivity data obtained by EIT measurement based on a finite element model constructed by anatomical prior information. Then, through the calculation and feature extraction of movement participation, feature data related to muscle activity can be obtained, which can realize the identification of the spatial position of deep muscles and improve the spatial resolution of existing bioelectrical impedance imaging technology in muscle identification tasks, so that the user's movement intention can be identified more accurately, and the accuracy and robustness of movement intention recognition can be improved, thereby improving the naturalness and efficiency of human-computer interaction.

[0057] Figure 3 The flowchart of the method for calculating sports participation according to an embodiment of the present invention is schematically shown.

[0058] like Figure 3 As shown, performing correlation calculation based on time domain information, frequency domain information and kinematic data to obtain the sports participation degree of each signal node includes operations S301 to S303:

[0059] In operation S301 , a time series correlation coefficient of each signal node is obtained according to the time domain information and kinematic data of the signal node. The time series correlation coefficient is used to characterize the time series correlation between each signal node and the kinematic data.

[0060] According to an embodiment of the present invention, the time series correlation coefficient is a statistic used to measure the correlation or similarity between two time series data sets. It is used to characterize the degree of correlation between the changes in the conductivity of the time domain information of each signal node and the degree of joint activity in the kinematic data over the time series. Correlation refers to the strength and direction of the relationship or connection between two variables, specifically the correlation between the changes in the conductivity of a signal node and the degree of joint activity in muscle movement over the time series.

[0061] Specifically, the time series correlation coefficient can be calculated using a predetermined algorithm, such as the Pearson Correlation Coefficient, the Autocorrelation Coefficient, the Windowed Time-Lagged Cross-Correlation (WTLCC), and the like.

[0062] In some specific embodiments of the present invention, the time series correlation coefficient can be calculated by obtaining kinematic data, which includes kinematic signal data of multiple consecutive time windows: m = [m1, m2, m3, ..., m T ];

[0063] Get the time domain information of each signal node, including the conductivity signal data of multiple consecutive time windows n=[n1,n2,n3,...,n T ];

[0064] The timing correlation coefficients of the signal node obtained based on the kinematic data n and the time domain information m include:

[0065] The correlation coefficient was calculated by formula (1);

[0066]

[0067] Wherein, in formula (1), ρ is the correlation coefficient of the signal node;

[0068] According to the coefficient ρ, the time series correlation coefficient is obtained through formula (2);

[0069] R=|ρ| (2)

[0070] Wherein, in formula (2), R is the timing correlation coefficient of the signal node.

[0071] Specifically, first obtain a continuous motion signal within a given time T as m = [m1, m2, m3, ..., m T ], synchronously obtain the time domain signal of a node n corresponding to the T time window as n=[n1,n2,n3,...,n T ], and the correlation coefficient ρ is calculated by formula (1).

[0072] The value of ρ calculated by equation (1) ranges from -1 to +1, where -1 indicates a negative correlation between the two sequences and +1 indicates a positive correlation between the two sequences. Both positive and negative correlations between motion signal m and EIT node signal n are strong, so the correlation between the two sequences is defined as R, which represents the absolute value of ρ. The value of R calculated by equation (2) is used as the time series correlation coefficient.

[0073] Furthermore, a threshold th may be set, for example, th=0.5, and a group of node signals that are strongly correlated with the motion signal may be screened out according to the condition of R>th.

[0074] In operation S302 , a signal power spectrum of each signal node is obtained according to the frequency domain information and kinematic data of the signal node. The signal power spectrum is used to characterize the amplitude response of each signal node at the frequency of the kinematic data.

[0075] According to an embodiment of the present invention, the signal power spectrum refers to a function that describes the power distribution of the signal at different frequencies. It is the square of the modulus of the signal spectrum, that is, the power of each frequency component. The power spectrum can be used to characterize the amplitude response of the signal of each node at each frequency, that is, the intensity or energy distribution of the signal at different frequencies. The amplitude response refers to the response amplitude of the kinematic data to input signals of different frequencies. The EIT signal is considered to be a power signal. The power spectrum of each node signal in the EIT image or finite element model is calculated to obtain the amplitude response of the EIT node signal at the frequency of the motion data, which is then used in the next step to calculate the motion participation, that is, to calculate the influence or response degree between each signal node and the motion data from the frequency domain perspective.

[0076] In some specific embodiments of the present invention, obtaining the signal power spectrum of each signal node according to the frequency domain information and kinematic data of the signal node includes calculating using the following formula (3):

[0077]

[0078] Wherein, in formula (3), T represents the signal length of a signal node n, fft(n) represents Fourier transform of the signal of a signal node n, and P(f) is a function of the power spectrum of a signal node n.

[0079] According to an embodiment of the present invention, the power spectrum is the signal power within a unit frequency band, and its derivation formula is relatively complex. The power spectrum of the EIT signal n can be obtained by a direct method, that is, formula (3).

[0080] In operation S303 , the motion participation degree of each signal node is obtained according to the time series correlation coefficient and the signal power spectrum of each signal node.

[0081] According to an embodiment of the present invention, the movement participation degree can be used to characterize the degree to which each signal node participates in muscle movement, combining the time series correlation coefficient and the signal power spectrum.

[0082] For example, the weighted average method can be used to take the time series correlation coefficient as the weight to measure the correlation between the signal node and the kinematic data, and the signal power spectrum as an indicator to measure the activity level of the signal node at different frequencies, and the motion participation of each signal node can be calculated by weighted average.

[0083] For example, a comprehensive scoring method can be used to set scoring criteria based on the temporal correlation coefficient and signal power spectrum, score each signal node's temporal correlation and frequency activity, and then combine these scores to obtain the degree of sports participation.

[0084] In some specific embodiments of the present invention, obtaining the sports participation degree of each signal node according to the time series correlation coefficient and the signal power spectrum of each signal node includes calculating using the following formula (4);

[0085] p=R·P(f motion ) (4)

[0086] Wherein, in formula (4), R is the time series correlation coefficient of the signal node, P is the function of the power spectrum of the signal node, and f motion is the frequency of the motion data, which is obtained by performing Fourier transform on the motion data and calculating the index corresponding to its extreme point.

[0087] According to an embodiment of the present invention, by calculating the sports participation of each signal node based on time domain information, frequency domain information and kinematic data, the characteristics of the signal node in time and frequency can be fully captured. The combined use of the timing correlation coefficient and the signal power spectrum can improve the accurate measurement of the relationship between the signal node and the kinematic data. Frequency domain analysis helps to identify and filter noise, improve the signal-to-noise ratio of the signal, and make the assessment of sports participation more reliable, thereby more accurately assessing muscle activity.

[0088] Figure 4 The flowchart of the motion feature data calculation method according to an embodiment of the present invention is schematically shown.

[0089] like Figure 4 As shown, obtaining motion feature data according to the motion participation of each signal node includes operations S401 to S405:

[0090] In operation S401 , a conductivity distribution map is obtained by reconstructing the conductivity data through calculation.

[0091] In operation S402 , a sports participation characteristic map is obtained based on the sports participation, the finite element model, and the conductivity distribution map of each signal node.

[0092] In operation S403 , down-sampling is performed on the sports participation feature map to obtain down-sampled image data.

[0093] In operation S404 , feature extraction is performed on the motion participation information of each signal node in the downsampled image data to obtain intermediate feature data.

[0094] In operation S405 , motion feature data is obtained based on the intermediate feature data and the finite element model.

[0095] In operation S401, conductivity data can be obtained by measuring a differential voltage signal using EIT. Based on physical principles, a forward model is established that describes the relationship between the conductivity distribution and the measured data. For example, Maxwell's equations for electromagnetic fields can be used to describe the relationship between conductivity and the electromagnetic field. This generates a conductivity distribution map, i.e., an EIT imaging image.

[0096] The reconstruction algorithm uses the time-differential conductivity imaging technology (tdEIT). The reconstruction algorithm adopts a single-step calculation method based on the maximum a posteriori probability. In tdEIT, the reconstruction algorithm is used to reconstruct the conductivity image inside the object based on the measured boundary voltage data.

[0097] In operation S402, an exercise participation feature map is generated by analyzing the exercise participation level of each signal node. This feature map can reflect the exercise status of different areas within a specific time period. Using the conductivity distribution map as a reference, the calculated exercise participation level values ​​for each signal node, and the established finite element model, an exercise participation feature map based on the exercise participation level values ​​can be generated.

[0098] In operation S403, downsampling the motion participation feature map to obtain downsampled image data can be achieved using methods such as simple averaging, pooling, and strided convolution. The downsampled image conforms to the definition of a general two-dimensional image. In subsequent feature extraction, downsampling can reduce the computational complexity during model training and inference, improving efficiency. Downsampling can also help reduce overfitting and improve the generalization ability of the model.

[0099] For example, the EIT forward model uses FEM node coordinates for calculations, and the downsampling module performs coordinate conversion and downsampling. Given the coordinates of node n as (x, y), its corresponding motion participation can be expressed as p or p(x, y), and the EIT scanning cross section is Ω, (x, y)∈Ω. The downsampling calculation process is:

[0100]

[0101] In formula (5), S i Yes (x i ,y i ) after downsampling,Ω iRepresents (x i ,y i ) in different block areas in the scanning plane as the center, K i Represents Ω i The number of internal EIT nodes.

[0102] In operation S404, the intermediate feature data may represent a set of numerical values, coordinates, or positions that are highly associated with the degree of participation in the exercise. For example, feature extraction may be performed using principal component analysis (PCA), linear discriminant analysis (LDA), Gaussian mixture model (GMM), or peak point extraction using a second-order difference discriminant method to obtain the intermediate feature data.

[0103] In some specific embodiments of the present invention, after downsampling the motion participation feature map, the image can be preprocessed using a compatible general image filtering algorithm. This embodiment uses median filtering and mean filtering to denoise the downsampled image. The motion participation in the denoised downsampled image exhibits a peak-like shape in the spatial domain. The peak indicates the point with the greatest response to motion and the highest degree of participation. Therefore, the peak center of the image needs to be extracted and described using Gaussian components.

[0104] For example, the second-order difference discriminant method is used to extract the peak point. According to the properties of the peak on the two-dimensional plane, the center of the peak should satisfy the first-order derivative of the image equal to 0 and the second-order derivative of the image less than 0. The set of peak centers in the downsampled image Ω peaks It is expressed as shown in the following formula (6):

[0105]

[0106] The image second-order difference peak discrimination algorithm is to obtain the peak set of each row of the image (horizontal peak set), and the peak set of each column of the image (vertical peak set), and then take the intersection of the two to obtain the peak of the image. Its pseudo code is shown in Algorithm 1.

[0107]

[0108]

[0109] Calculate the peak value Ω of a row or column in the downsampled image of EIT pxi and Ω pyi The method is as follows: for the time series T = [t1, t2, t3, ..., t N ], first solve the first-order forward difference of the sequence, the calculation method is:

[0110] TD i =t i+1 -t i ;i=1,...,N (7)

[0111] Since we only care about the positive and negative values ​​of the first-order difference signal, we use -1, 0, and 1 to represent the positive, zero (extreme value), and negative values ​​of the TD sequence respectively. Therefore, the signed first-order difference sequence is expressed as:

[0112]

[0113] Traverse the TDS sequence from index N to 1 and set the 0 element to the next element value, that is, when TDS i =0, TDS i =TDS i +1. The second-level difference calculation of the TDS sequence is:

[0114] TDD=TDS i+1 -TDS i (9)

[0115] Then the peak position set Ω of sequence T is p For TDD i =-2Next element index:

[0116] Ω p ={i+1|TDD i =-2} (10)

[0117] According to an embodiment of the present invention, the motion feature data is obtained by a nearest neighbor algorithm based on a finite element model with the intermediate feature data as the center and a preset signal node radius r.

[0118] According to an embodiment of the present invention, motion feature data can be understood as a series of coordinates, and the values ​​of this series of coordinates can be used as representative data of the corresponding muscle movement and the corresponding joint activity degree, and used as input values ​​for subsequent algorithms such as motion intention recognition. The peak position is the position in the downsampled image, and the peak position indicates that the motion participation corresponding to the node is the largest, and the nodes around the peak also contain motion-related information. Therefore, the preset signal node radius r can be set based on experience, such as setting it to 3, 4 or 5 signal points, etc., and extracting the node signal in the neighborhood of each peak center with a radius of r as the final feature signal, and then obtaining the motion feature data based on the previously established finite element model and nearest neighbor algorithm.

[0119] Figure 5 A flowchart of a specific embodiment of the method for extracting motion feature data according to an embodiment of the present invention; Figure 6 The figure is a schematic diagram of a specific embodiment of extracting motion feature data according to an embodiment of the present invention.

[0120] like Figure 5 and Figure 6As shown, a user wearing an EIT measurement front-end and an inertial measurement unit (IMU) measures repeated forearm wrist movements, including wrist extension, wrist neutral, and wrist neutral. This synchronously records wrist kinematic data, generating a waveform with time as the horizontal axis and joint motion as the vertical axis. Simultaneously, using EIT measurement and reconstruction techniques, waveforms of the conductivity of multiple signal nodes over time are generated. Correspondingly, EIT images are reconstructed from the cross-section of the muscle being measured by EIT.

[0121] In this embodiment, the EIT measurement system used is a 16-channel system, with the EIT electrode measurement plane located at the thickest part of the user's forearm. The EIT measurement system employs a four-pole method for current signal excitation (injection) and voltage signal measurement. This means that both current signal injection and voltage signal measurement are performed via adjacent electrodes. Therefore, each EIT signal frame contains 16 current signal injections (each current signal injection is called a pattern), and each pattern contains differential voltage signals from 13 pairs of adjacent electrodes. Therefore, each EIT signal frame contains 208 voltage signal values. The EIT signal frame rate in this embodiment is 100 fps. Motion signal measurement utilizes an inertial measurement unit (IMU) with a sampling frequency of 100 Hz.

[0122] The reconstruction algorithm uses tdEIT, and the reconstruction algorithm adopts a single-step calculation method based on maximum a posteriori probability. The forward model uses the finite element (FEM) method to discretize the cross-sectional area. This embodiment refers to the CT image of the forearm in anatomical theoretical research and the muscle and bone contour data annotated by experts on the CT image to construct the forearm finite element model required for EIT image reconstruction. The number of finite element nodes in this embodiment is 2368.

[0123] Furthermore, the temporal correlation between the node signal and the motion signal is considered in the time domain, and the amplitude response of the node signal at the motion frequency is considered in the frequency domain. After obtaining the motion participation, the motion participation feature map is obtained by combining it with the previously obtained EIT image. After further downsampling, the downsampled image is obtained, and the node signal in the neighborhood with a radius of r around the center of each peak is further extracted as the final feature signal, which can be specifically expressed as Figure 6 Peak-detected EIT images in .

[0124] Based on the above method for extracting motion feature data, the present invention also provides a device for extracting motion feature data. Figure 7 The device is described in detail.

[0125] Figure 7 The structure block diagram of the device for extracting motion feature data according to an embodiment of the present invention is schematically shown.

[0126] like Figure 7 As shown, the motion feature data extraction device 700 of this embodiment includes an acquisition module 710 , a construction module 720 , an extraction module 730 , a first processing module 740 and a second processing module 750 .

[0127] The acquisition module 710 is used to acquire kinematic data of the muscle to be measured, and the kinematic data includes data on the change of the joint activity degree corresponding to the muscle to be measured over time. In one embodiment, the acquisition module 710 can be used to perform the operation S201 described above, which will not be repeated here.

[0128] Construction module 720 is used to establish a finite element model of the muscle to be tested based on the EIT differential voltage signal value of the muscle to be tested and anatomical prior information of the cross-section of the muscle to be tested, wherein the finite element model includes multiple signal nodes, each of which has conductivity data of a conductivity varying over time obtained by reconstructing the EIT differential voltage signal. In one embodiment, construction module 720 can be used to perform operation S202 described above, which will not be repeated here.

[0129] Extraction module 730 is configured to perform correlation calculations based on the time domain information, frequency domain information, and kinematic data to obtain a motor participation degree for each signal node, where the motor participation degree represents the degree to which each signal node participates in muscle movement. In one embodiment, extraction module 730 may be configured to perform operation S203 described above, which will not be further described herein.

[0130] The first processing module 740 is configured to perform a correlation calculation based on the time domain information, the frequency domain information, and the kinematic data to obtain a motor participation degree for each signal node, where the motor participation degree is used to characterize the degree to which each signal node participates in muscle movement. In one embodiment, the first processing module 740 can be configured to perform operation S204 described above, which will not be further described herein.

[0131] The second processing module 750 is configured to obtain motion feature data based on the motion participation, finite element model, and conductivity data of each signal node. The motion feature data is used to characterize the spatial position distribution of multiple key muscle units in the muscle to be tested, so as to determine the morphological characteristics of the muscle to be tested based on the spatial position distribution. In one embodiment, the second processing module 750 can be configured to perform operation S205 described above, which will not be further described here.

[0132] According to an embodiment of the present invention, the concept of movement participation is introduced by combining the actually measured joint movement motion data and the conductivity data obtained by EIT measurement based on a finite element model constructed by anatomical prior information. Then, through the calculation and feature extraction of movement participation, feature data related to muscle activity can be obtained, which can realize the identification of the spatial position of deep muscles and improve the spatial resolution of existing bioelectrical impedance imaging technology in muscle identification tasks, so that the user's movement intention can be identified more accurately, and the accuracy and robustness of movement intention recognition can be improved, thereby improving the naturalness and efficiency of human-computer interaction.

[0133] According to an embodiment of the present invention, any multiple modules among the acquisition module 710, the construction module 720, the extraction module 730, the first processing module 740 and the second processing module 750 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the acquisition module 710, the construction module 720, the extraction module 730, the first processing module 740 and the second processing module 750 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation modes of software, hardware and firmware or in an appropriate combination of any of them. Alternatively, at least one of the acquisition module 710 , the construction module 720 , the extraction module 730 , the first processing module 740 , and the second processing module 750 may be at least partially implemented as a computer program module, which may perform corresponding functions when executed.

[0134] Figure 8 The block diagram schematically shows an electronic device suitable for implementing the method for extracting motion feature data according to an embodiment of the present invention.

[0135] Figure 8 The block diagram schematically shows an electronic device suitable for implementing the method described above according to an embodiment of the present invention. Figure 8 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0136] like Figure 8As shown, the electronic device 800 according to an embodiment of the present invention includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 into a random access memory (RAM) 803. The processor 801 may, for example, include a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include an onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0137] Various programs and data required for the operation of the electronic device 800 are stored in the RAM 803. The processor 801, ROM 802, and RAM 803 are connected to each other via a bus 804. The processor 801 executes the programs in the ROM 802 and / or RAM 803 to perform various operations according to the method flow of the embodiment of the present invention. It should be noted that the programs may also be stored in one or more memories other than the ROM 802 and RAM 803. The processor 801 may also execute the programs stored in the one or more memories to perform various operations according to the method flow of the embodiment of the present invention.

[0138] According to an embodiment of the present invention, electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to bus 804. Electronic device 800 may also include one or more of the following components connected to I / O interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including devices such as a cathode ray tube (CRT), liquid crystal display (LCD), and speakers; a storage section 808 including a hard disk; and a communication section 809 including a network interface card such as a LAN card or modem. Communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to I / O interface 805 as needed. Removable media 811, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in drive 810 as needed, so that computer programs read from the removable media can be installed into storage section 808 as needed.

[0139] According to an embodiment of the present invention, the method flow according to an embodiment of the present invention can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the processor 801, the above-mentioned functions defined in the system of the embodiment of the present invention are executed. According to an embodiment of the present invention, the system, device, apparatus, module, unit, etc. described above can be implemented by a computer program module.

[0140] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present invention.

[0141] According to an embodiment of the present invention, a computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0142] For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 802 and / or RAM 803 described above and / or one or more memories other than the ROM 802 and RAM 803 .

[0143] An embodiment of the present invention also includes a computer program product, which includes a computer program, which contains program code for executing the method provided by the embodiment of the present invention. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the method for extracting motion feature data provided by the embodiment of the present invention.

[0144] When the computer program is executed by the processor 801, the above functions defined in the system / device of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0145] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 809, and / or installed from a removable medium 811. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0146] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, Python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect via the Internet).

[0147] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of the systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the boxes may occur in an order different from that marked in the accompanying drawings. For example, two boxes shown in succession may actually be executed substantially in parallel, or they may sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, as well as the combination of boxes in the block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or may be implemented using a combination of dedicated hardware and computer instructions. It will be understood by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or coupled in various ways, even if such combinations or couplings are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention may be combined and / or coupled in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or combinations fall within the scope of the present invention.

[0148] The above describes embodiments of the present invention. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be advantageously used in combination. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A method for extracting motion feature data, wherein: include: Acquiring kinematic data of the muscle to be tested, wherein the kinematic data includes data on changes in the degree of joint activity corresponding to the muscle to be tested over time; Establishing a finite element model of the muscle to be measured based on the electrical impedance imaging differential voltage signal value of the muscle to be measured and anatomical prior information of the cross section of the muscle to be measured, wherein the finite element model includes a plurality of signal nodes, each of the plurality of signal nodes has conductivity data of conductivity changing with time obtained by reconstructing the electrical impedance imaging differential voltage signal; extracting time domain information and frequency domain information of each of the conductivity data of a plurality of signal nodes from the finite element model; performing correlation calculation based on the time domain information, the frequency domain information, and the kinematic data to obtain a degree of exercise participation of each signal node, wherein the degree of exercise participation is used to characterize the degree to which each signal node participates in muscle movement; Motion feature data is obtained based on the motion participation degree of each signal node, the finite element model, and the conductivity data. The motion feature data is used to characterize the spatial position distribution of multiple key muscle units in the muscle to be tested, so as to determine the morphological characteristics of the muscle to be tested based on the spatial position distribution.

2. The method according to claim 1, wherein The performing correlation calculation based on the time domain information, the frequency domain information, and the kinematic data to obtain the motion participation degree of each signal node includes: Obtaining a time series correlation coefficient of each signal node according to the time domain information and kinematic data of the signal node, wherein the time series correlation coefficient is used to characterize the time series correlation between each signal node and the kinematic data; Obtaining a signal power spectrum of each signal node according to the frequency domain information and kinematic data of the signal node, wherein the signal power spectrum is used to characterize the amplitude response of each signal node at the frequency of the kinematic data; The motion participation degree of each signal node is obtained according to the time series correlation coefficient and signal power spectrum of each signal node.

3. The method according to claim 1, wherein Obtaining motion feature data according to the motion participation degree of each signal node includes: Obtaining a conductivity distribution map by reconstructing the conductivity data; Obtaining a sports participation characteristic map based on the sports participation of each signal node, the finite element model, and the conductivity distribution map; Downsampling the sports participation feature map to obtain downsampled image data; Extracting features of motion participation information of each signal node in the downsampled image data to obtain intermediate feature data; Motion feature data is obtained based on the intermediate feature data and the finite element model.

4. The method according to claim 3, wherein: The downsampling process of the sports participation feature map to obtain downsampled image data includes: Downsampling the sports participation feature map to obtain pre-processed downsampled image data; The pre-processed down-sampled image data is subjected to noise reduction processing to obtain the down-sampled image data.

5. The method according to claim 3 or 4, wherein: The feature extraction of the motion participation information of each signal node in the downsampled image data to obtain intermediate feature data includes: The motion participation information of each signal node is subjected to peak extraction through the second-order difference discriminant method to obtain intermediate feature data.

6. The method according to claim 3, wherein: The obtaining of motion feature data based on the intermediate feature data and the finite element model comprises: Taking the intermediate characteristic data as the center and the preset signal node radius r, the motion characteristic data is obtained by the nearest neighbor algorithm based on the finite element model.

7. The method according to claim 1, wherein The obtaining of kinematic data of the muscle to be measured comprises: The kinematic data of the muscle cross section to be measured is collected through an inertial measurement unit; The step of establishing a finite element model of the muscle to be measured based on the electrical impedance imaging differential voltage signal value of the muscle to be measured and the anatomical prior information of the cross section of the muscle to be measured includes: The finite element model is obtained by using a time-differential electrical impedance imaging algorithm and a finite element algorithm according to the electrical impedance imaging differential voltage signal value of the muscle to be measured and the anatomical prior information of the cross section of the muscle to be measured.

8. The method according to claim 2, wherein: The kinematic data includes a plurality of consecutive time window kinematic signal data: m=[m1,m2,m3,...,m T ]; The time domain information of each signal node includes a plurality of consecutive time window conductivity signal data n=[n1,n2,n3,...,n T ]; The time series correlation coefficient of the signal node obtained according to the kinematic data n and the time domain information m includes: The correlation coefficient was calculated by formula (1); Wherein, in formula (1), ρ is the correlation coefficient of the signal node; According to the coefficient ρ, the time series correlation coefficient is obtained through formula (2); R=|ρ (2) In formula (2), R is the timing correlation coefficient of the signal node.

9. The method according to claim 2 or 8, wherein: The signal power spectrum of each signal node obtained according to the frequency domain information and kinematic data of the signal node includes calculating using the following formula (3); Wherein, T represents the signal length of a signal node n, fft(n) represents Fourier transform of a signal of a signal node n, and P(f) is a function of the power spectrum of a signal node n.

10. The method according to claim 9, wherein: The motion participation degree of each signal node obtained according to the time series correlation coefficient and signal power spectrum of each signal node includes calculating using the following formula (4); p=R·P(f motion ) (4) Among them, R is the time series correlation coefficient of the signal node, P is the function of the power spectrum of the signal node, f motion is the frequency of the motion data, which is obtained by performing Fourier transform on the motion data and calculating the index corresponding to its extreme point.

Images