Surface electromyogram signal detection method and device based on image processing

Abstract

The invention discloses a surface electromyographic signal detection method and device based on image processing, and relates to the technical field of electromyographic detection. The surface electromyogram signal detection method based on image processing comprises the following steps: acquiring a skin surface data set to obtain a camera adjustment scheme; acquiring a muscle surface temperature distribution image and a muscle tissue cross-sectional image, and constructing a dynamic strain field model to obtain muscle key parameters; obtaining anti-artifact myoelectricity characteristic data based on the muscle key parameters; the anti-artifact myoelectricity characteristic data are converted into sEMG waveforms, and denoising optimization is carried out; on the basis of the sorted historical data, the optimized sEMG waveform and the muscle tissue cross-sectional image, an electromyographic signal prediction model is constructed, an electromyographic signal detection result is obtained, and the problems that in the prior art, physiological parameter analysis is insufficient, and copying interference is serious are solved.

Description

Technical Field

[0001] This invention relates to the field of electromyography (EMG) detection technology, specifically to a method and apparatus for detecting surface electromyography (EMG) signals based on image processing. Background Technology

[0002] The application of image processing technology in the biomedical field stems from the need for refined analysis of tissue morphology and function information. With the development of digital image processing algorithms and hardware technology, deep learning-based image analysis models are widely used in medical image segmentation, feature extraction, and quality assessment. For example, texture features are analyzed through optical images of the skin surface to assist in the diagnosis of skin diseases, or infrared thermal imaging data is used to monitor the metabolic state of local tissues.

[0003] Surface electromyography (EMG) signals are collected by electrodes placed on the skin surface, which capture the combined potential signal of the motor unit action potential during muscle contraction. It reflects the electrophysiological state of muscles during activity and is an objective quantitative indicator of neuromuscular system activity. Its waveform characteristics (such as amplitude, frequency, and time-frequency distribution) are directly related to muscle contraction force, fatigue level, and movement patterns.

[0004] It can be applied in sports science to analyze movement patterns, monitor muscle fatigue, and assess athlete performance; in rehabilitation medicine for assessing motor dysfunction, monitoring the effectiveness of rehabilitation training, and diagnosing neuromuscular diseases; in human factors engineering for enabling human-computer interaction and health monitoring and early warning in intelligent devices; and in biomedical research to reveal muscle physiological mechanisms and develop rehabilitation technologies. Because of its non-invasive and real-time advantages, detecting surface electromyography (EMG) signals can directly characterize muscle function and neural control status, providing a crucial data foundation for multidisciplinary applications.

[0005] A search revealed that Chinese invention patent CN115329803A discloses a method and device for detecting surface electromyography (EMG) signals based on image recognition. The detection method includes: capturing hand movement data based on image recognition and extracting hand movement features from the hand movement data; identifying the movement parts of the muscle groups involved in the movement based on the hand movement features, including primary and secondary movement positions; collecting surface EMG signals of the muscle groups during hand movement and processing the data to obtain changes in hand surface EMG signals; and performing curve fitting on the changes in hand surface EMG signals according to weights based on the primary movement parts of the muscle groups to generate a hand surface EMG signal map.

[0006] However, existing technologies suffer from insufficient physiological parameter analysis and severe interference from ghosting. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention provides a method and apparatus for detecting surface electromyography (EMG) signals based on image processing, which solves the problems of existing methods and apparatuses for detecting surface EMG signals based on image processing.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a surface electromyography (EMG) signal detection method based on image processing, comprising the following steps: acquiring a skin surface dataset and obtaining a camera adjustment scheme, wherein the camera includes an RGB camera and an infrared camera; adjusting the camera according to the camera adjustment scheme, acquiring a muscle surface temperature distribution image and a muscle tissue tomographic image, constructing a dynamic strain field model, and obtaining key muscle parameters; obtaining anti-artifact EMG feature data based on the key muscle parameters; converting the anti-artifact EMG feature data into an sEMG waveform and performing noise reduction optimization; constructing an EMG signal prediction model based on the organized historical data, the optimized sEMG waveform, and the muscle tissue tomographic image, and obtaining the EMG signal detection result.

[0009] Further, a camera adjustment scheme is obtained, as follows: The skin surface dataset includes optical image data and thermal imaging data. The optical image data includes skin texture feature data, skin surface geometric feature data, and skin surface dynamic feature data. The thermal imaging data includes skin surface temperature distribution feature data and thermal dynamic feature data. Based on the pre-built deep learning image analysis model, the skin surface dataset is input into the deep learning image analysis model to obtain image quality index data. The image quality index data includes sharpness, color saturation, contrast, and edge detail processing factor. The values ​​of each index in the analyzed image quality index data are compared with the values ​​of each reference index in each reference image quality index data stored in the data repository. The reference image quality index data corresponding to the closest reference index values ​​corresponds to the camera adjustment scheme stored in the data repository, which is the camera adjustment scheme corresponding to the current image quality index data. Based on this camera adjustment scheme, the camera is adjusted.

[0010] Further, the acquisition of muscle surface temperature distribution images and muscle tissue tomographic images is carried out as follows: The target action issued by the command center is acquired, and the subject performs the target action; based on the adjusted infrared camera, the target action is continuously acquired at a set frame rate, and the start and end times of the action are simultaneously marked using pulse signals from electromyography electrodes to obtain a sequence of muscle surface temperature distribution images, generating a muscle surface temperature distribution image; the target muscle of the subject performing the target action is scanned using an OCT probe, with axial scanning to measure the spectral distribution of the interference signal, and Fourier transform is performed on the spectral signal to obtain the light intensity distribution in the axial direction; a two-dimensional transverse scan is performed on the surface of the target muscle tissue, acquiring one light intensity distribution at each transverse position to form a two-dimensional data matrix, which is then converted into a grayscale image. After processing using the OCT analysis platform, a muscle tissue tomographic image is obtained; the number of rows in the two-dimensional data matrix represents the number of transverse pixels, and the number of columns represents the number of axial pixels.

[0011] Furthermore, a dynamic strain field model is constructed to obtain key muscle parameters. The process is as follows: Based on a high-speed camera combined with DIC, a dynamic deformation sequence of speckle patterns on the surface of the target muscle tissue is acquired, and electromyographic signals are recorded simultaneously. The time phase of muscle contraction is marked to obtain a multimodal image sequence. Noise reduction processing is performed on each frame of the image. The first frame of the image is used as the reference frame, and the frames other than the first frame are used as the target frames. A square subset is defined in the reference frame, and a pyramid hierarchical search is used in the target frames to obtain the best matching position, generating a continuous two-dimensional displacement field. The gradient of the two-dimensional displacement field is approximated using the central difference method to generate a time-varying strain component sequence, which is stored in matrix form, which is the dynamic strain field model, outputting key muscle parameters. Key muscle parameters include peak displacement, temperature gradient, peak strain, strain rate, principal strain, principal strain direction, shear strain energy, strain distribution uniformity, contraction efficiency index, EMG-strain hysteresis time, and metabolic-mechanical coupling coefficient.

[0012] Further, artifact-resistant electromyographic (EMG) feature data is obtained as follows: Based on key muscle parameters, EMG signal segments containing artifacts are removed. For time periods with artifacts, a Kalman filter is used to predict the true EMG signal, and wavelet decomposition is performed on the true EMG signal to obtain the reconstructed and filtered EMG signal. Based on the constructed CNN and LSTM hybrid model, the reconstructed and filtered EMG signal is divided into multiple EMG signal segments. The EMG signal segments and the corresponding key muscle parameters for each time period are synchronously input into the CNN and LSTM hybrid model. The CNN layer extracts the local time-frequency features of the EMG signal, and the L... The STM layer captures muscle biomechanical parameters and outputs a clean electromyographic (EMG) signal, which is the base waveform. Based on the filtered EMG signal, its root mean square (RMS) and mean absolute value are calculated to obtain time-domain feature data. Based on the clean EMG signal, the median frequency and average power frequency are calculated using Fast Fourier Transform (FFT) to obtain frequency-domain feature data. Based on the clean EMG signal, wavelet packet decomposition is used to extract the frequency band energy features of the EMG signal, and the number of decomposition layers is dynamically adjusted in combination with strain rate to obtain time-frequency domain feature data. The obtained time-domain feature data, frequency-domain feature data, and time-frequency domain feature data are recorded as anti-artifact EMG feature data.

[0013] Furthermore, based on key muscle parameters, electromyographic signal segments containing artifacts are removed, as follows: An artifact feature library is constructed based on key muscle parameters; if the strain rate of a local region is greater than the strain rate threshold stored in the database, the local region is marked as a high-motion region, and the electromyographic signal in this region is determined to be susceptible to artifact interference; if the angle between the principal strain direction and the muscle fiber orientation is greater than the angle threshold stored in the database, it is determined that electrode displacement causes artifacts; if the lag time between the strain peak and the electromyographic peak during normal muscle contraction exceeds the lag time range stored in the database... If the temperature difference between adjacent pixels exceeds the temperature difference threshold stored in the database, it is determined that contact noise is generated by tissue sliding. A mechanical artifact model is constructed, artifact components are output, and the source of artifacts is located. A physiological artifact model is constructed, artifact energy is output, and the degree of artifact contamination on the effective signal is evaluated. Based on the artifact feature library and artifact source, the corresponding segments in the electromyographic signal are removed. Based on the artifact energy, if the artifact energy is greater than the energy threshold of the total energy of the electromyographic signal, the segment corresponding to the artifact energy in the electromyographic signal is removed.

[0014] Furthermore, the anti-artifact electromyography (EMG) feature data is converted into sEMG waveforms and then denoised and optimized. The process is as follows: Based on time-domain feature data, frequency-domain feature data, and time-frequency-domain feature data, the amplitude of the base waveform is adjusted; the adjusted waveforms are superimposed and synthesized to form a preliminary sEMG waveform; a least mean square (LMS) filter is used, with key muscle parameters as reference signals, to filter the preliminary sEMG waveform in real time; a Kalman filter is then used to predict and correct the waveform, remove random noise, and smooth the waveform curve; median filtering is applied to the denoised sEMG waveform to eliminate small-amplitude spikes and discontinuities, resulting in an optimized sEMG waveform.

[0015] Furthermore, based on time-domain feature data, frequency-domain feature data, and time-frequency-domain feature data, the amplitude of the base waveform is adjusted as follows: The amplitude of the base waveform is adjusted based on the root mean square (RMS) value and the mean absolute value (MAV): If the RMS value is not equal to the baseline value of the base waveform stored in the data repository, the amplitude of each sampling point of the base waveform is multiplied by a scaling factor, which is the ratio of the RMS value to the baseline value; if the MAV is not equal to the baseline value of the base waveform stored in the data repository, the amplitude of each sampling point of the base waveform is multiplied by a scaling factor, which is the ratio of the MAV to the baseline value; Based on the median frequency and the average power frequency, the main frequency components of the pure electromyographic signal are determined, and the corresponding sine waves are superimposed; Based on the time-frequency-domain feature data, the waveform is dynamically adjusted according to the frequency band energy distribution in different time periods. If high-frequency energy decreases in the early stage of muscle contraction, the high-frequency components are reduced accordingly when adjusting the base waveform; if high-frequency energy increases in the peak stage of muscle contraction, the high-frequency components are enhanced accordingly when adjusting the base waveform.

[0016] Further, an electromyography (EMG) signal prediction model is constructed as follows: Historical data includes historical EMG data, historical image data, and historical action label data. Historical EMG feature data and historical image feature data are extracted and concatenated into a multi-dimensional vector at time steps. The multi-dimensional vector is then input into a hybrid neural network model, which includes an input layer, a feature extraction layer, and a fusion and output layer. The historical data is divided into a training set, a validation set, and a test set. The training set is input into the hybrid neural network model for training, resulting in an EMG signal prediction model. Based on the trained EMG signal prediction model, the model is validated using a validation set, and then tested using a test set. Finally, the optimized sEMG waveform and muscle tissue tomographic images are input into the validated EMG signal prediction model to obtain the EMG signal detection results.

[0017] The surface electromyography (EMG) signal detection device based on image processing includes a camera adjustment scheme acquisition module, an image acquisition and strain modeling module, an EMG signal anti-artifact processing module, an sEMG waveform optimization module, and a prediction model construction and result output module. Specifically, the module includes: a camera adjustment scheme acquisition module, used to acquire skin surface datasets and obtain camera adjustment schemes, with cameras including RGB cameras and infrared cameras; an image acquisition and strain modeling module, used to adjust the cameras based on the camera adjustment scheme, acquire muscle surface temperature distribution images and muscle tissue tomographic images, construct a dynamic strain field model, and obtain key muscle parameters; an EMG signal anti-artifact processing module, used to obtain anti-artifact EMG feature data based on key muscle parameters; an sEMG waveform optimization module, used to convert anti-artifact EMG feature data into sEMG waveforms and perform noise reduction optimization; and a prediction model construction and result output module, used to construct an EMG signal prediction model based on organized historical data, optimized sEMG waveforms, and muscle tissue tomographic images, and obtain EMG signal detection results.

[0018] The present invention has the following beneficial effects: This image processing-based surface electromyography (sEMG) signal detection method and device improves the accuracy of artifact recognition by analyzing the correlation between key muscle parameters and EMG signals. The hybrid neural network model integrates temporal and spatial features to solve the problem of insufficient information from single-modality data. It can dynamically adapt to the physiological differences of different subjects, reduce the risk of erroneously removing effective signals, avoid signal fragment loss due to artifacts, improve data integrity, and the reconstructed sEMG waveform is more consistent with the physiological stages of muscle contraction, improving the physiological interpretability of the signal. The model can automatically extract common features across individuals, reduce overfitting caused by data bias of a single subject, reduce human intervention, and is suitable for real-time monitoring scenarios.

[0019] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0020] Figure 1 This is a flowchart of the surface electromyography signal detection method based on image processing according to the present invention.

[0021] Figure 2 This is a block diagram of the surface electromyography signal detection device based on image processing according to the present invention.

[0022] Figure 3 This is a flowchart illustrating the generation of anti-artifact electromyographic feature data in the surface electromyographic signal detection method based on image processing of the present invention. Detailed Implementation

[0023] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0024] Please see Figure 1 and Figure 3 The present invention provides a technical solution: a surface electromyography signal detection method based on image processing, comprising the following steps: acquiring a skin surface dataset, obtaining a camera adjustment scheme, wherein the camera includes an RGB camera and an infrared camera.

[0025] The skin surface dataset includes optical image data and thermal imaging data. The optical image data includes skin texture feature data, skin surface geometric feature data, and skin surface dynamic feature data. The thermal imaging data includes skin surface temperature distribution feature data and thermal dynamic feature data.

[0026] Based on the established deep learning image analysis model, the skin surface dataset is input into the deep learning image analysis model to obtain image quality index data. The image quality index data includes sharpness, color saturation, contrast, and edge detail processing factor.

[0027] The values ​​of each indicator in the analyzed image quality index data are compared with the values ​​of each reference indicator in the reference image quality index data stored in the data repository. The camera adjustment scheme corresponding to the reference image quality index data with the closest reference indicator value is the camera adjustment scheme corresponding to the current image quality index data. The camera is then adjusted based on this camera adjustment scheme.

[0028] The analysis of optical and thermal imaging data of the skin surface involves adjusting the parameters of the RGB and infrared cameras according to the skin characteristics of different subjects. This avoids poor image quality caused by fixed parameters, such as overexposure and temperature-induced blurring, and ensures that the acquired image data meets the requirements for subsequent analysis.

[0029] By leveraging deep learning image analysis models to output quantitative image quality metrics such as sharpness and color saturation, image quality can be measured more objectively and accurately. Real-time image quality metrics are compared with reference values ​​in a data repository to quickly determine the optimal camera adjustment scheme for the current scene. This eliminates the need for repeated manual parameter adjustments, shortening equipment preparation time and improving detection efficiency. For example, when continuously testing multiple subjects, camera parameter adjustments can be completed quickly, ensuring the continuity of the detection process.

[0030] Using verified reference image quality data from the data repository as a benchmark, we ensure that the image acquisition quality is stable and reliable after each camera adjustment. The comparability of detection image data from different times and different subjects provides a high-quality data foundation for subsequent correlation analysis of electromyographic signals and image data, model training, and other tasks.

[0031] After adjusting the camera based on the camera adjustment scheme, images of muscle surface temperature distribution and muscle tissue tomographic images are acquired, a dynamic strain field model is constructed, and key muscle parameters are obtained.

[0032] The target action issued by the command center is obtained, and the subject performs the target action. Based on the adjusted infrared camera, the target action is continuously acquired at a set frame rate, and the pulse signal of the electromyography electrode is used simultaneously to mark the start and end times of the action to obtain a muscle surface temperature distribution image sequence and generate a muscle surface temperature distribution image.

[0033] The target muscle of the subject performing the target action is scanned using an OCT probe. The axial scan is performed to measure the spectral distribution of the interference signal. The spectral signal is then subjected to Fourier transform to obtain the light intensity distribution in the axial direction. A two-dimensional transverse scan is performed on the surface of the target muscle tissue, and a light intensity distribution is acquired at each transverse position to form a two-dimensional data matrix. The two-dimensional data matrix is ​​then converted into a grayscale image and processed using an OCT analysis platform to obtain a tomographic image of the muscle tissue. The number of rows in the two-dimensional data matrix represents the number of transverse pixels, and the number of columns represents the number of axial pixels.

[0034] The command center standardizes target actions and uses electromyographic (EMG) electrode pulse signals to mark the start and end times of the actions, ensuring precise temporal alignment between the temperature distribution image sequences captured by the infrared camera, the muscle tissue tomographic images acquired by the OCT probe, and the EMG signals. For example, when analyzing the muscle contraction process, temperature changes, tissue structure changes, and electrical activity changes can be analyzed in a corresponding manner, avoiding data misinterpretation due to temporal misalignment.

[0035] Based on a high-speed camera combined with DIC, dynamic deformation sequences of speckle patterns on the surface of target muscle tissue are acquired, electromyographic signals are recorded simultaneously, and the temporal phase of muscle contraction is marked to obtain a multimodal image sequence; noise reduction processing is performed on each frame of the image.

[0036] The first frame is used as the reference frame, and all other frames are used as the target frames. A square subset is defined in the reference frame, and a pyramid hierarchical search is used in the target frames to obtain the best matching position, generating a continuous two-dimensional displacement field. The gradient of the two-dimensional displacement field is approximated by the central difference method to generate a time-varying strain component sequence, which is stored in matrix form, thus forming the dynamic strain field model, and outputting key muscle parameters.

[0037] Key muscle parameters include peak displacement, temperature gradient, peak strain, strain rate, principal strain, principal strain direction, shear strain energy, strain distribution uniformity, contraction efficiency index, EMG-strain hysteresis time, and metabolic-mechanical coupling coefficient.

[0038] Image acquisition is performed based on pre-adjusted camera parameters to ensure the clarity of muscle surface temperature distribution images and the accuracy of temperature data. The OCT probe uses a combination of axial and two-dimensional transverse scanning to quickly acquire a two-dimensional data matrix of muscle tissue tomography. After being converted into grayscale images, the internal structure of the muscle can be intuitively presented, providing high-quality image data for subsequent analysis and reducing repeated acquisitions caused by image blurring or missing information.

[0039] By utilizing high-speed cameras and DIC technology, and employing pyramid hierarchical search and central difference calculation, a precise two-dimensional displacement field and strain component sequence of muscle tissue are acquired, thereby constructing a dynamic strain field model. Key muscle parameters are output, quantifying the mechanical changes of the muscle during movement. These key muscle parameters encompass multiple dimensions, including mechanics, thermodynamics, and electrical properties, comprehensively reflecting the physiological state of the muscle when performing a target movement.

[0040] Based on key muscle parameters, anti-artifact electromyographic characteristic data were obtained.

[0041] Artifacts refer to false signals or interference components introduced during signal acquisition, processing, or imaging due to non-target factors such as equipment interference, physiological motion, operational errors, or environmental factors. These artifacts can cause data to deviate from the true characteristics and affect the accuracy of subsequent analysis.

[0042] Based on key muscle parameters, electromyographic signal segments containing artifacts are removed.

[0043] Based on key muscle parameters, an artifact feature library is constructed. If the strain rate of a local area is greater than the strain rate threshold stored in the database, the local area is marked as a high-motion area, and the electromyography (EMG) signal in this area is determined to be susceptible to artifact interference. If the angle between the principal strain direction and the muscle fiber orientation is greater than the angle threshold stored in the database, it is determined that there is an artifact caused by electrode displacement. If the lag time between the strain peak and the EMG peak during normal muscle contraction exceeds the lag time range stored in the database, the EMG signal exceeding this range is determined to contain artifacts. If the temperature difference between adjacent pixels is greater than the temperature difference threshold stored in the database, it is determined that contact noise is generated by tissue sliding.

[0044] Driven by key muscle parameters, artifact recognition is achieved by comparing parameters such as strain rate and temperature gradient with preset thresholds to establish clear artifact judgment rules, thereby realizing accurate mapping from data features to artifact types and increasing the basis for judgment in the physiological and mechanical dimensions of muscles.

[0045] Construct a mechanical artifact model, output artifact components, and locate the source of artifacts.

[0046] The formula for the mechanical artifact model is: ; In the formula, For artifact components, For strain rate, Let A be the temperature gradient, B be the first fitting coefficient, and T be the second fitting coefficient. It is random noise.

[0047] A physiological artifact model is constructed, the artifact energy is output, and the degree of artifact contamination on the effective signal is assessed.

[0048] The formula for the physiological artifact model is: ; In the formula, denoted by , where C is the artifact component, U is the strain distribution uniformity factor, and SEI is the shrinkage efficiency index. For reference only.

[0049] Based on the artifact feature library and artifact sources, corresponding segments in the electromyography (EMG) signal are removed; based on artifact energy, if the artifact energy is greater than the energy threshold of the total EMG signal energy, the segment corresponding to that artifact energy in the EMG signal is removed.

[0050] For the time periods where artifacts exist, a Kalman filter is used to predict the true electromyography (EMG) signal, and wavelet decomposition is performed on the true EMG signal to obtain the reconstructed and filtered EMG signal.

[0051] Based on the constructed CNN and LSTM hybrid model, the reconstructed and filtered electromyographic signal is divided into multiple electromyographic signal segments. The electromyographic signal segments and the key muscle parameters of the corresponding time period are synchronously input into the CNN and LSTM hybrid model. The CNN layer extracts the local time-frequency features of the electromyographic signal, and the LSTM layer captures the muscle mechanical parameters, outputting a pure electromyographic signal, which is the base waveform.

[0052] Based on the filtered electromyographic (EMG) signal, its root mean square (RMS) value and mean absolute value are calculated to obtain time-domain feature data. Based on the clean EMG signal, the median frequency and average power frequency are calculated through Fast Fourier Transform (FFT) to obtain frequency-domain feature data. Based on the clean EMG signal, wavelet packet decomposition is used to extract the frequency band energy features of the EMG signal, and the number of decomposition layers is dynamically adjusted in combination with strain rate to obtain time-frequency domain feature data. The obtained time-domain feature data, frequency-domain feature data, and time-frequency domain feature data are denoted as anti-artifact EMG feature data.

[0053] By utilizing mechanical and physiological artifact models, key muscle parameters are transformed into quantifiable artifact components and artifact energy indices, enabling a numerical assessment of artifact impact. For example, in the physiological artifact model, artifact energy is calculated by combining parameters such as the contraction efficiency index, which can intuitively measure the degree of artifact contamination of the effective signal and provide an accurate numerical reference standard for subsequent processing.

[0054] After identifying and evaluating artifacts, not only are artifact segments removed based on rules, but Kalman filters and wavelet decomposition are also used to recover contaminated signals. This step focuses on improving the quality of electromyographic (EMG) signals, ensuring that the signals entering subsequent analysis are as close as possible to real muscle electrical activity. A hybrid CNN and LSTM model is used to extract EMG features by combining key muscle parameters. Deep learning models enable in-depth feature mining, further enhancing the ability of feature data to represent the true state of muscles.

[0055] Anti-artifact electromyographic feature data is converted into sEMG waveforms and then optimized for denoising.

[0056] The amplitude of the basic waveform is adjusted based on time-domain feature data, frequency-domain feature data, and time-frequency-domain feature data.

[0057] The amplitude of the base waveform is adjusted based on the root mean square (RMS) value and the mean absolute value: if the RMS value is not equal to the baseline value of the base waveform stored in the data repository, the amplitude of each sampling point of the base waveform is multiplied by a scaling factor, which is the ratio of the RMS value to the baseline value; if the mean absolute value is not equal to the baseline value of the base waveform stored in the data repository, the amplitude of each sampling point of the base waveform is multiplied by a scaling factor, which is the ratio of the mean absolute value to the baseline value.

[0058] Based on the median frequency and average power frequency, the main frequency components of the pure electromyographic signal are determined and superimposed with the corresponding sine waves. Based on time-frequency domain characteristic data, the waveform is dynamically adjusted according to the frequency band energy distribution in different time periods. If the high-frequency energy decreases in the early stage of muscle contraction, the high-frequency component is reduced accordingly when adjusting the base waveform. If the high-frequency energy increases in the peak stage of muscle contraction, the high-frequency component is enhanced accordingly when adjusting the base waveform.

[0059] The adjusted waveforms are superimposed to form a preliminary sEMG waveform. A minimum mean square (LMS) filter is used, with key muscle parameters as reference signals, to filter the preliminary sEMG waveform in real time. A Kalman filter is then used to predict and correct the waveform, remove random noise, and smooth the waveform curve. Median filtering is then applied to the denoised sEMG waveform to eliminate small-amplitude spikes and discontinuities, resulting in an optimized sEMG waveform.

[0060] Based on time-domain feature data, frequency-domain feature data, and time-frequency-domain feature data, sEMG waveforms that match physiological states are actively reconstructed to ensure that the waveform amplitude matches the actual electrical activity intensity of muscle contraction and to avoid signal amplitude distortion caused by artifact removal.

[0061] By reducing high-frequency components in the early stages of muscle contraction and enhancing high-frequency components during the peak phase, the dynamic correlation between energy metabolism and electrical signal frequency during muscle physiological activities (such as the decrease of high-frequency energy during fatigue) is brought into account, making the reconstructed waveform more realistically reflect the time-frequency characteristics of muscle contraction and emphasizing the dynamic fidelity of signal frequency characteristics.

[0062] The LMS filter uses key muscle parameters as reference signals to specifically suppress noise related to muscle mechanics (such as motion artifacts). The Kalman filter smooths out random fluctuations introduced by signal recovery through state prediction and correction. The median filter further eliminates isolated glitches and improves waveform continuity, together ensuring waveform quality.

[0063] By superimposing sine waves corresponding to the main frequency components and dynamically adjusting the waveform based on the energy distribution in the time and frequency domains, the final output sEMG waveform is not only a "denoised signal" but also a dynamic electrical activity representation highly correlated with the physiological characteristics of muscle contraction phases (such as the initial and peak stages). This directly reflects the physiological phenomenon of increased motor unit recruitment, providing a more reliable signal basis for subsequent electromyographic-mechanical coupling analysis.

[0064] Based on the organized historical data, optimized sEMG waveforms, and muscle tissue tomographic images, an electromyography (EMG) signal prediction model was constructed to obtain EMG signal detection results.

[0065] Historical data includes historical electromyography (EMG) data, historical image data, and historical action label data. Historical EMG feature data and historical image feature data are extracted and concatenated into a multidimensional vector at time steps. The multidimensional vector is then input into a hybrid neural network model, which includes an input layer, a feature extraction layer, and a fusion and output layer.

[0066] Historical data is divided into training, validation, and test sets. The training set is input into a hybrid neural network model to train the model and obtain an electromyography (EMG) signal prediction model. Based on the trained EMG signal prediction model, it is input into the validation set for validation, and then into the test set for testing. The optimized sEMG waveform and muscle tissue tomographic images are input into the validated EMG signal prediction model to obtain the EMG signal detection results.

[0067] The optimized sEMG waveform (temporal features) and muscle tissue tomographic images (spatial structural features) are combined with historical data and input into the model. Spatial and temporal features are extracted through a hybrid neural network to achieve cross-modal correlation analysis of "electrical activity-anatomical structure", thereby improving the accuracy of motion recognition or fatigue prediction.

[0068] A training set containing multiple subjects and various movement types is constructed to enable the model to learn the differences in muscle anatomy and electrical activity patterns among different individuals. To avoid overfitting caused by the bias of data from a single subject, historical data is used to enhance the adaptability of the electromyography signal prediction model to "generalization scenarios".

[0069] The optimized sEMG waveforms and tomographic images can be directly used as model inputs without the need for manual feature selection or parameter adjustment. This is suitable for real-time monitoring scenarios, reduces human intervention errors, and improves detection efficiency and consistency.

[0070] A surface electromyography signal detection device based on image processing, referring to Figure 2 It includes a camera adjustment scheme acquisition module, an image acquisition and strain modeling module, an electromyography signal anti-artifact processing module, an sEMG waveform optimization module, and a prediction model construction and result output module. Specifically, it includes a camera adjustment scheme acquisition module, which is used to acquire skin surface datasets and obtain camera adjustment schemes. The cameras include RGB cameras and infrared cameras.

[0071] The image acquisition and strain modeling module is used to adjust the camera based on the camera adjustment scheme, acquire images of muscle surface temperature distribution and muscle tissue tomographic images, construct a dynamic strain field model, and obtain key muscle parameters.

[0072] The electromyography (EMG) signal anti-artifact processing module is used to obtain anti-artifact EMG feature data based on key muscle parameters.

[0073] The sEMG waveform optimization module is used to convert anti-artifact electromyography feature data into sEMG waveforms and perform noise reduction optimization.

[0074] The prediction model is constructed and the results are output. Based on the sorted historical data, optimized sEMG waveforms and muscle tissue tomographic images, an electromyography (EMG) signal prediction model is constructed to obtain the EMG signal detection results.

[0075] The above description is merely an example and illustration of the structure of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described or use similar methods to replace them, as long as they do not deviate from the structure of the invention or exceed the scope defined in the claims, they should all fall within the protection scope of the present invention.

[0076] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0077] This invention is described with reference to flowchart illustrations and / or block diagrams of systems, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0078] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0079] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0080] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0081] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for detecting surface electromyography signals based on image processing, characterized in that, Includes the following steps: Obtain a dataset of the skin surface to determine the camera adjustment scheme. The cameras include an RGB camera and an infrared camera. After adjusting the camera based on the camera adjustment scheme, the surface temperature distribution image of the muscle and the tomographic image of the muscle tissue are acquired, a dynamic strain field model is constructed, and key muscle parameters are obtained. Based on key muscle parameters, anti-artifact electromyographic characteristic data were obtained; The anti-artifact electromyography feature data was converted into sEMG waveforms and then denoised and optimized. Based on the organized historical data, optimized sEMG waveforms, and muscle tissue tomographic images, an electromyography (EMG) signal prediction model was constructed to obtain EMG signal detection results.

2. The surface electromyography signal detection method based on image processing according to claim 1, characterized in that, The camera adjustment solution was obtained through the following process: The skin surface dataset includes optical image data and thermal imaging data. The optical image data includes skin texture feature data, skin surface geometric feature data, and skin surface dynamic feature data. The thermal imaging data includes skin surface temperature distribution feature data and thermal dynamic feature data. Based on the established deep learning image analysis model, the skin surface dataset is input into the deep learning image analysis model to obtain image quality index data; The image quality metrics include sharpness, color saturation, contrast, and edge detail processing factors. The values ​​of each indicator in the analyzed image quality index data are compared with the values ​​of each reference indicator in the reference image quality index data stored in the data repository. The camera adjustment scheme corresponding to the reference image quality index data corresponding to the closest reference index values ​​is the camera adjustment scheme corresponding to the current image quality index data, and the camera is adjusted based on the camera adjustment scheme.

3. The surface electromyography signal detection method based on image processing according to claim 1, characterized in that, The process of acquiring images of muscle surface temperature distribution and muscle tissue tomographic images is as follows: The subject receives the target action issued by the command center and executes the target action. Based on the adjusted infrared camera, the target action is continuously acquired at a set frame rate, and the pulse signal of the electromyography electrode is used simultaneously to mark the start and end times of the action to obtain a muscle surface temperature distribution image sequence and generate a muscle surface temperature distribution image. Based on the OCT probe scanning of the target muscle of the subject performing the target action, the axial scanning is performed to measure the spectral distribution of the interference signal, and the Fourier transform of the spectral signal is performed to obtain the light intensity distribution in the axial direction. A two-dimensional transverse scan of the target muscle tissue surface was performed, and a light intensity distribution was acquired at each transverse position to form a two-dimensional data matrix. The two-dimensional data matrix was then converted into a grayscale image, and after processing based on the OCT analysis platform, a tomographic image of the muscle tissue was obtained. The number of rows in a two-dimensional data matrix represents the number of pixels horizontally, and the number of columns represents the number of pixels axially.

4. The surface electromyography signal detection method based on image processing according to claim 1, characterized in that, A dynamic strain field model was constructed to obtain key muscle parameters. The process is as follows: Based on a high-speed camera combined with DIC, the dynamic deformation sequence of speckle patterns on the surface of target muscle tissue is acquired, electromyographic signals are recorded simultaneously, and the temporal phase of muscle contraction is marked to obtain a multimodal image sequence. Denoising is applied to each frame of the image; The first frame image is used as the reference frame, and the images other than the first frame image are used as the target frames. A square subset is defined in the reference frame, and a pyramid hierarchical search is used in the target frames to obtain the best matching position and generate a continuous two-dimensional displacement field. The gradient of the two-dimensional displacement field is approximated by the central difference method, and a time-varying strain component sequence is generated and stored in matrix form, which is the dynamic strain field model, and outputs key muscle parameters. Key muscle parameters include peak displacement, temperature gradient, peak strain, strain rate, principal strain, principal strain direction, shear strain energy, strain distribution uniformity, contraction efficiency index, EMG-strain hysteresis time, and metabolic-mechanical coupling coefficient.

5. The surface electromyography signal detection method based on image processing according to claim 1, characterized in that, The electromyographic characteristics data for artifact resistance were obtained as follows: Based on key muscle parameters, electromyographic signal segments containing artifacts are removed. For the time periods containing artifacts, a Kalman filter is used to predict the real electromyographic signal, and wavelet decomposition is performed on the real electromyographic signal to obtain the reconstructed and filtered electromyographic signal. Based on the constructed CNN and LSTM hybrid model, the reconstructed and filtered electromyographic signal is divided into multiple electromyographic signal segments. The electromyographic signal segments and the key muscle parameters of the corresponding time period are synchronously input into the CNN and LSTM hybrid model. The CNN layer extracts the local time-frequency features of the electromyographic signal, the LSTM layer captures the muscle mechanical parameters, and outputs a pure electromyographic signal, which is the base waveform. Based on the filtered electromyographic signal, its root mean square value and mean absolute value are calculated to obtain time-domain feature data. Based on pure electromyographic signals, the median frequency and average power frequency are calculated by fast Fourier transform to obtain frequency domain feature data. Based on pure electromyographic signals, wavelet packet decomposition is used to extract the frequency band energy features of the electromyographic signals. Combined with strain rate, the number of decomposition layers is dynamically adjusted to obtain time-frequency domain feature data. The obtained time-domain feature data, frequency-domain feature data, and time-frequency-domain feature data are denoted as anti-artifact electromyographic feature data.

6. The surface electromyography signal detection method based on image processing according to claim 5, characterized in that, Based on key muscle parameters, electromyographic signal segments containing artifacts are removed, as follows: Based on key muscle parameters, an artifact feature library was constructed. If the strain rate of a local area is greater than the strain rate threshold stored in the data repository, the local area is marked as a high-motion area, and the electromyography signal in the area is determined to be susceptible to artifact interference. If the angle between the principal strain direction and the muscle fiber orientation is greater than the angle threshold stored in the data repository, it is determined that there is an artifact caused by electrode displacement. If the lag time between the peak strain and the peak electromyography (EMG) during normal muscle contraction exceeds the lag time range stored in the data repository, then the EMG signal exceeding this range is determined to contain artifacts. If the temperature difference between adjacent pixels is greater than the temperature difference threshold stored in the data repository, it is determined that contact noise is generated by tissue sliding. Construct a mechanical artifact model, output artifact components, and locate the source of artifacts; A physiological artifact model is constructed, the artifact energy is output, and the degree of artifact contamination on the effective signal is assessed. Based on the artifact feature library and artifact sources, corresponding segments in the electromyographic signals are removed; Based on artifact energy, if the artifact energy is greater than the energy threshold of the total energy of the electromyographic signal, the segment corresponding to that artifact energy in the electromyographic signal is removed.

7. The surface electromyography signal detection method based on image processing according to claim 1, characterized in that, The anti-artifact electromyography (EMG) feature data was converted into sEMG waveforms and then denoised and optimized. The process is as follows: The amplitude of the basic waveform is adjusted based on time-domain feature data, frequency-domain feature data, and time-frequency-domain feature data. The adjusted waveforms are superimposed and synthesized to form a preliminary sEMG waveform; The minimum mean square (LMS) filter is used, with key muscle parameters as reference signals, to perform real-time filtering on the initial sEMG waveform. Then, a Kalman filter is used to predict and correct the waveform, remove random noise, and smooth the waveform curve. Median filtering is applied to the denoised sEMG waveform to eliminate small-amplitude spikes and discontinuities, resulting in an optimized sEMG waveform.

8. The surface electromyography signal detection method based on image processing according to claim 7, characterized in that, The amplitude of the base waveform is adjusted based on time-domain feature data, frequency-domain feature data, and time-frequency-domain feature data, as follows: Adjust the amplitude of the base waveform based on the root mean square value and the mean absolute value: If the root mean square value is not equal to the baseline value of the base waveform stored in the data repository, then the amplitude of each sampling point of the base waveform is multiplied by a scaling factor, which is the ratio of the root mean square value to the baseline value. If the mean absolute value is not equal to the reference value of the base waveform stored in the data repository, then the amplitude of each sampling point of the base waveform is multiplied by a scaling factor, which is the ratio of the mean absolute value to the reference value. Based on the median frequency and average power frequency, the main frequency components of the pure electromyographic signal are determined and superimposed with the corresponding sine wave. Based on time-frequency domain feature data, the waveform is dynamically adjusted according to the frequency band energy distribution in different time periods. If the high-frequency energy decreases in the early stage of muscle contraction, the high-frequency components are reduced accordingly when adjusting the base waveform. If high-frequency energy increases during the peak phase of muscle contraction, the high-frequency components should be increased accordingly when adjusting the base waveform.

9. The surface electromyography signal detection method based on image processing according to claim 1, characterized in that, The process of constructing an electromyography signal prediction model is as follows: Historical data includes historical electromyography (EMG) data, historical image data, and historical action label data. Historical EMG feature data and historical image feature data are extracted, and the historical EMG feature data and historical image feature data are concatenated into a multi-dimensional vector according to time steps. Multidimensional vectors are input into a hybrid neural network model, which includes an input layer, a feature extraction layer, and a fusion and output layer. Historical data is divided into training set, validation set and test set. The training set is input into the hybrid neural network model to train the hybrid neural network model and obtain the electromyography signal prediction model. Based on the trained electromyography signal prediction model, it is input into the validation set for validation, and then input into the test set for testing. The optimized sEMG waveform and muscle tissue tomographic images are input into the validated electromyography signal prediction model to obtain the electromyography signal detection results.

10. A surface electromyography (sEMG) signal detection device based on image processing, used in the surface electromyography (sEMG) signal detection method based on image processing according to any one of claims 1-9, comprising a camera adjustment scheme acquisition module, an image acquisition and strain modeling module, an EMG signal anti-artifact processing module, an sEMG waveform optimization module, and a prediction model construction and result output module, characterized in that, Specifically, it includes: The camera adjustment scheme acquisition module is used to acquire skin surface dataset and obtain camera adjustment scheme. The cameras include RGB cameras and infrared cameras. The image acquisition and strain modeling module is used to acquire images of muscle surface temperature distribution and muscle tissue tomographic images after adjusting the camera based on the camera adjustment scheme, construct a dynamic strain field model, and obtain key muscle parameters. The electromyography signal anti-artifact processing module is used to obtain anti-artifact electromyography feature data based on key muscle parameters; The sEMG waveform optimization module is used to convert anti-artifact electromyography feature data into sEMG waveforms and perform noise reduction optimization. The prediction model is constructed and the results are output. Based on the sorted historical data, optimized sEMG waveforms and muscle tissue tomographic images, an electromyography (EMG) signal prediction model is constructed to obtain the EMG signal detection results.

Images