Facial nerve closed-loop regulation and control stimulation method and device based on visual feedback

Abstract

The invention discloses a facial nerve closed-loop regulation and control stimulation method and device based on visual feedback. The method comprises the following steps: acquiring a facial movement mode of a healthy side of a patient; the face movement mode of the unhealthy side of the patient is used as expected movement information, the feedback face movement mode of the affected side is used as actual movement information, the generation process of the stimulation current is iteratively optimized, and the corresponding stimulation current is generated; the stimulation current is input into an implanted stimulation electrode in the body of the patient, and the stimulation current enables muscles on the affected side of the patient to contract and move; the image acquisition device acquires a facial movement image of an affected side of a patient and determines a facial movement mode of the affected side; and repeating the steps until facial stimulation is completed. According to the motion information generated after actual muscle stimulation, the stimulation current which generates the motion and is implanted into the stimulation electrode is fed back, generation of the stimulation current is iteratively optimized, the stimulation current is made to meet the expectation better, and therefore the stimulation effect is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of rehabilitation stimulation, and particularly relates to a method and device for closed-loop regulation and stimulation of facial nerves based on visual feedback. Background Art

[0002] The nervous system is divided into the central nervous system and the peripheral nervous system. The facial nerve belongs to the 7th pair of the 12 pairs of cranial nerves in the peripheral nervous system. Due to reasons such as the severance, injury, inflammation, and compression of the 7th pair of cranial nerves, the orbicularis oculi muscle controlled by the facial nerve will have movement disorders, manifested as the loss of static and dynamic expressions on one side of the face, deviation of the mouth and eyes, incomplete closure, etc. In severe cases, the eyelids cannot be closed, resulting in risks such as dryness, inflammation, and even blindness of the eye cornea.

[0003] In the process of realizing motion control by specific nerve electrical stimulation, that is, in the process of nerve regulation, the result of regulating or controlling muscle movement is not necessarily consistent with the expected goal, resulting in poor stimulation effects. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention proposes a method and device for closed-loop regulation and stimulation of facial nerves based on visual feedback.

[0005] In order to achieve the above purpose, the technical solution of the present invention is as follows:

[0006] In the first aspect, the present invention discloses a method for closed-loop regulation and stimulation of facial nerves based on visual feedback, including:

[0007] Step S1: Collect the facial movement pattern of the healthy side of the patient;

[0008] Step S2: Take the facial movement pattern of the healthy side of the patient as the expected movement information, and take the feedback facial movement pattern of the affected side as the actual movement information, and iteratively optimize the generation process of the stimulation current and generate the corresponding stimulation current;

[0009] Step S3: Input the stimulation current into the implanted stimulation electrode in the patient's body. The stimulation current stimulates the wrapped facial nerve trunk through the implanted stimulation electrode to generate nerve electrical impulses. The nerve electrical impulses are transmitted to the distal muscles under control, causing the muscles on the affected side of the patient to contract and produce movement;

[0010] Step S4: The image acquisition device collects the facial movement images of the affected side of the patient;

[0011] Step S5: Analyze the collected facial movement images of the affected side and identify the facial movement parameters of the affected side;

[0012] Step S6: Determine the facial movement pattern of the affected side according to the identified facial movement parameters of the affected side;

[0013] Step S7: Repeat Step S2 - Step S6 until facial stimulation is completed.

[0014] Based on the above technical solution, the following improvements can be made:

[0015] As a preferred solution, in Step S2,

[0016] Take the facial movement pattern of the healthy side of the patient as the desired movement information X T , and take the feedback facial movement pattern of the affected side as the actual movement information X A , and use the following formula to describe the error between X A and X T :

[0017] ||e|| = ||X A -X T || = ||S·I - X T ||;

[0018] There is the following mapping relationship between the stimulation current and the feedback facial movement pattern of the affected side;

[0019] X A = S·I;

[0020] Where: I is the stimulation current in the n-channel stimulation electrodes;

[0021] S is the movement information transformation matrix of the control muscles corresponding to the stimulation current;

[0022] Minimize the error between X A and X T , and solve for the parameters in the stimulation current I and the movement information transformation matrix S.

[0023] As a preferred solution, based on the case where the error between X A and X T is minimized, use one or more of the LMS method, the combination of the LMS method and the fixed-step gradient descent method, the combination of the LMS method and the variable-step gradient descent method, and artificial intelligence algorithms to solve for the parameters in the stimulation current I and the movement information transformation matrix S.

[0024] As a preferred solution, Step S5 includes:

[0025] Step S5.1: Analyze the collected facial movement images of the affected side, extract the key points of the face, and calculate the movement parameters of the key points;

[0026] Step S5.2: Based on the movement parameters of the key points, analyze the facial movement images and identify and obtain the facial movement parameters of the affected side.

[0027] In a second aspect, the present invention discloses a facial nerve closed-loop regulation and stimulation device based on visual feedback, comprising:

[0028] A healthy-side acquisition module for acquiring the facial movement pattern of the healthy side of the patient;

[0029] A stimulation current generation module for iteratively optimizing the generation process of the stimulation current with the facial movement pattern of the healthy side of the patient as the desired movement information and the facial movement pattern of the feedback diseased side as the actual movement information, and generating the corresponding stimulation current;

[0030] A transmission module for inputting the stimulation current into the implanted stimulation electrode in the patient's body. The stimulation current stimulates the wrapped facial nerve trunk through the implanted stimulation electrode to generate nerve electrical impulses, and the nerve electrical impulses are transmitted to the controlled distal muscles, causing the muscles on the diseased side of the patient to contract and generate movement;

[0031] A diseased-side acquisition module for an imaging acquisition device to acquire the facial movement images of the diseased side of the patient;

[0032] An identification module for analyzing the acquired facial movement images of the diseased side and identifying the facial movement parameters of the diseased side;

[0033] A determination module for determining the facial movement pattern of the diseased side according to the identified facial movement parameters of the diseased side;

[0034] A repeated execution module for repeatedly executing the methods in the healthy-side acquisition module, the stimulation current generation module, the transmission module, the diseased-side acquisition module, the identification module, and the determination module until facial stimulation is completed.

[0035] As a preferred solution, in the stimulation current generation module,

[0036] Taking the facial movement pattern of the healthy side of the patient as the desired movement information X T , and taking the facial movement pattern of the feedback diseased side as the actual movement information X A , the error between X A and X T is described by the following formula:

[0037] ||e|| = ||X A - X T || = ||S·I - X T ||;

[0038] There is the following mapping relationship between the stimulation current and the facial movement pattern of the feedback diseased side;

[0039] X A = S·I;

[0040] Where: I is the stimulation current in the n-channel stimulation electrode;

[0041] S is the transformation matrix of the movement information of the control muscle corresponding to the stimulation current;

[0042] Minimize X A and X T The error of is solved to obtain the parameters in the stimulation current I and the movement information transformation matrix S.

[0043] As a preferred solution, based on X A and X T When the error is minimized, one or more methods among the LMS method, the combination of the LMS method and the fixed-step gradient descent method, the combination of the LMS method and the variable-step gradient descent method, and the artificial intelligence algorithm are used to solve the parameters in the stimulation current I and the movement information transformation matrix S.

[0044] As a preferred solution, the recognition module includes:

[0045] The first recognition unit is used to analyze the collected facial movement images of the affected side, extract the key points of the face, and calculate the movement parameters of the key points;

[0046] The second recognition unit is used to analyze the facial movement images based on the movement parameters of the key points and identify and obtain the facial movement parameters of the affected side.

[0047] In a third aspect, the present invention discloses a computing device, including:

[0048] One or more processors;

[0049] A memory;

[0050] And one or more programs, where one or more programs are stored in the memory and are configured to be executed by one or more processors, and one or more programs include instructions of any of the above-mentioned facial nerve closed-loop regulation and stimulation methods based on visual feedback.

[0051] In a fourth aspect, the present invention discloses a storage medium, and the storage medium stores one or more computer-readable programs, and one or more programs include instructions, and the instructions are adapted to be loaded and executed by the memory to perform any of the above-mentioned facial nerve closed-loop regulation and stimulation methods based on visual feedback.

[0052] The present invention discloses a facial nerve closed-loop regulation and stimulation method and device based on visual feedback, which have the following beneficial effects:

[0053] According to the movement information generated after the actual stimulation of the muscle, the present invention feeds back the stimulation current in the implanted stimulation electrode that generates the movement, iteratively optimizes the generation of the stimulation current, makes it more in line with the expectation, and thus improves the stimulation effect. Description of the Drawings

[0054] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings required in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0055] Figure 1 It is a flowchart of the facial nerve closed-loop regulation and stimulation method provided by the embodiments of the present invention.

[0056] Figure 2 It is a schematic diagram of the implanted stimulating electrode provided by the embodiments of the present invention.

[0057] Figure 3 It is a schematic diagram of the wearable stimulating glasses provided by the embodiments of the present invention.

[0058] Figure 4 It is a flowchart of the healthy-side facial movement pattern recognition provided by the embodiments of the present invention. Detailed implementation manners

[0059] The following will describe in detail the preferred implementation manners of the present invention with reference to the accompanying drawings.

[0060] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0061] Using ordinal numbers such as "first", "second", "third", etc. to describe ordinary objects only represents different instances of similar objects and does not intend to imply that the objects so described must have a given order in terms of time, space, sorting, or any other way.

[0062] In addition, the expression of "including" elements is an "open-ended" expression. This "open-ended" expression only means that there are corresponding components or steps and should not be construed as excluding additional components or steps.

[0063] In order to achieve the purpose of the present invention, in some embodiments of the facial nerve closed-loop regulation and stimulation method based on visual feedback, as Figure 1 shown, the facial nerve closed-loop regulation and stimulation method includes:

[0064] Step S101: Collect the facial movement pattern of the healthy side of the patient;

[0065] Step S102: Using the facial movement pattern of the healthy side of the patient as the desired movement information and the facial movement pattern of the affected side obtained through feedback as the actual movement information, iteratively optimize the generation process of the stimulation current and generate the corresponding stimulation current.

[0066] Step S103: Input the stimulation current into the implanted stimulation electrode in the patient's body. The stimulation current stimulates the wrapped facial nerve trunk through the implanted stimulation electrode to generate nerve impulses. The nerve impulses are transmitted to the distal muscles under control, causing the muscles on the affected side of the patient to contract and produce movement.

[0067] Step S104: The image acquisition device acquires the facial movement images of the affected side of the patient.

[0068] Step S105: Analyze the acquired facial movement images of the affected side to identify the facial movement parameters of the affected side.

[0069] Step S106: Determine the facial movement pattern of the affected side based on the identified facial movement parameters of the affected side.

[0070] Step S107: Repeat Step S102 - Step S106 until facial stimulation is completed.

[0071] The present invention uses a stimulation electrode implanted in the body, which is wrapped around the facial nerve trunk or the branches of the facial nerve in the parotid gland, either inside or outside the parotid gland, after the facial nerve trunk exits the stylomastoid foramen. This electrode can output a stimulation current for stimulating the nerve fibers in the wrapped facial nerve trunk to generate electrical impulses and achieve the movement control of the corresponding muscles.

[0072] The generation of the stimulation current signal in the implanted stimulation electrode comes from the signal processing unit and the current driver (amplifier). According to the facial movement pattern of the patient, the pattern information includes: the amplitude, speed, duration of the movement, and their movement combinations, etc. After receiving the movement pattern, the signal processing unit will generate an excitation waveform of the corresponding waveform according to the pre-trained model and the adaptive error correction algorithm. The excitation waveform passes through the current driver to generate a stimulation current, which is sent to the implanted stimulation electrode in the body. The stimulation electrode stimulates the nearby nerve fibers through the stimulation current to generate electrical impulse signals and achieve the movement control of the distal muscles.

[0073] During the process of realizing movement control through specific nerve electrical stimulation, that is, during the process of nerve regulation, in order to ensure that the result of regulating or controlling muscle movement is consistent with the expected goal, the present invention iteratively optimizes the generation process of the stimulation current based on visual feedback.

[0074] Such as Figure 2As shown, the present invention uses a multi-channel stimulating electrode implanted into the facial nerve trunk. By means of the stimulating currents with different time sequences and parameters in the stimulating electrode, the internal branches of the facial nerve trunk near each stimulating electrode of the multi-channel are directly stimulated to generate nerve electrical impulses, which are transmitted to the distal neuromuscular effector to cause the contraction of the corresponding controlled muscle and achieve the corresponding movement.

[0075] Furthermore, under the action of the current stimulation of the external stimulating electrode, the nerve will generate corresponding signals of nerve electrical impulses according to the parameters of the current. The amplitude, frequency, waveform, duty cycle, etc. of the nerve electrical impulse signals will affect the movement achieved by controlling the muscle.

[0076] In some specific embodiments, a wearable stimulating glasses can be used for facial movement image acquisition and stimulating treatment. If the current driver is directly connected to the stimulating electrode and placed in the body, only the input waveform and control signal of the stimulating current are provided at the glasses end.

[0077] Taking the user wearing the glasses as the coordinate system:

[0078] On the left and right temple arms of the glasses, percutaneous electrodes LegL and LegR that are in direct contact with the skin are respectively arranged;

[0079] On the nose pad of the glasses frame, percutaneous electrodes NoseL and NoseR that are in direct contact with the skin are respectively arranged.

[0080] On the left and right temple arms of the glasses, near the glasses frame, cameras LenL and LenR are respectively arranged to observe the facial expressions of the wearer on the left and right sides, including the forehead, eyes, cheeks, mouth, etc.

[0081] For the feedback after the stimulation of the facial movement pattern on the affected side, the facial movements on the left and right sides of the eye wearer can be directly collected by the cameras LenL (left camera) and LenR (right camera) at the temple arm and the glasses frame of the glasses, and the specific movements can be recognized by means of image recognition or artificial intelligence.

[0082] In some other embodiments, an implantable nerve electrode method can also be used, such as using a Cuff electrode, a coiled electrode, a needle electrode, etc., to collect and analyze the nerve electrical signals that control the facial muscle movement in the facial nerve on the affected side to obtain the facial movement pattern on the affected side.

[0083] In some other embodiments, the myoelectric signals corresponding to the facial movement can also be collected through percutaneous muscle electrodes for analysis and interpretation to obtain the facial movement pattern on the affected side.

[0084] The present invention realizes the generation of the stimulating current, neuromuscular regulation and closed-loop visual feedback through the wearable glasses method.

[0085] The movement state of muscles is mainly determined by the controlled nerve electrical signals. Therefore, information such as the amplitude, frequency, waveform, and duty cycle of the stimulation current in the implanted stimulation electrodes will determine the nerve electrical impulses generated within the nerve.

[0086] For example, the amplitude of the amplitude stimulation current mainly determines the amplitude of the nerve impulse electrical signal and directly affects the contraction amplitude of the muscle; the stimulation duration determines the contraction time of the muscle after stimulation; the frequency and waveform affect the patient's perception and avoid discomfort such as muscle fatigue and acupuncture for the patient.

[0087] Therefore, according to the movement information generated after actual muscle stimulation, such as amplitude, duration, and amplitude change information corresponding to the movement pattern, feedback is performed on the stimulation current in the implanted electrode that generates the movement, so that the stimulation current conforms to the designed target and is consistent with the actual situation, and can achieve the purpose of dynamic intelligent adjustment for different people and different physical states of the same person.

[0088] Taking the facial movement pattern of the healthy side of the patient as the desired movement information X T , and taking the facial movement pattern of the affected side obtained by feedback as the actual movement information X A , according to the amplitude, waveform, and duration of the stimulation current, there is the following mapping relationship between the stimulation current and the facial movement pattern of the affected side obtained by feedback;

[0089] X A = S·I;

[0090] Where: I is the stimulation current in the stimulation electrodes of n channels, I = [i1, i2,..., i n T ;

[0091] S is the transformation matrix of the movement information of the muscle controlled by the stimulation current;

[0092] X A is the movement information generated by the muscle receiving the stimulation, including displacement and speed, that is:

[0093] X A = [x1, v1, x2, v2,..., x m , v m T ;

[0094] Where:

[0095] x i and v i are respectively the displacement and speed information of the i-th key point after muscle stimulation movement;

[0096] m is the number of key points.

[0097] S is the transformation matrix of the movement information of the controlled muscle corresponding to the stimulation current, and S is a 2m×n matrix;

[0098]

[0099] Therefore, the stimulation current is:

[0100] I = S -1 ·X A .

[0101] Taking the facial movement pattern of the healthy side of the patient as the desired movement information X T , and taking the facial movement pattern of the affected side of the feedback as the actual movement information X A , the following formula is used to describe X A and X T error:

[0102] ||e|| = ||X A - X T || = ||S·I - X T ||;

[0103]

[0104] For parameters such as movement amplitude, speed, and time interval, different weighting parameters can be designed, and various algorithms are used to solve the parameters in the stimulation current I and the movement information transformation matrix S.

[0105] Method 1: Least Mean Square, LMS.

[0106] LMS is a classic adaptive filtering algorithm. To minimize e, the method of least squares to find the extreme value can be used to calculate the parameters in the stimulation current and the transformation matrix S of muscle movement.

[0107] Method 2: On the basis of Method 1, the gradient descent method is used. For example, the weight formula is:

[0108]

[0109] where: μ is the step size parameter, e k is the error signal, and k is the k-step error iteration.

[0110] Method 3: In order to further improve the calculation speed and accuracy, a variable step size method is adopted, that is, each step is μ k ,

[0111]

[0112] where, μ k = f(μ k-1 , ek-1 ) According to the error e recognized last time k-1 , change the step size or the rate of change of the step size to achieve fast calculation of S k goal.

[0113] Method 4: In addition to the above methods, artificial intelligence algorithms can also be used, such as convolutional neural network CNN and recurrent neural network RNN, to calculate parameters by combining visual feedback results.

[0114] Furthermore, the above step S105 includes:

[0115] Step S105.1: Analyze the collected facial movement images of the affected side, extract the key points of the face, and calculate the movement parameters of the key points;

[0116] Step S105.2: Based on the movement parameters of the key points, analyze the facial movement images to identify and obtain the facial movement parameters of the affected side.

[0117] The face mainly includes five regions: forehead, eyes, zygomatic, cheek, and mandible. In terms of visual detection and recognition, the present invention identifies the movement information of eye closure (such as position, speed, time interval, etc.) and the movement information of facial expressions (such as movement displacement and duration, etc.) according to the key points.

[0118] In some other embodiments, for patients with unilateral facial paralysis, an implantable brain-computer interface method can be used. For the facial expressions of the healthy side, implantable nerve electrodes are used to collect and analyze the nerve electrical signals, and compare them with the facial movement parameters of the images collected by an image acquisition device (such as a camera) to correct the results of the collected electrode facial movement parameters.

[0119] Such as Figure 4 shown, specifically including the following steps:

[0120] Step A: Use the implantable nerve electrode to collect the nerve electrical signals of the facial movement of the healthy side of the patient in real time, decode them, and obtain the electrode facial movement parameters;

[0121] Synchronously, use the image acquisition device to collect the facial movement images of the healthy side of the patient in real time, and identify them to obtain the image facial movement parameters;

[0122] Step B: Based on the feedback of the image facial movement parameters, iteratively optimize the decoding process of the nerve electrical signals, and output the facial movement pattern of the healthy side.

[0123] Specifically, step A includes:

[0124] Step A1: Use an implanted nerve electrode (such as a nerve signal sensor) to collect the nerve electrical signals of the healthy side of the patient's face in real time, such as the nerve electrical signals in the facial nerve that control facial expressions and eye closure, etc.;

[0125] Step A2: Preprocess the collected nerve electrical signals, such as filtering and noise reduction to reduce myoelectric interference, external interference, etc.;

[0126] Step A3: Based on the image facial movement parameters, analyze the preprocessed nerve electrical signals, decode the muscle movement information corresponding to the nerve electrical signal waveforms in different nerve bundles, so as to obtain the electrode facial movement parameters.

[0127] Adopt the LMS method, the combination of the LMS method and the fixed-step gradient descent method, the combination of the LMS method and the variable-step gradient descent method, artificial intelligence algorithms, etc. to adjust the identified nerve electrical signal parameters to reduce errors and improve the decoding and recognition accuracy.

[0128] LMS is a classic adaptive filtering algorithm that adjusts and optimizes the decoding parameters of the filter or algorithm by minimizing the mean square value of the error signal.

[0129] To make minimum, the method of finding the extreme value can be used to calculate the parameters ω of the filter or algorithm k .

[0130] e ki = y ki - z ki ;

[0131] y k = ω k ·x k ;

[0132] Where: x k is the input signal of the nerve electrical signal;

[0133] y k is the decoded output signal (i.e., the electrode facial movement parameter), y ki is the k-th step and i-th movement amount of the nerve electrical signal decoding;

[0134] z k is the motion vector of visual recognition (i.e., the image facial movement parameter), z ki is the k-th step and 1st movement amount of visual recognition;

[0135] ω k is the coefficient of the filter or algorithm.

[0136] Further, based on the above method, the gradient descent method is adopted to quickly calculate the filter or algorithm parameters and improve the stability of the algorithm.

[0137] The weight formula is:

[0138] ω k+1 = ω k + μ · e k · x k ;

[0139] Where: μ is the step size parameter, and e k is the error signal.

[0140] Further, the difference from the above method is that, in order to further improve the calculation accuracy and speed, a variable step size method is adopted, that is, the weight formula is:

[0141] ω k+1 = ω k + μ k · e k · x k ;

[0142] Wherein, μ k = f(μ k-1 , e k-1 ), according to the error e k-1 recognized last time, change the step size or the rate of change of the step size to achieve the goal of quickly calculating ω k .

[0143] Of course, in some other embodiments, artificial intelligence algorithms can also be adopted, such as convolutional neural network CNN and recurrent neural network RNN, to combine the visual feedback results for neural electrical signal decoding and motion pattern recognition.

[0144] In some other embodiments, the present invention discloses a facial nerve closed-loop regulation and stimulation device based on visual feedback, including:

[0145] A healthy side acquisition module for acquiring the facial motion pattern of the healthy side of the patient;

[0146] A stimulation current generation module for taking the facial motion pattern of the healthy side of the patient as the desired motion information and the facial motion pattern of the feedback diseased side as the actual motion information, iteratively optimizing the generation process of the stimulation current, and generating the corresponding stimulation current;

[0147] A transmission module for inputting the stimulation current into the implanted stimulation electrode in the patient's body. The stimulation current stimulates the wrapped facial nerve trunk through the implanted stimulation electrode to generate nerve impulses, and the nerve impulses are transmitted to the controlled distal muscles, causing the muscles on the diseased side of the patient to contract and generate motion;

[0148] The affected - side acquisition module is used for the imaging acquisition device to acquire the facial movement images of the affected side of the patient;

[0149] The recognition module is used to analyze the acquired facial movement images of the affected side and recognize the facial movement parameters of the affected side;

[0150] The determination module is used to determine the facial movement pattern of the affected side according to the recognized facial movement parameters of the affected side;

[0151] The repeated execution module is used to repeatedly execute the methods in the healthy - side acquisition module, the stimulation current generation module, the transmission module, the affected - side acquisition module, the recognition module, and the determination module until facial stimulation is completed.

[0152] Furthermore, in the stimulation current generation module,

[0153] Taking the facial movement pattern of the healthy side of the patient as the desired movement information \(X\) T , and taking the feedback facial movement pattern of the affected side as the actual movement information \(X\) A , using the following formula to describe the error between \(X\) A and \(X\) T :

[0154] \(\vert\vert e\vert\vert=\vert\vert X\) A - \(X\) T \(\vert\vert=\vert\vert S\cdot I - X\) T \(\vert\vert\);

[0155] There is the following mapping relationship between the stimulation current and the feedback facial movement pattern of the affected side;

[0156] \(X\) A \(=S\cdot I\);

[0157] where: \(I\) is the stimulation current in the \(n\) - channel stimulation electrodes;

[0158] \(S\) is the movement information transformation matrix of the control muscles corresponding to the stimulation current;

[0159] Minimize the error between \(X\) A and \(X\) T , and solve for the parameters in the stimulation current \(I\) and the movement information transformation matrix \(S\).

[0160] Furthermore, based on the case where the error between \(X\) A and \(X\) T is minimized, use one or more of the LMS method, the combination of the LMS method and the fixed - step - size gradient - descent method, the combination of the LMS method and the variable - step - size gradient - descent method, and artificial intelligence algorithms to solve for the parameters in the stimulation current \(I\) and the movement information transformation matrix \(S\).

[0161] Furthermore, the recognition module includes:

[0162] The first recognition unit is used to analyze the collected facial movement images of the affected side, extract the key points of the face, and calculate the movement parameters of the key points.

[0163] The second recognition unit is used to analyze the facial movement images based on the movement parameters of the key points, and identify and obtain the facial movement parameters of the affected side.

[0164] Furthermore, it should be noted that: when the above-mentioned facial nerve closed-loop regulation and stimulation device based on visual feedback is performing regulation, only the above-mentioned division of each functional module is used for illustration. In practical applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the facial nerve closed-loop regulation and stimulation device based on visual feedback is divided into different functional modules to complete all or part of the functions described above.

[0165] In addition, the embodiments of the facial nerve closed-loop regulation and stimulation device based on visual feedback and the facial nerve closed-loop regulation and stimulation method provided in the above embodiments belong to the same concept. For the specific implementation process, please refer to the method embodiments and will not be elaborated here.

[0166] In some other embodiments, the present invention discloses a computing device, including:

[0167] One or more processors;

[0168] A memory;

[0169] And one or more programs, where one or more programs are stored in the memory and are configured to be executed by one or more processors. The one or more programs include instructions of any of the above-mentioned facial nerve closed-loop regulation and stimulation methods based on visual feedback.

[0170] In some other embodiments, the present invention discloses a storage medium, which stores one or more computer-readable programs. The one or more programs include instructions, and the instructions are adapted to be loaded and executed by the memory to perform any of the above-mentioned facial nerve closed-loop regulation and stimulation methods.

[0171] The present invention discloses a facial nerve closed-loop regulation and stimulation method and device based on visual feedback, which has the following beneficial effects:

[0172] The present invention feeds back the stimulation current in the implanted stimulation electrode that generates the movement according to the movement information generated after the actual stimulation of the muscle, so as to iteratively optimize the generation of the stimulation current to make it more in line with the expectation, thereby improving the stimulation effect.

[0173] The foregoing has shown and described the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and the description in the specification are only to illustrate the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of the present invention claimed is defined by the appended claims and their equivalents.

Claims

1. A facial nerve closed-loop regulation and stimulation method based on visual feedback, characterized in that, Including: Step S1: Collect the facial movement pattern of the healthy side of the patient; Step S2: Take the facial movement pattern of the healthy side of the patient as the desired movement information, and the facial movement pattern of the affected side obtained by feedback as the actual movement information, and iteratively optimize the generation process of the stimulation current to generate the corresponding stimulation current; Step S3: Input the stimulation current into the implanted stimulation electrode in the patient's body. The stimulation current stimulates the wrapped facial nerve trunk through the implanted stimulation electrode to generate nerve impulses. The nerve impulses are transmitted to the controlled distal muscles, causing the muscles on the affected side of the patient to contract and generate movement; Step S4: The image acquisition device collects the facial movement images of the affected side of the patient; Step S5: Analyze the collected facial movement images of the affected side to identify the facial movement parameters of the affected side; Step S6: Determine the facial movement pattern of the affected side according to the identified facial movement parameters of the affected side; Step S7: Repeat Step S2 - Step S6 until facial stimulation is completed.

2. The facial nerve closed-loop regulation and stimulation method according to claim 1, wherein In the above Step S2, Take the facial movement pattern of the healthy side of the patient as the expected movement information X T , and take the facial movement pattern of the affected side of the feedback as the actual movement information X A , and use the following formula to describe X A and X T error: ||e|| = ||X A -X T || = ||S·I - X T ||; The following mapping relationship exists between the stimulation current and the facial movement pattern of the affected side obtained by feedback; X A = S·I; Where: I is the stimulation current in the stimulation electrodes of n channels; S is the movement information transformation matrix of the controlled muscles corresponding to the stimulation current; Minimize X A and X T for the error, and solve for each parameter in the stimulation current I and the motion information transformation matrix S.

3. The facial nerve closed-loop regulation and stimulation method according to claim 2, wherein Based on X A and X T In the case of the smallest error, one or more methods among the LMS method, the combination of the LMS method and the fixed-step gradient descent method, the combination of the LMS method and the variable-step gradient descent method, and artificial intelligence algorithms are used to solve for the parameters in the stimulation current I and the motion information transformation matrix S.

4. The facial nerve closed-loop regulation and stimulation method according to any one of claims 1-3, characterized in that The above Step S5 includes: Step S5.1: Analyze the collected facial movement images of the affected side, extract the key points of the face, and calculate the movement parameters of the key points; Step S5.2: Based on the movement parameters of the key points, analyze the facial movement images to identify and obtain the facial movement parameters of the affected side.

5. A facial nerve closed-loop regulation and stimulation device based on visual feedback, characterized in that Including: A healthy side acquisition module for collecting the facial movement pattern of the healthy side of the patient; A stimulation current generation module for taking the facial movement pattern of the healthy side of the patient as the desired movement information, and the facial movement pattern of the affected side obtained by feedback as the actual movement information, and iteratively optimizing the generation process of the stimulation current to generate the corresponding stimulation current; A transmission module for inputting the stimulation current into the implanted stimulation electrode in the patient's body. The stimulation current stimulates the wrapped facial nerve trunk through the implanted stimulation electrode to generate nerve impulses. The nerve impulses are transmitted to the controlled distal muscles, causing the muscles on the affected side of the patient to contract and generate movement; An affected side acquisition module for the image acquisition device to collect the facial movement images of the affected side of the patient; An identification module for analyzing the collected facial movement images of the affected side to identify the facial movement parameters of the affected side; A determination module for determining the facial movement pattern of the affected side according to the identified facial movement parameters of the affected side; A repeated execution module for repeatedly executing the methods in the healthy side acquisition module, stimulation current generation module, transmission module, affected side acquisition module, identification module, and determination module until facial stimulation is completed.

6. The facial nerve closed-loop regulation and stimulation device according to claim 5, wherein In the above stimulation current generation module, Take the facial movement pattern of the healthy side of the patient as the desired movement information X T , and take the facial movement pattern of the affected side of the feedback as the actual movement information X A , and describe X using the following formula A and X T error: ||e|| = ||X A -X T || = ||S·I - X T ||; The following mapping relationship exists between the stimulation current and the facial movement pattern of the affected side obtained by feedback; X A = S·I; Where: I is the stimulation current in the stimulation electrodes of n channels; S is the movement information transformation matrix of the controlled muscles corresponding to the stimulation current; Minimize X A and X T of the error, and solve for each parameter in the stimulation current I and the motion information transformation matrix S.

7. The facial nerve closed-loop regulation and stimulation device according to claim 6, characterized in that, Based on X A and X T In the case of the smallest error, one or more methods among the LMS method, the combination of the LMS method and the fixed-step gradient descent method, the combination of the LMS method and the variable-step gradient descent method, and artificial intelligence algorithms are used to solve for the parameters in the stimulation current I and the motion information transformation matrix S.

8. The facial nerve closed-loop regulation and stimulation device according to any one of claims 5-7, characterized in that, The above identification module includes: A first identification unit for analyzing the collected facial movement images of the affected side, extracting the key points of the face, and calculating the movement parameters of the key points; A second recognition unit, configured to analyze facial motion images based on the motion parameters of key points, and recognize and obtain the facial motion parameters of the affected side.

9. A computing device, characterized in that, Comprising: One or more processors; A memory; And one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for the method of closed-loop regulation and stimulation of the facial nerve based on visual feedback according to any one of claims 1-4 above.

10. Storage medium, characterized in that, The storage medium stores one or more computer-readable programs, and the one or more programs include instructions, and the instructions are adapted to be loaded and executed by the memory for the method of closed-loop regulation and stimulation of the facial nerve based on visual feedback according to any one of claims 1-4 above.

Images