Medical instrument operation simulation teaching method and platform based on electromagnetic detection

Abstract

The invention relates to the technical field of medical teaching, in particular to a medical instrument operation simulation teaching method and platform based on electromagnetic detection, and the method comprises the following steps: arranging a medical teaching three-dimensional virtual scene in an induction space of a sensor array; respectively sensing magnetic signals of magnetic markers arranged on the hands of an operation student and on the end part of an operation instrument through the sensor array; performing practical operation evaluation on medical operation of an operation student in the medical teaching three-dimensional virtual scene based on the magnetic signal; and performing operation feedback and correction guidance on the medical operation of the operation trainee according to the evaluation result of the practical operation evaluation. Electromagnetic detection is applied to virtual teaching, the hands of students, the ends of instruments and the like are marked through the magnetic markers to form multi-point magnetic marks, real-time magnetic signals are detected through the sensor array, operation tracks are drawn, depth, frequency and other data are recorded, a magnetic coordinate system is combined with a mixed reality coordinate system, and the operation quality is evaluated efficiently and visually.

Description

Technical Field

[0001] The present invention relates to the field of medical teaching technology, and in particular to a medical device operation simulation teaching method and platform based on electromagnetic detection. Background Art

[0002] The development of minimally invasive diagnosis and treatment, robot-assisted diagnosis and treatment, etc., has promoted the continuous precision, minimally invasive and safe treatment of diseases. However, minimally invasive diagnosis and treatment methods such as laparoscopic surgery, endoscopic diagnosis and treatment, and da Vinci robot surgery are difficult to learn and have a long learning curve. Young doctors often undergo long-term, systematic animal experiments, model training, surgical video learning and other skills learning and operation training, as well as virtual simulation teaching that has emerged in recent years.

[0003] Virtual simulation teaching can be used for surgical simulation training, including laparoscopic gastrointestinal surgery, thoracic surgery, gastrointestinal endoscopic diagnosis and treatment, etc. It can simulate surgical operations in a virtual environment and provide efficiency, interactivity and safety of surgical training. However, today's virtual simulation teaching platforms focus on surgical operation training simulation, and the evaluation of operation quality mainly depends on subjective evaluations such as operation success rate and action standardization.

[0004] At present, the operation quality evaluation of virtual simulation teaching platform mainly depends on the operation success rate, action standardization, operation time consumption, etc. These relatively subjective evaluation methods may lead to insufficient accuracy of teaching evaluation, difficulty in matching operation standards, and affect the learning effect and efficiency of students. Therefore, there is a need for an auxiliary technology to improve the virtual simulation teaching evaluation of minimally invasive diagnosis and treatment, so as to further improve the evaluation quality and learning effect, and promote the development of precise minimally invasive diagnosis and treatment in clinical practice. Summary of the invention

[0005] The purpose of the present invention is to provide a medical device operation simulation teaching method and platform based on electromagnetic detection, so as to solve the technical problems in the prior art that the operation quality evaluation mainly depends on the operation success rate, action standardization, operation time consumption, etc. These relatively subjective evaluation methods may lead to insufficient accuracy of teaching evaluation, difficulty in matching operation specifications, and affect the learning effect and efficiency of students.

[0006] In order to solve the above technical problems, the present invention specifically provides the following technical solutions:

[0007] A medical device operation simulation teaching method based on electromagnetic detection comprises the following steps:

[0008] Selecting a three-dimensional virtual scene for medical teaching, and arranging the three-dimensional virtual scene for medical teaching in a sensing space of a sensor array;

[0009] The sensor array senses magnetic signals of magnetic markers disposed on the hands of the trainee and on the ends of the operating instruments held by the trainee respectively;

[0010] Based on the magnetic signals, a practical evaluation is performed on the medical operation of the trainee in the three-dimensional virtual scene of medical teaching;

[0011] Based on the evaluation results of the practical evaluation, operational feedback and correction guidance are provided to trainees on their medical operations in the three-dimensional virtual scene of medical teaching.

[0012] As a preferred embodiment of the present invention, the three-dimensional virtual scene for medical teaching comes from the three-dimensional modeling of real medical tissue anatomical structures, and the real medical tissue anatomical structures include the anatomical structure of the human respiratory system, the anatomical structure of the human digestive system, the anatomical structure of the human blood circulation system, the anatomical structure of the human urinary system, and the anatomical structure of the human nervous system. The three-dimensional virtual scene for medical teaching includes the three-dimensional virtual scene of the respiratory system, the three-dimensional virtual scene of the digestive system, the three-dimensional virtual scene of the blood circulation system, the three-dimensional virtual scene of the urinary system, and the three-dimensional virtual scene of the nervous system.

[0013] As a preferred embodiment of the present invention, the practical evaluation includes:

[0014] The actual depth, actual amplitude, actual speed and actual strength of the pulling and cutting operation of the tissue part by the operator are obtained by using the magnetic signal of the magnetic marker arranged at the end of the operating instrument held by the operator trainee;

[0015] Using the magnetic signals of the magnetic markers placed on the hands of the trainees, the actual hand motion trajectory, actual hand shaking amplitude and actual instrument shaking amplitude of the trainees' hands are obtained;

[0016] The trainee's operational stability and standardization are evaluated based on the actual depth, actual amplitude, actual speed, actual force, actual hand movement trajectory, actual hand shaking amplitude, and actual instrument shaking amplitude.

[0017] As a preferred solution of the present invention, a method for obtaining the actual depth, actual amplitude, actual speed and actual force of the pulling and cutting operation of the tissue part by the operator using the magnetic signal of the magnetic marker arranged at the end of the operating instrument held by the operator includes:

[0018] The host computer program is used to track the magnetic signal of the magnetic marker arranged at the end of the operating instrument held by the trainee, wherein the host computer program automatically starts a timer inside and records the corresponding timer time after receiving a signal from the sensor array each time;

[0019] The signal of the sensor array is parsed into the coordinate position information of the magnetic marker arranged at the end of the operating instrument held by the trainee through the EPSO-LM algorithm, and the motion information of the magnetic marker arranged at the end of the operating instrument held by the trainee is formed based on the one-to-one correspondence between the position of the magnetic target and the time of the timer, wherein the nth path point in the motion information is [t n ,p n ] T ,n=1,2,3…,N, where t n and p n are the reading and position coordinate vector of the timer when the magnetic marker arranged at the end of the operating instrument held by the trainee moves to the nth path point, p n =[p nx ,p ny ,p nz ] T , where p nx ,p ny ,p nz They are p in the three-dimensional rectangular coordinate system n The three components of the coordinates in the X, Y, and Z axis directions, T is the transposition operator;

[0020] The instantaneous speed of the magnetic marker placed at the end of the operating instrument held by the trainee is approximately described by the average speed of the marker passing through two adjacent path points. According to the known motion information consisting of discrete coordinates, the instantaneous speed information of the magnetic marker placed at the end of the operating instrument held by the trainee is also composed of discrete data, and t is recorded. v The instantaneous velocity V of the magnetic marker placed at the end of the operating instrument held by the trainee at time s (t v ) is calculated as:

[0021]

[0022] In the formula, t n and p n are the reading and position coordinate vector of the timer when the magnetic marker arranged at the end of the operating instrument held by the trainee moves to the nth path point, t n+1 and p n+1 They are the reading and position coordinate vector of the timer when the magnetic marker arranged at the end of the operating instrument held by the trainee moves to the n+1th path point, and N represents the number of position data points collected during the entire operation process;

[0023] Based on the instantaneous velocity of the magnetic marker placed at the end of the operating instrument held by the trainee, the average acceleration method is used to approximate the instantaneous acceleration of the magnetic marker during its movement, and t aThe instantaneous acceleration A of the magnetic marker placed at the end of the operating instrument held by the trainee at time s (t a ) is calculated as:

[0024]

[0025] In the formula, for The instantaneous speed of the magnetic marker placed at the end of the operating instrument held by the trainee at the time. for The instantaneous speed of the magnetic marker placed at the end of the operating instrument held by the trainee at time t n and p n are the reading and position coordinate vector of the timer when the magnetic marker arranged at the end of the operating instrument held by the trainee moves to the nth path point, t n+1 and p n+1 are the reading and position coordinate vector of the timer when the magnetic marker arranged at the end of the operating instrument held by the trainee moves to the n+1th path point, t n+2 and p n+2 They are respectively the reading and position coordinate vector of the timer when the magnetic marker arranged at the end of the operating instrument held by the trainee moves to the n+2th path point;

[0026] t a The instantaneous acceleration of the magnetic marker placed at the end of the operating instrument held by the trainee is converted into a three-dimensional coordinate system, and A is obtained. s (t a )=[a sx ,a sy ,a sz ] T , where a sx ,a sy ,a sz A in the three-dimensional rectangular coordinate system s (t a ) components in the X, Y, and Z axis directions;

[0027] The force analysis of the tissue parts is carried out, and the force diagram of the tissue parts is obtained, including: F = [f x ,f y ,f z ] T is the force exerted on the tissue, f x ,f y ,f zIt is the three components of F in the X, Y, and Z axes in a three-dimensional rectangular coordinate system. G is the gravity exerted on the local tissue under traction. F1 and F2 are the components of the traction force in the vertical and horizontal directions, respectively. According to the laws of mechanics, the relationship between F and G is:

[0028]

[0029] Where m is the mass of the local tissue being stretched, g is the gravitational acceleration of the earth, and its direction is along the negative direction of the Z axis;

[0030] Using A s (t a )=[a sx ,a sy ,a sz ] T Find the tissue location at t a The pulling force F = [f x ,f y ,f z ] T ,get:

[0031]

[0032] Based on the movement information of the magnetic marker arranged at the end of the operating instrument held by the trainee, the moving distance of the magnetic marker representing the actual depth and actual amplitude of tissue pulling and cutting is obtained;

[0033] The instantaneous velocity and instantaneous acceleration of the magnetic markers placed on the end of the operating instrument held by the trainee are used to characterize the actual speed of tissue pulling and cutting;

[0034] The pulling force of the magnetic marker placed on the end of the operating instrument held by the trainee is used to represent the actual force of tissue pulling and cutting.

[0035] As a preferred solution of the present invention, a method for obtaining the actual hand motion trajectory, actual hand shaking amplitude and actual instrument shaking amplitude of the operator's hand operation by using the magnetic signal of the magnetic marker arranged on the operator's hand includes:

[0036] The theoretical value of the three components of the magnetic field generated by the magnetic marker placed on the operator's hand at the mth sensor in the sensor array is described by the magnetic dipole formula:

[0037]

[0038] In the formula, B T =μ r μ0M T / 4π,μ ris the relative magnetic permeability of the medium where the magnetic sensor is located, μ0 is the vacuum magnetic permeability, M T is the magnitude of the magnetic moment vector, [B mx ,B my ,B mz ] T are the three components of the magnetic induction intensity at the mth magnetic sensor, m = 1, 2, 3, ..., M, M is the total number of magnetic sensors on the magnetic sensor array, P m0 Indicates that the position of the magnetic mark P0 = [a, b, c] T Point to the mth magnetic sensor coordinate P m =[x m ,y m ,z m ] T The vector, P m is the coordinate of the mth magnetic sensor, P m Using the three-dimensional rectangular coordinate system [x m ,y m ,z m ] T The three parameters represent that H0 is the magnetic moment vector of the magnetic marker;

[0039] The sum of the squares of the three-component errors between the magnetic sensor measurement value and the theoretical value is used as the objective function of the magnetic sensor array system error. The objective function of the magnetic sensor array system error is:

[0040]

[0041] Where, E is the magnetic sensor array system error, [B ′ mx ,B ′ my ,B ′ mz ] T is the three-component measurement value of the magnetic induction intensity of the mth magnetic sensor in the X, Y, and Z axis directions in the three-dimensional rectangular coordinate system, [B mx ,B my ,B mz ] T is the three-component theoretical value of the magnetic induction intensity of the mth magnetic sensor in the X, Y, and Z axis directions in the three-dimensional rectangular coordinate system, and M is the total number of magnetic sensors on the magnetic sensor array;

[0042] The EPSO-LM algorithm can be used to solve the nonlinear equation E=0, thereby solving the coordinate position of the magnetic marker placed on the operator's hand. Based on the fact that the magnetic marker placed on the operator's hand is rigidly linked to the operator's hand, the position information of the magnetic marker is continuously collected as the position information of the hand, and the position information of the magnetic marker is recorded corresponding to the time, and the actual hand motion trajectory of the hand position information changing with time is drawn, wherein the actual motion trajectory The hand position at the moment is In the formula, In the three-dimensional rectangular coordinate system The coordinates of the hand position information in the X, Y, and Z axes at the moment, where n is a positive integer;

[0043] Keep your hands The amplitude of the shaking at the time is used during operation Hand position at the moment Distance from the average hand position during the entire process The actual hand shaking amplitude is measured by the straight-line distance, and the calculation formula is:

[0044]

[0045] In the formula, for The hand shaking amplitude at the moment, N represents the number of position data points collected during the entire operation;

[0046] During the use of the electromagnetic detection rod, based on the fact that the tip of the electromagnetic detection rod is rigidly linked to the electromagnetic coil, the position information of the tip of the electromagnetic detection rod is calculated by detecting the position of the electromagnetic coil, wherein the position information of the tip of the electromagnetic detection rod is:

[0047] P cusp =P pole +LH pole ;

[0048] Where P cusp is the position vector of the tip of the electromagnetic probe rod, P pole and H pole are the position vector and the axial direction vector of the cylindrical electromagnetic coil built into the electromagnetic detection rod, respectively, and L is the distance from the center of the electromagnetic coil to the tip of the detection rod;

[0049] The tip position coordinates at each moment are recorded as the motion information of the tip of the electromagnetic detection rod, and the instantaneous velocity and instantaneous acceleration calculation formula of the magnetic marker arranged at the end of the operating instrument held by the trainee are used to obtain the instantaneous velocity and instantaneous acceleration of the tip of the electromagnetic detection rod. Then, the calculation formula of the hand shaking amplitude is used to calculate the shaking amplitude of the electromagnetic detection rod during use as the actual instrument shaking amplitude.

[0050] As a preferred solution of the present invention, a method for providing operation feedback and correction guidance to trainees for medical operations in a three-dimensional virtual scene for medical teaching according to the evaluation results of the practical evaluation includes:

[0051] The coordinate transformation matrix T between the magnetic coordinate system of the sensor array and the virtual coordinate system of the three-dimensional virtual scene of medical teaching is used. A B , the actual depth, actual amplitude, actual speed, actual strength of the pulling and cutting operations of the tissue parts by the trainees, as well as the actual hand movement trajectory, actual hand shaking amplitude, and actual instrument shaking amplitude of the trainees are converted and displayed from the magnetic coordinate system of the sensor array to the virtual coordinate system of the three-dimensional virtual scene of medical teaching;

[0052] The actual depth, actual amplitude, actual speed, actual strength of the pulling and cutting operations of the tissue parts displayed by the trainees in the three-dimensional virtual scene of medical teaching, as well as the actual hand movement trajectory, actual hand shaking amplitude, and actual instrument shaking amplitude of the trainees are compared with the standard values.

[0053] When the actual depth, amplitude, speed, and force of the pulling and cutting operations of the tissue parts, as well as the actual hand movement trajectory, hand shaking amplitude, and instrument shaking amplitude of the trainees are consistent with the standard values, positive feedback will be given to the trainees;

[0054] When the actual depth, actual amplitude, actual speed, actual force of the pulling and cutting operations on the tissue parts, as well as the actual hand movement trajectory, actual hand shaking amplitude, and actual instrument shaking amplitude of the trainee are inconsistent with the standard values, the trainee will be given negative feedback and corrective guidance.

[0055] As a preferred embodiment of the present invention, the method for correcting the guide includes:

[0056] The depth, amplitude, speed, strength of the pulling and cutting operations on the tissue parts by the trainees at the moment of negative feedback, as well as the standard values ​​of the trainees' hand movement trajectory, hand shaking amplitude, and instrument shaking amplitude are displayed in the three-dimensional virtual scene of medical teaching, guiding the trainees to make corrections to the actual depth, actual amplitude, actual speed, actual strength, actual hand movement trajectory, actual hand shaking amplitude, and actual instrument shaking amplitude according to the standard values.

[0057] As a preferred solution of the present invention, the coordinate transformation matrix between the magnetic coordinate system of the sensor array and the virtual coordinate system of the three-dimensional virtual scene of medical teaching is The establishment methods include:

[0058] The magnetic coordinate system of the sensor array where the magnetic marker is located is recorded as coordinate system A, and the virtual coordinate system of the three-dimensional virtual scene of medical teaching is recorded as coordinate system B, where the coordinate transformation matrix from coordinate system A to B is The calculation formula is:

[0059]

[0060] In the formula, is a 3×3 coordinate rotation matrix, is a 3×1 coordinate translation matrix;

[0061] Select four non-coplanar feature points in the space, and record the coordinates of the kth feature point in the coordinate system A as The coordinates in coordinate system B are

[0062] Through the coordinate transformation matrix The coordinates P of any point in the coordinate system A A Transform to coordinate P in coordinate system B B ,get:

[0063]

[0064] Using the known coordinates of the four characteristic points, a system of 12 equations is listed, and the coordinate transformation matrix is ​​solved simultaneously. The 9 unknowns of the coordinate translation matrix and the 3 unknowns of the coordinate translation matrix, a total of 12 unknowns, give:

[0065]

[0066] Using the least squares method to find the best fit gives and Then solve the coordinate transformation matrix

[0067] As a preferred solution of the present invention, the present invention provides a medical device operation simulation teaching platform based on electromagnetic detection, which is applied to a medical device operation simulation teaching method based on electromagnetic detection. The platform includes:

[0068] A sensor array, used to sense magnetic signals of magnetic markers arranged on the hands of the trainee operator and on the ends of the operating instruments held by the trainee operator;

[0069] Magnetic markers are used to be placed on the hands of trainees and on the ends of the operating instruments held by trainees to generate magnetic signals for induction by the sensor array;

[0070] A data processing unit, used for performing a practical evaluation of the medical operation performed by the trainee in the three-dimensional virtual scene of medical teaching based on the magnetic signal;

[0071] An interface interaction unit, used to provide operational feedback and correction guidance to trainees on medical operations in a three-dimensional virtual scene for medical teaching according to the evaluation results of the practical evaluation;

[0072] The data storage unit is used to store three-dimensional virtual scenes for medical teaching.

[0073] As a preferred embodiment of the present invention, the magnetic marker includes magnetic fluid, magnetic hydrogel, magnetic soft patch, sensor and coil;

[0074] The sensor array is an array composed of magnetic sensors of different models.

[0075] Compared with the prior art, the present invention has the following beneficial effects:

[0076] The present invention applies electromagnetic detection to virtual simulation teaching. Magnetic markers are used to mark the trainees' hands, instrument ends, etc. to form multi-point magnetic markings. Real-time magnetic signals are detected through a sensor array. The program draws the operation trajectory, records depth, frequency and other data, and combines the magnetic coordinate system with the mixed reality coordinate system to efficiently and intuitively evaluate the operation quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0078] Figure 1 A flow chart of a medical device operation simulation teaching method based on electromagnetic detection provided by an embodiment of the present invention;

[0079] Figure 2A block diagram of a medical device operation simulation teaching system based on electromagnetic detection provided by an embodiment of the present invention;

[0080] Figure 3 A tissue site force diagram provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0081] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0082] like Figure 1 As shown, the present invention provides a medical device operation simulation teaching method based on electromagnetic detection, comprising the following steps:

[0083] Selecting a three-dimensional virtual scene for medical teaching, and arranging the three-dimensional virtual scene for medical teaching in a sensing space of a sensor array;

[0084] The sensor array senses magnetic signals of magnetic markers arranged on the hands of the trainee and on the ends of the operating instruments held by the trainee respectively;

[0085] Based on magnetic signals, the medical operations of trainees in the three-dimensional virtual scene of medical teaching are evaluated practically;

[0086] Based on the evaluation results of the practical evaluation, operational feedback and correction guidance are provided to trainees on their medical operations in the three-dimensional virtual scene of medical teaching.

[0087] The three-dimensional virtual scene for medical teaching comes from the three-dimensional modeling of the real medical tissue anatomical structure. The real medical tissue anatomical structure includes the human respiratory system anatomical structure, the human digestive system anatomical structure, the human blood circulation system anatomical structure, the human urinary system anatomical structure, and the human nervous system anatomical structure. The three-dimensional virtual scene for medical teaching includes the three-dimensional virtual scene of the respiratory system, the three-dimensional virtual scene of the digestive system, the three-dimensional virtual scene of the blood circulation system, the three-dimensional virtual scene of the urinary system, and the three-dimensional virtual scene of the nervous system.

[0088] The practical evaluation includes:

[0089] The actual depth, actual amplitude, actual speed and actual strength of the pulling and cutting operation of the tissue part by the operator are obtained by using the magnetic signal of the magnetic marker arranged at the end of the operating instrument held by the operator trainee;

[0090] Using the magnetic signals of the magnetic markers placed on the hands of the trainees, the actual hand motion trajectory, actual hand shaking amplitude and actual instrument shaking amplitude of the trainees' hands are obtained;

[0091] The trainee's operational stability and standardization are evaluated based on the actual depth, actual amplitude, actual speed, actual force, actual hand movement trajectory, actual hand shaking amplitude, and actual instrument shaking amplitude.

[0092] The moving distance of the magnetic marker reflects the true depth and amplitude of tissue pulling and cutting. By calculating the moving distance per unit time, the speed can be obtained, and the acceleration can be further calculated to deduce the applied force. Therefore, the electromagnetic detection of the present invention combines the EPSO-LM algorithm to track the position of the magnetic marker in real time to measure the movement information of the magnetic marker, as follows:

[0093] The method of obtaining the actual depth, actual amplitude, actual speed and actual force of the pulling and cutting operation of the tissue part by the operator using the magnetic signal of the magnetic marker arranged at the end of the operating instrument held by the operator includes:

[0094] The host computer program is used to track the magnetic signal of the magnetic marker arranged at the end of the operating instrument held by the trainee, wherein the host computer program automatically starts a timer inside and records the corresponding timer time after receiving a signal from the sensor array each time;

[0095] The signal of the sensor array is parsed into the coordinate position information of the magnetic marker arranged at the end of the operating instrument held by the trainee through the EPSO-LM algorithm, and the motion information of the magnetic marker arranged at the end of the operating instrument held by the trainee is formed based on the one-to-one correspondence between the position of the magnetic target and the time of the timer, wherein the nth path point in the motion information is [t n ,p n ] T ,n=1,2,3…,N, where t n and p n are the reading and position coordinate vector of the timer when the magnetic marker arranged at the end of the operating instrument held by the trainee moves to the nth path point, p n =[p nx ,p ny ,p nz ] T , where p nx ,p ny ,p nz They are p in the three-dimensional rectangular coordinate system n The three components of the coordinates in the X, Y, and Z axis directions, T is the transposition operator;

[0096] The instantaneous speed of the magnetic marker placed at the end of the operating instrument held by the trainee is approximately described by the average speed of the marker passing through two adjacent path points. According to the known motion information consisting of discrete coordinates, the instantaneous speed information of the magnetic marker placed at the end of the operating instrument held by the trainee is also composed of discrete data, and t is recorded. v The instantaneous velocity V of the magnetic marker placed at the end of the operating instrument held by the trainee at time s (t v ) is calculated as:

[0097]

[0098] In the formula, t n and p n are the reading and position coordinate vector of the timer when the magnetic marker arranged at the end of the operating instrument held by the trainee moves to the nth path point, t n+1 and p n+1 They are the reading and position coordinate vector of the timer when the magnetic marker arranged at the end of the operating instrument held by the trainee moves to the n+1th path point, and N represents the number of position data points collected during the entire operation process;

[0099] Based on the instantaneous velocity of the magnetic marker placed at the end of the operating instrument held by the trainee, the average acceleration method is used to approximate the instantaneous acceleration of the magnetic marker during its movement, and t a The instantaneous acceleration A of the magnetic marker placed at the end of the operating instrument held by the trainee at time s (t a ) is calculated as:

[0100]

[0101] In the formula, for The instantaneous speed of the magnetic marker placed at the end of the operating instrument held by the trainee at the time. for The instantaneous speed of the magnetic marker placed at the end of the operating instrument held by the trainee at time t n and p n are the reading and position coordinate vector of the timer when the magnetic marker arranged at the end of the operating instrument held by the trainee moves to the nth path point, t n+1 and p n+1 are the reading and position coordinate vector of the timer when the magnetic marker arranged at the end of the operating instrument held by the trainee moves to the n+1th path point, t n+2 and p n+2They are respectively the reading and position coordinate vector of the timer when the magnetic marker arranged at the end of the operating instrument held by the trainee moves to the n+2th path point;

[0102] t a The instantaneous acceleration of the magnetic marker placed at the end of the operating instrument held by the trainee is converted into a three-dimensional coordinate system, and A is obtained. s (t a )=[a sx ,a sy ,a sz ] T , where a sx ,a sy ,a sz A in the three-dimensional rectangular coordinate system s (t a ) components in the X, Y, and Z axis directions;

[0103] like Figure 3 As shown, the force analysis of the tissue part is carried out, and the force diagram of the tissue part is obtained, including: F = [f x ,f y ,f z ] T is the force exerted on the tissue, f x ,f y ,f z It is the three components of F in the X, Y, and Z axes in a three-dimensional rectangular coordinate system. G is the gravity exerted on the local tissue under traction. F1 and F2 are the components of the traction force in the vertical and horizontal directions, respectively. According to the laws of mechanics, the relationship between F and G is:

[0104]

[0105] Where m is the mass of the local tissue being stretched, g is the gravitational acceleration of the earth, and its direction is along the negative direction of the Z axis;

[0106] Using A s (t a )=[a sx ,a sy ,a sz ] T Find the tissue location at t a The pulling force F = [f x ,f y ,f z ] T ,get:

[0107]

[0108] Based on the movement information of the magnetic marker arranged at the end of the operating instrument held by the trainee, the moving distance of the magnetic marker representing the actual depth and actual amplitude of tissue pulling and cutting is obtained;

[0109] The instantaneous velocity and instantaneous acceleration of the magnetic markers placed on the end of the operating instrument held by the trainee are used to characterize the actual speed of tissue pulling and cutting;

[0110] The pulling force of the magnetic marker placed on the end of the operating instrument held by the trainee is used to represent the actual force of tissue pulling and cutting.

[0111] The instantaneous velocity, instantaneous acceleration and pulling force of the above magnetic markers are all discrete data.

[0112] The magnetic markers placed on the hands of the trainees can be detected and identified by electromagnetics. The position of the trainees' hands can be calculated through the sensor array, and the EPSO-LM algorithm can be used to track the magnetic markers on the hands in real time to obtain the actual motion trajectory of the hand operation, as follows:

[0113] The method of obtaining the actual hand motion trajectory, actual hand shaking amplitude and actual instrument shaking amplitude of the operator's hand operation by using the magnetic signal of the magnetic marker arranged on the operator's hand includes:

[0114] The theoretical value of the three components of the magnetic field generated by the magnetic marker placed on the operator's hand at the mth sensor in the sensor array is described by the magnetic dipole formula:

[0115]

[0116] In the formula, B T =μ r μ0M T / 4π,μ r is the relative magnetic permeability of the medium where the magnetic sensor is located, μ0 is the vacuum magnetic permeability, M T is the magnitude of the magnetic moment vector, [B mx ,B my ,B mz ] T are the three components of the magnetic induction intensity at the mth magnetic sensor, m = 1, 2, 3, ..., M, M is the total number of magnetic sensors on the magnetic sensor array, P m0 Indicates that the position of the magnetic mark P0 = [a, b, c] T Point to the mth magnetic sensor coordinate P m =[x m ,y m ,z m ] T The vector, P m is the coordinate of the mth magnetic sensor, Pm Using the three-dimensional rectangular coordinate system [x m ,y m ,z m ] T The three parameters represent that H0 is the magnetic moment vector of the magnetic marker;

[0117] The sum of the squares of the three-component errors between the magnetic sensor measurement value and the theoretical value is used as the objective function of the magnetic sensor array system error. The objective function of the magnetic sensor array system error is:

[0118]

[0119] Where, E is the magnetic sensor array system error, [B ′ mx ,B ′ my ,B ′ mz ] T is the three-component measurement value of the magnetic induction intensity of the mth magnetic sensor in the X, Y, and Z axis directions in the three-dimensional rectangular coordinate system, [B mx ,B my ,B mz ] T is the three-component theoretical value of the magnetic induction intensity of the mth magnetic sensor in the X, Y, and Z axis directions in the three-dimensional rectangular coordinate system, and M is the total number of magnetic sensors on the magnetic sensor array;

[0120] The EPSO-LM algorithm can be used to solve the nonlinear equation E=0, thereby solving the coordinate position of the magnetic marker placed on the operator's hand. Based on the fact that the magnetic marker placed on the operator's hand is rigidly linked to the operator's hand, the position information of the magnetic marker is continuously collected as the position information of the hand, and the position information of the magnetic marker is recorded corresponding to the time, and the actual hand motion trajectory of the hand position information changing with time is drawn, wherein the actual motion trajectory The hand position at the moment is In the formula, In the three-dimensional rectangular coordinate system The coordinates of the hand position information in the X, Y, and Z axes at the moment, where n is a positive integer;

[0121] Keep your hands The amplitude of the shaking at the time is used during operation Hand position at the moment Distance from the average hand position during the entire process The actual hand shaking amplitude is measured by the straight-line distance, and the calculation formula is:

[0122]

[0123] In the formula, for The hand shaking amplitude at the moment, N represents the number of position data points collected during the entire operation;

[0124] During the use of the electromagnetic detection rod, based on the fact that the tip of the electromagnetic detection rod is rigidly linked to the electromagnetic coil, the position information of the tip of the electromagnetic detection rod is calculated by detecting the position of the electromagnetic coil, wherein the position information of the tip of the electromagnetic detection rod is:

[0125] P cusp =P pole +LH pole ;

[0126] Where P cusp is the position vector of the tip of the electromagnetic probe rod, P pole and H pole are the position vector and the axial direction vector of the cylindrical electromagnetic coil built into the electromagnetic detection rod, respectively, and L is the distance from the center of the electromagnetic coil to the tip of the detection rod;

[0127] The tip position coordinates at each moment are recorded as the motion information of the tip of the electromagnetic detection rod, and the instantaneous velocity and instantaneous acceleration calculation formula of the magnetic marker arranged at the end of the operating instrument held by the trainee are used to obtain the instantaneous velocity and instantaneous acceleration of the tip of the electromagnetic detection rod. Then, the calculation formula of the hand shaking amplitude is used to calculate the shaking amplitude of the electromagnetic detection rod during use as the actual instrument shaking amplitude.

[0128] The operation process of several experienced doctors is recorded in the electromagnetic detection platform, including operation steps, time, speed, strength, etc., as a standard reference. During the training process, the operator's hand shaking amplitude, instrument shaking amplitude, operation process and pulling strength data can be compared with the standard reference. Experienced doctors evaluate the stability and standardization of the trainees' training operations, specifically:

[0129] If the actual depth, amplitude, speed, and force of the pulling and cutting operations of the tissue parts by the trainees, as well as the actual hand movement trajectory, hand shaking amplitude, and instrument shaking amplitude of the trainees are consistent with the standard values, the trainees’ operation stability and standardization level are high, and the practical evaluation is high;

[0130] If the actual depth, amplitude, speed, and strength of the pulling and cutting operations of the tissue parts by the trainees, as well as the actual hand movement trajectory, actual hand shaking amplitude, and actual instrument shaking amplitude of the trainees are inconsistent with the standard values, the trainees' operation stability and standardization level are low, and the practical evaluation is low. The method of providing operational feedback and correction guidance to the trainees' medical operations in the three-dimensional virtual scene of medical teaching based on the evaluation results of the practical evaluation includes:

[0131] The coordinate transformation matrix T between the magnetic coordinate system of the sensor array and the virtual coordinate system of the three-dimensional virtual scene of medical teaching is used. A B , the actual depth, actual amplitude, actual speed, actual strength of the pulling and cutting operations of the tissue parts by the trainees, as well as the actual hand movement trajectory, actual hand shaking amplitude, and actual instrument shaking amplitude of the trainees are converted and displayed from the magnetic coordinate system of the sensor array to the virtual coordinate system of the three-dimensional virtual scene of medical teaching;

[0132] The actual depth, actual amplitude, actual speed, actual strength of the pulling and cutting operations of the tissue parts displayed by the trainees in the three-dimensional virtual scene of medical teaching, as well as the actual hand movement trajectory, actual hand shaking amplitude, and actual instrument shaking amplitude of the trainees are compared with the standard values.

[0133] When the actual depth, amplitude, speed, and force of the pulling and cutting operations of the tissue parts, as well as the actual hand movement trajectory, hand shaking amplitude, and instrument shaking amplitude of the trainees are consistent with the standard values, positive feedback will be given to the trainees;

[0134] When the actual depth, actual amplitude, actual speed, actual force of the pulling and cutting operations on the tissue parts, as well as the actual hand movement trajectory, actual hand shaking amplitude, and actual instrument shaking amplitude of the trainee are inconsistent with the standard values, the trainee will be given negative feedback and corrective guidance.

[0135] Methods for correcting guidelines include:

[0136] The depth, amplitude, speed, strength of the pulling and cutting operations on the tissue parts by the trainees at the moment of negative feedback, as well as the standard values ​​of the trainees' hand movement trajectory, hand shaking amplitude, and instrument shaking amplitude are displayed in the three-dimensional virtual scene of medical teaching, guiding the trainees to make corrections to the actual depth, actual amplitude, actual speed, actual strength, actual hand movement trajectory, actual hand shaking amplitude, and actual instrument shaking amplitude according to the standard values.

[0137] Coordinate transformation matrix between the magnetic coordinate system of the sensor array and the virtual coordinate system of the 3D virtual scene for medical teaching The establishment methods include:

[0138] The magnetic coordinate system of the sensor array where the magnetic marker is located is recorded as coordinate system A, and the virtual coordinate system of the three-dimensional virtual scene of medical teaching is recorded as coordinate system B, where the coordinate transformation matrix from coordinate system A to B is The calculation formula is:

[0139]

[0140] In the formula, is a 3×3 coordinate rotation matrix, is a 3×1 coordinate translation matrix;

[0141] Select four non-coplanar feature points in the space, and record the coordinates of the kth feature point in the coordinate system A as The coordinates in coordinate system B are

[0142] Through the coordinate transformation matrix The coordinates P of any point in the coordinate system A A Transform to coordinate P in coordinate system B B ,get:

[0143]

[0144] in,

[0145] Using the known coordinates of the four characteristic points, a system of 12 equations is listed, and the coordinate transformation matrix is ​​solved simultaneously. The 9 unknowns (r 11 ,r 12 , r 13 ,r 21 , r 22 ,r 23 , r 31 ,r 32 , r 33 ) and the three unknowns of the coordinate translation matrix (l1, l2, l3), a total of 12 unknowns, we get:

[0146]

[0147] Use the least squares method or other optimization algorithm to find the best fit and Then solve the coordinate transformation matrix

[0148] The EPSO-LM algorithm in the present invention is an EPSO algorithm constructed by adding a last-place elimination mechanism to a particle swarm search algorithm (PSO algorithm), which can search and solve the lesion posture information to obtain an accurate lesion posture, that is, the accurate positioning of the lesion is achieved through the search algorithm, and due to the addition of the last-place elimination mechanism, the specific results are as follows:

[0149] The initial position positioning method of the magnetic marker includes:

[0150] The magnetic markers are abstracted as particles in the EPSO algorithm, and the particle population size in the EPSO algorithm is set to N, and the number of population iterations is set to TN;

[0151] The position and posture of the magnetic marker are described by the rectangular coordinate system of the magnetic sensor array, and the position and posture vector of the particle in the EPSO algorithm is determined. The fitness of each particle can be obtained by substituting the six-dimensional coordinate parameters of each particle into the fitness calculation formula, that is, E=f(X). The smaller the fitness E, the better the position and posture X of the particle.

[0152] The particle's pose vector is: In the formula, is the position vector of the ith particle in the particle population after the tth iteration, [a i ,b i ,c i ] T is the position vector of the ith particle in the particle population in the rectangular coordinate system after the tth iteration, is the magnetic moment vector of the ith particle in the particle population in the spherical coordinate system after the tth iteration, M Ti is the modulus of the magnetic moment vector of the ith particle in the particle population after the tth iteration.

[0153] The objective function of the magnetic sensor array system error is used as the fitness function of the particles in the EPSO algorithm;

[0154] The objective function is determined as the fitness function of the particles in the EPSO algorithm. The fitness function of the particles in the EPSO algorithm is:

[0155] f(X) = E;

[0156] Where f(X) is the fitness function value of the particle, and E is the system error of the magnetic sensor array.

[0157] Based on the fitness function, particle population size and population iteration times, the EPSO algorithm uses the particle swarm search mechanism to perform a global search for the optimal position of the particle population, and uses the last-place elimination mechanism in the global search to avoid particles from falling into the local optimal solution and speed up the search, so as to obtain the optimal position of the particle population as the initial position of the magnetic target.

[0158] The above-mentioned EPSO algorithm can be used to obtain the initial position of the magnetic target. Since the marker will change its position along with the lesion in the actual environment, the target needs to be tracked, and tracking is to locate it quickly and continuously.

[0159] The LM algorithm can solve X to minimize the nonlinear equation E = f (X), thereby obtaining the magnetic marker pose. Its advantage is that it has a fast solution speed and is very suitable for target tracking needs in actual environments. However, the algorithm has high requirements for initial values ​​and is not suitable for large-scale search targets. Here, the LM algorithm is combined with the EPSO algorithm to make up for the shortcomings of the LM algorithm. At the beginning of tracking, the EPSO algorithm can provide the initial pose of the magnetic target, and then the initial pose is used as the initial value of the LM algorithm for calculation to achieve real-time tracking and positioning of the magnetic marker.

[0160] The method for acquiring the real-time position and posture of the magnetic marker includes:

[0161] The initial position of the magnetic marker obtained by the EPSO algorithm is used as the initial value of the first LM algorithm, and the first LM algorithm is run;

[0162] Starting from the first LM algorithm, multiple LM algorithms are continuously executed to continuously locate the magnetic marker and obtain the real-time position and posture of the magnetic marker. Among the adjacent LM algorithms, the last LM algorithm run obtains the position and posture vector of the magnetic marker as the initial value of the next LM algorithm run.

[0163] like Figure 2 As shown, the present invention provides a medical device operation simulation teaching platform based on electromagnetic detection, which is applied to a medical device operation simulation teaching method based on electromagnetic detection. The platform includes:

[0164] A sensor array, used to sense magnetic signals of magnetic markers arranged on the hands of the trainee operator and on the ends of the operating instruments held by the trainee operator;

[0165] Magnetic markers are used to be placed on the hands of trainees and on the ends of the operating instruments held by trainees to generate magnetic signals for induction by the sensor array;

[0166] A data processing unit, used for evaluating the medical operations performed by trainees in a three-dimensional virtual scene for medical teaching based on magnetic signals;

[0167] An interface interaction unit, used to provide operational feedback and correction guidance to trainees on medical operations in a three-dimensional virtual scene for medical teaching according to the evaluation results of the practical evaluation;

[0168] The data storage unit is used to store three-dimensional virtual scenes for medical teaching.

[0169] Magnetic markers include magnetic fluids, magnetic hydrogels, magnetic soft patches, sensors, and coils;

[0170] The sensor array is an array composed of magnetic sensors of different models.

[0171] The present invention also provides a method for quickly and comprehensively perceiving the weak operations of trainees, which is as follows:

[0172] Record the local scenes of each operator at the negative feedback moment in a single medical teaching three-dimensional virtual scene, and perform cluster analysis on all the local scenes to obtain a set of multiple common teaching difficulties and key points of a single medical teaching three-dimensional virtual scene;

[0173] The local scene of a single operator at the negative feedback moment in a single medical teaching three-dimensional virtual scene is used as the teaching individual key and difficult points of the single medical teaching three-dimensional virtual scene;

[0174] The distance matching is performed using the cluster centers of the teaching individual key points and difficulties and the teaching common key points and difficulties set of a single medical teaching three-dimensional virtual scene, and the teaching common key points and difficulties set corresponding to the cluster center with the smallest distance to the teaching individual key points and difficulties is used as the teaching consolidation set for a single operation trainee in a single medical teaching three-dimensional virtual scene;

[0175] The teaching consolidation set of a single operator in a single medical teaching three-dimensional virtual scene is matched with multiple sets of common teaching difficulties and key points in other medical teaching three-dimensional virtual scenes based on the distance of the cluster center, and the set of common teaching difficulties and key points corresponding to the cluster center with the smallest distance to the teaching consolidation set is used as the teaching consolidation set of the single operator in other medical teaching three-dimensional virtual scenes.

[0176] The present invention can record the individual weaknesses / scenes of the operating trainee (the individual key and difficult points of teaching), and then can quickly associate all the operating scenes related to the individual weaknesses / scenes in the same three-dimensional virtual scene and all the operating scenes related to the individual weaknesses / scenes between different three-dimensional virtual scenes based on commonality analysis, so as to quickly identify all the operating weak scenes of the operating trainee, and make personalized teaching, and make up for the deficiencies more comprehensive, so as to consolidate the teaching of the operating trainee.

[0177] The present invention also provides a simulation teaching example: in the virtual simulation training of laparoscopic gastrointestinal cancer surgery, the sensor is placed on the top of the laparoscopic instrument (separation forceps, scissors), and the sensor array is used to sense the signal, which can measure the operator's real depth, amplitude, speed and strength of pulling and cutting operations on the tissue part;

[0178] The magnetic soft patch is attached to the operator's hand, and the sensor array is used to sense the signal, which can measure the movement trajectory of the operator's hand, the shaking amplitude of the hand, and the shaking amplitude of the instrument during operation;

[0179] Based on the above-mentioned measurement data, the trainee's operational stability, operational standardization, etc. are evaluated.

[0180] Furthermore, through application analysis, combined with the magnetic coordinate system and the virtual reality coordinate system, the real-time operation information is fed back to the virtual simulation platform, and real-time interaction with students is carried out to provide timely feedback and correction of operations, thereby improving the learning effect and efficiency of students.

[0181] In summary, the present invention applies electromagnetic detection to virtual simulation teaching, marks the trainees' hands, instrument ends, etc. with magnetic markers to form multi-point magnetic markings, detects real-time magnetic signals through a sensor array, and uses a program to draw operation trajectories, record depth, frequency and other data, and combines the magnetic coordinate system with the mixed reality coordinate system to efficiently and intuitively evaluate the operation quality.

[0182] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.

Claims

1. A medical device operation simulation teaching method based on electromagnetic detection, characterized in that: The following steps are involved: Selecting a three-dimensional virtual scene for medical teaching, and arranging the three-dimensional virtual scene for medical teaching in a sensing space of a sensor array; The sensor array senses magnetic signals of magnetic markers disposed on the hands of the trainee and on the ends of the operating instruments held by the trainee respectively; Based on the magnetic signals, a practical evaluation is performed on the medical operation of the trainee in the three-dimensional virtual scene of medical teaching; Based on the evaluation results of the practical evaluation, operational feedback and correction guidance are provided to trainees on their medical operations in the three-dimensional virtual scene of medical teaching.

2. According to claim 1, a medical device operation simulation teaching method based on electromagnetic detection is characterized in that: The three-dimensional virtual scene for medical teaching comes from the three-dimensional modeling of real medical tissue anatomical structures, and the real medical tissue anatomical structures include the anatomical structure of the human respiratory system, the anatomical structure of the human digestive system, the anatomical structure of the human blood circulation system, the anatomical structure of the human urinary system, and the anatomical structure of the human nervous system. The three-dimensional virtual scene for medical teaching includes the three-dimensional virtual scene of the respiratory system, the three-dimensional virtual scene of the digestive system, the three-dimensional virtual scene of the blood circulation system, the three-dimensional virtual scene of the urinary system, and the three-dimensional virtual scene of the nervous system.

3. A medical device operation simulation teaching method based on electromagnetic detection according to claim 2, characterized in that: The practical evaluation includes: The actual depth, actual amplitude, actual speed and actual strength of the pulling and cutting operation of the tissue part by the operator are obtained by using the magnetic signal of the magnetic marker arranged at the end of the operating instrument held by the operator trainee; Using the magnetic signals of the magnetic markers placed on the hands of the trainees, the actual hand motion trajectory, actual hand shaking amplitude and actual instrument shaking amplitude of the trainees' hands are obtained; The trainee's operational stability and standardization are evaluated based on the actual depth, actual amplitude, actual speed, actual force, actual hand movement trajectory, actual hand shaking amplitude, and actual instrument shaking amplitude.

4. The medical device operation simulation teaching method based on electromagnetic detection according to claim 3 is characterized by: The method of obtaining the actual depth, actual amplitude, actual speed and actual force of the pulling and cutting operation of the tissue part by the operator using the magnetic signal of the magnetic marker arranged at the end of the operating instrument held by the operator includes: The host computer program is used to track the magnetic signal of the magnetic marker arranged at the end of the operating instrument held by the trainee, wherein the host computer program automatically starts a timer inside and records the corresponding timer time after receiving a signal from the sensor array each time; The signal of the sensor array is parsed into the coordinate position information of the magnetic marker arranged at the end of the operating instrument held by the trainee through the EPSO-LM algorithm, and the motion information of the magnetic marker arranged at the end of the operating instrument held by the trainee is formed based on the one-to-one correspondence between the position of the magnetic target and the time of the timer, wherein the nth path point in the motion information is [t n ,p n ] T ,n=1,2,3…,N, where t n and p n are the reading and position coordinate vector of the timer when the magnetic marker arranged at the end of the operating instrument held by the trainee moves to the nth path point, p n =[p nx ,p ny ,p nz ] T , where p nx ,p ny ,p nz They are p in the three-dimensional rectangular coordinate system n The three components of the coordinates in the X, Y, and Z axis directions, T is the transposition operator; The instantaneous speed of the magnetic marker placed at the end of the operating instrument held by the trainee is approximately described by the average speed of the marker passing through two adjacent path points. According to the known motion information consisting of discrete coordinates, the instantaneous speed information of the magnetic marker placed at the end of the operating instrument held by the trainee is also composed of discrete data, and t is recorded. v The instantaneous velocity V of the magnetic marker placed at the end of the operating instrument held by the trainee at time s (t v ) is calculated as: Where, t n and p n are the reading and position coordinate vector of the timer when the magnetic marker arranged at the end of the operating instrument held by the trainee moves to the nth path point, t n+1 and p n+1 They are the reading and position coordinate vector of the timer when the magnetic marker arranged at the end of the operating instrument held by the trainee moves to the n+1th path point, and N represents the number of position data points collected during the entire operation process; Based on the instantaneous velocity of the magnetic marker placed at the end of the operating instrument held by the trainee, the average acceleration method is used to approximate the instantaneous acceleration of the magnetic marker during its movement, and t a The instantaneous acceleration A of the magnetic marker placed at the end of the operating instrument held by the trainee at time s (t a ) is calculated as: In the formula, for The instantaneous speed of the magnetic marker placed at the end of the operating instrument held by the trainee at the time. for The instantaneous speed of the magnetic marker placed at the end of the operating instrument held by the trainee at time t n and p n are the reading and position coordinate vector of the timer when the magnetic marker arranged at the end of the operating instrument held by the trainee moves to the nth path point, t n+1 and p n+1 are the reading and position coordinate vector of the timer when the magnetic marker arranged at the end of the operating instrument held by the trainee moves to the n+1th path point, t n+2 and p n+2 They are respectively the reading and position coordinate vector of the timer when the magnetic marker arranged at the end of the operating instrument held by the trainee moves to the n+2th path point; t a The instantaneous acceleration of the magnetic marker placed at the end of the operating instrument held by the trainee is converted into a three-dimensional coordinate system, and A is obtained. s (t a )=[a sx ,a sy ,a sz ] T , where a sx ,a sy ,a sz A in the three-dimensional rectangular coordinate system s (t a ) components in the X, Y, and Z axis directions; The force analysis of the tissue parts is carried out, and the force diagram of the tissue parts is obtained, including: F = [f x ,f y ,f z ] T is the force exerted on the tissue, f x ,f y ,f z It is the three components of F in the X, Y, and Z axes in a three-dimensional rectangular coordinate system. G is the gravity exerted on the local tissue under traction. F1 and F2 are the components of the traction force in the vertical and horizontal directions, respectively. According to the laws of mechanics, the relationship between F and G is: Where m is the mass of the local tissue being stretched, g is the gravitational acceleration of the earth, and its direction is along the negative direction of the Z axis; Using A s (t a )=[a sx ,a sy ,a sz ] T Find the tissue location at t a The pulling force F at the moment = [f x ,f y ,f z ] T ,get: Based on the movement information of the magnetic marker arranged at the end of the operating instrument held by the trainee, the moving distance of the magnetic marker representing the actual depth and actual amplitude of tissue pulling and cutting is obtained; The instantaneous velocity and instantaneous acceleration of the magnetic markers placed on the end of the operating instrument held by the trainee are used to characterize the actual speed of tissue pulling and cutting; The pulling force of the magnetic marker placed on the end of the operating instrument held by the trainee is used to represent the actual force of tissue pulling and cutting.

5. The medical device operation simulation teaching method based on electromagnetic detection according to claim 4 is characterized in that: The method of obtaining the actual hand motion trajectory, actual hand shaking amplitude and actual instrument shaking amplitude of the operator's hand operation by using the magnetic signal of the magnetic marker arranged on the operator's hand includes: The theoretical value of the three components of the magnetic field generated by the magnetic marker placed on the operator's hand at the mth sensor in the sensor array is described by the magnetic dipole formula: In the formula, B T =μ r μ0M T / 4π,μ r is the relative magnetic permeability of the medium where the magnetic sensor is located, μ0 is the vacuum magnetic permeability, M T is the magnitude of the magnetic moment vector, [B mx ,B my ,B mz ] T are the three components of the magnetic induction intensity at the mth magnetic sensor, m = 1, 2, 3, ..., M, M is the total number of magnetic sensors on the magnetic sensor array, P m0 Indicates that the position of the magnetic mark P0 = [a, b, c] T Point to the mth magnetic sensor coordinate P m =[x m ,y m ,z m ] T The vector, P m is the coordinate of the mth magnetic sensor, P m Using the three-dimensional rectangular coordinate system [x m ,y m ,z m ] T The three parameters represent that H0 is the magnetic moment vector of the magnetic marker; The sum of the squares of the three-component errors between the magnetic sensor measurement value and the theoretical value is used as the objective function of the magnetic sensor array system error. The objective function of the magnetic sensor array system error is: Where, E is the magnetic sensor array system error, [B ′ mx ,B ′ my ,B ′ mz ] T is the three-component measurement value of the magnetic induction intensity of the mth magnetic sensor in the X, Y, and Z axis directions in the three-dimensional rectangular coordinate system, [B mx ,B my ,B mz ] T is the three-component theoretical value of the magnetic induction intensity of the mth magnetic sensor in the X, Y, and Z axis directions in the three-dimensional rectangular coordinate system, and M is the total number of magnetic sensors on the magnetic sensor array; The EPSO-LM algorithm can be used to solve the nonlinear equation E=0, thereby solving the coordinate position of the magnetic marker placed on the operator's hand. Based on the fact that the magnetic marker placed on the operator's hand is rigidly linked to the operator's hand, the position information of the magnetic marker is continuously collected as the position information of the hand, and the position information of the magnetic marker is recorded corresponding to the time, and the actual hand motion trajectory of the hand position information changing with time is drawn, wherein the actual motion trajectory The hand position at the moment is In the formula, In the three-dimensional rectangular coordinate system The coordinates of the hand position information in the X, Y, and Z axes at the moment, where n is a positive integer; Keep your hands The amplitude of the shaking at the time is used during operation Hand position at the moment Distance from the average hand position during the entire process The actual hand shaking amplitude is measured by the straight-line distance, and the calculation formula is: In the formula, for The hand shaking amplitude at the moment, N represents the number of position data points collected during the entire operation; During the use of the electromagnetic detection rod, based on the fact that the tip of the electromagnetic detection rod is rigidly linked to the electromagnetic coil, the position information of the tip of the electromagnetic detection rod is calculated by detecting the position of the electromagnetic coil, wherein the position information of the tip of the electromagnetic detection rod is: P cusp =P pole +LH pole ; Where P cusp is the position vector of the tip of the electromagnetic probe rod, P pole and H pole are the position vector and the axial direction vector of the cylindrical electromagnetic coil built into the electromagnetic detection rod, respectively, and L is the distance from the center of the electromagnetic coil to the tip of the detection rod; The tip position coordinates at each moment are recorded as the motion information of the tip of the electromagnetic detection rod, and the instantaneous velocity and instantaneous acceleration calculation formula of the magnetic marker arranged at the end of the operating instrument held by the trainee are used to obtain the instantaneous velocity and instantaneous acceleration of the tip of the electromagnetic detection rod. Then, the calculation formula of the hand shaking amplitude is used to calculate the shaking amplitude of the electromagnetic detection rod during use as the actual instrument shaking amplitude.

6. The medical device operation simulation teaching method based on electromagnetic detection according to claim 5 is characterized in that: The method of providing operation feedback and correction guidance to the medical operation of the trainees in the three-dimensional virtual scene of medical teaching according to the evaluation results of the practical evaluation includes: The coordinate transformation matrix T between the magnetic coordinate system of the sensor array and the virtual coordinate system of the three-dimensional virtual scene of medical teaching is used. A B , the actual depth, actual amplitude, actual speed, actual strength of the pulling and cutting operations of the tissue parts by the trainees, as well as the actual hand movement trajectory, actual hand shaking amplitude, and actual instrument shaking amplitude of the trainees are converted and displayed from the magnetic coordinate system of the sensor array to the virtual coordinate system of the three-dimensional virtual scene of medical teaching; The actual depth, actual amplitude, actual speed, actual strength of the pulling and cutting operations of the tissue parts displayed by the trainees in the three-dimensional virtual scene of medical teaching, as well as the actual hand movement trajectory, actual hand shaking amplitude, and actual instrument shaking amplitude of the trainees are compared with the standard values. When the actual depth, amplitude, speed, and force of the pulling and cutting operations of the tissue parts, as well as the actual hand movement trajectory, hand shaking amplitude, and instrument shaking amplitude of the trainees are consistent with the standard values, positive feedback will be given to the trainees; When the actual depth, actual amplitude, actual speed, actual force of the pulling and cutting operations on the tissue parts, as well as the actual hand movement trajectory, actual hand shaking amplitude, and actual instrument shaking amplitude of the trainee are inconsistent with the standard values, the trainee will be given negative feedback and corrective guidance.

7. A medical device operation simulation teaching method based on electromagnetic detection according to claim 6, characterized in that: Methods for correcting guidelines include: The depth, amplitude, speed, strength of the pulling and cutting operations on the tissue parts by the trainees at the moment of negative feedback, as well as the standard values ​​of the trainees' hand movement trajectory, hand shaking amplitude, and instrument shaking amplitude are displayed in the three-dimensional virtual scene of medical teaching, guiding the trainees to make corrections to the actual depth, actual amplitude, actual speed, actual strength, actual hand movement trajectory, actual hand shaking amplitude, and actual instrument shaking amplitude according to the standard values.

8. The medical device operation simulation teaching method based on electromagnetic detection according to claim 6 is characterized by: Coordinate transformation matrix between the magnetic coordinate system of the sensor array and the virtual coordinate system of the 3D virtual scene for medical teaching The establishment methods include: The magnetic coordinate system of the sensor array where the magnetic marker is located is recorded as coordinate system A, and the virtual coordinate system of the three-dimensional virtual scene of medical teaching is recorded as coordinate system B, where the coordinate transformation matrix from coordinate system A to B is The calculation formula is: In the formula, is a 3×3 coordinate rotation matrix, is a 3×1 coordinate translation matrix; Select four non-coplanar feature points in the space, and record the coordinates of the kth feature point in the coordinate system A as The coordinates in coordinate system B are Through the coordinate transformation matrix The coordinates P of any point in the coordinate system A A Transform to coordinate P in coordinate system B B ,get: Using the known coordinates of the four characteristic points, a system of 12 equations is listed, and the coordinate transformation matrix is ​​solved simultaneously. The 9 unknowns of the coordinate translation matrix and the 3 unknowns of the coordinate translation matrix, a total of 12 unknowns, give: Use the least squares method to find the best fit and Then solve the coordinate transformation matrix T A B .

9. A medical device operation simulation teaching platform based on electromagnetic detection, characterized in that: A medical simulation teaching method based on electromagnetic detection applied to any one of claims 1 to 8, wherein the medical simulation teaching platform comprises: A sensor array, used to sense magnetic signals of magnetic markers arranged on the hands of the trainee operator and on the ends of the operating instruments held by the trainee operator; Magnetic markers are used to be placed on the hands of trainees and on the ends of the operating instruments held by trainees to generate magnetic signals for induction by the sensor array; A data processing unit, used for performing a practical evaluation of the medical operation performed by the trainee in the three-dimensional virtual scene of medical teaching based on the magnetic signal; An interface interaction unit, used to provide operational feedback and correction guidance to trainees on medical operations in a three-dimensional virtual scene for medical teaching according to the evaluation results of the practical evaluation; The data storage unit is used to store three-dimensional virtual scenes for medical teaching.

10. The medical device operation simulation teaching platform based on electromagnetic detection according to claim 9, characterized in that: The magnetic markers include magnetic fluid, magnetic hydrogel, magnetic soft patch, sensor and coil; The sensor array is an array composed of magnetic sensors of different models.

Images