An AR technology-based lumbar puncture simulation teaching system

Abstract

The present application relates to the technical field of medical training, and discloses a lumbar puncture simulation teaching system based on AR technology, which comprises an AR content generation module, which collects lumbar vertebra medical images and performs tissue segmentation and lesion identification; performs three-dimensional geometric modeling based on the tissue segmentation result and the lesion identification result, and generates a lumbar vertebra 3D model; an AR display interaction module, which develops the interactive function of the lumbar vertebra 3D model by using a Unity platform and an AR device; monitors the interactive operation of a user and provides real-time feedback; and an operation evaluation module, which evaluates the interactive operation of the user and specifies improvement suggestions according to the evaluation result. The three-dimensional visual scene constructed by the AR technology can show the complex anatomical structure of the lumbar vertebra part, the user can observe the needle insertion process from different angles, and dynamic effects such as the change of cerebrospinal fluid pressure can be simulated; in the lumbar puncture simulation training, the AR system can provide real-time feedback on operation data such as the needle insertion angle, the puncture strength, the puncture depth and the like, and help the user accurately master the operation skills.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical training, and particularly relates to a lumbar puncture simulation teaching system based on AR technology. BACKGROUND

[0002] Traditional lumbar puncture teaching mainly relies on human body models and a small number of clinical operation opportunities, but professional lumbar puncture simulation models are high in cost and have problems such as model loss and difficulty in updating; and the use effect is general, and it is difficult to fully display the lumbar puncture needle insertion path (such as needle insertion angle and depth). The Chinese invention patent with the publication number CN115035767B discloses a spine surgery teaching and training system based on AR and personification model, which simulates the physical environment of surgery teaching and training through an operation training platform, and automatically collects objective data of surgery operation in real time to evaluate the surgery operation skills, but the model still adopts 3D printing, and the monitoring of operation still depends on cameras and sensors and other devices installed on actual surgery instruments, and still has problems of high cost, high loss and difficulty in monitoring. SUMMARY

[0003] The present application aims to solve the problems in the prior art.

[0004] The technical scheme adopted by the present application to solve the technical problems is to provide a lumbar puncture simulation teaching system based on AR technology, comprising:

[0005] An AR content generation module collects lumbar spine medical images and performs tissue segmentation and lesion identification; based on the tissue segmentation result and the lesion identification result, a three-dimensional geometric model is established, and a lumbar spine 3D model is generated;

[0006] An AR display interaction module uses the Unity platform and AR equipment to develop the interactive function of the lumbar spine 3D model; the interactive operation of the user is monitored, and real-time feedback is performed;

[0007] An operation evaluation module evaluates the interactive operation of the user, and specifies improvement suggestions according to the evaluation result.

[0008] Preferably, the three-dimensional geometric modeling based on the tissue segmentation result and the lesion identification result to generate the lumbar spine 3D model comprises the following steps:

[0009] The lumbar spine medical images are segmented into vertebrae, dural sacs and nerve roots, and based on the segmentation result, different parts are modeled respectively to obtain a lumbar spine 3D preliminary model;

[0010] The lumbar spine medical images are identified for lesions, and the identified lesions are labeled in the lumbar spine 3D preliminary model to obtain the lumbar spine 3D model.

[0011] Preferably, the lumbar vertebrae medical image is segmented into vertebrae, dural sac and nerve root, different parts are modeled based on the segmentation results, and a preliminary model of the lumbar vertebrae 3D is obtained, including the following steps:

[0012] Vertebra modeling, constructing the geometric shape of the vertebra according to the lumbar vertebrae medical image and assigning corresponding physical attribute parameters, including the elastic modulus of the cortical bone of the vertebra 15±2GPa, the cancellous bone 0.5±0.1GPa, and the shear modulus of the yellow ligament ; Establish the mapping relationship between bone mineral density BMD and elastic modulus E and introduce the BMI adaptive adjustment mechanism; The color of the bone tissue is white, and different textures are set for the cortical bone and cancellous bone; BMI represents the body mass index;

[0013] Dural sac modeling, positioning the dural sac end at the lower edge of L2 vertebral body, marking the puncture point at the 0.5-1cm area beside the midline after L3-L4 intervertebral space or L4-L5 intervertebral space; The color of the dural sac is blue with a transparency of 50%;

[0014] Dangerous area modeling, marking the epidural venous plexus and the running path of L5 nerve root; The color of the nerve root is yellow with a transparency of 80%.

[0015] Preferably, the interactive function of the lumbar vertebrae 3D model developed by the Unity platform and AR equipment includes:

[0016] Display function, displaying the lumbar vertebrae 3D model on AR glasses;

[0017] Puncture function, the physical engine of Unity sets different puncture resistance feedback for different parts of the lumbar vertebrae 3D model; Calculate the theoretical puncture path; The user uses the 3DoF ray of AR glasses as a virtual puncture needle to perform puncture operation, and real-time monitors the puncture operation of the user to update the state of the lumbar vertebrae 3D model;

[0018] Prompt function, using the display function and vibration function of AR glasses to realize various prompts.

[0019] Preferably, the physical engine of Unity sets different puncture resistance feedback for different parts of the lumbar vertebrae 3D model, and the relationship between puncture resistance and depth is represented as:

[0020] ;

[0021] Puncture resistance The unit of puncture resistance is N, and the unit of depth d is cm.

[0022] Preferably, the prompt function includes:

[0023] When the puncture speed is greater than 10 mm / s, the AR glasses main machine sends a low-frequency vibration when puncturing into the skin-subcutaneous tissue;

[0024] When the puncture speed is greater than 10 mm / s, the AR glasses main machine sends a low-frequency vibration when puncturing into the skin-subcutaneous tissue;

[0025] When the puncture needle contacts the yellow ligament, the AR glasses main machine sends a strong vibration;

[0026] When the puncture needle contacts the yellow ligament, the AR glasses main machine sends a strong vibration;

[0027] When the puncture needle contacts the yellow ligament, the AR glasses main machine sends a strong vibration;

[0028] ;

[0029] .

[0030] Preferably, the theoretical puncture path includes:

[0031] The puncture point is located on the line connecting the iliac crest, and the posterior median line is The posterior median line is The intersection of the line connecting the iliac crest and the posterior median line is taken as the origin, and the coordinates of the puncture point are represented as:

[0032] ;

[0033] wherein, is the z-coordinate of the midpoint of the L3-L4 intervertebral space; BMI represents the body mass index;

[0034] The needle insertion angle dynamic model includes the standard angle BMD represents the bone mineral density, with the unit of mg / cm³; when there is spinal stenosis, the angle is adjusted downward by 5°-10°; when there is lumbar disc herniation, the system automatically adjusts the needle insertion angle downward by 3°-8°; when there is nerve root injury, the needle insertion angle is slightly adjusted to the contralateral side by 2°-5°; when there is cerebrospinal fluid leakage, the angle fluctuation threshold is set, and the user's puncture operation angle fluctuation exceeding the threshold will issue a warning.

[0035] Preferably, the prompt function includes:

[0036] The needle insertion angle monitoring displays the needle insertion angle in real time, and issues a prompt when the actual angle deviates from the calculation result of the needle insertion angle dynamic model;

[0037] The three-dimensional path deviation map displays the spatial distance between the actual puncture trajectory and the theoretical puncture path in real time, and issues a prompt when the deviation exceeds the threshold;

[0038] ​Depth monitoring, real-time monitoring of puncture depth, prompting when the puncture depth exceeds the standard depth range.

[0039] Preferably, the real-time monitoring of the user's puncture operation updates the state of the lumbar 3D model, including:

[0040] After the puncture needle breaks through the yellow ligament and enters the dural sac, the resistance drops to ≤3N, and the AR glasses host vibration is weakened; the particle system is used to present the cerebrospinal fluid flow effect, and the flow rate is positively correlated with the cerebrospinal fluid pressure value;

[0041] When the puncture needle touches the venous plexus, the AR glasses present the blood exudation particle effect with vibration feedback prompt, simulating the bleeding scene.

[0042] Preferably, the evaluation performed by the operation evaluation module includes:

[0043] Puncture point score, comparing the puncture point of the user's operation with the puncture point of the theoretical puncture path to evaluate the puncture point score;

[0044] Angle score, comparing the puncture angle of the user's operation with the puncture angle of the theoretical puncture path to evaluate the angle score;

[0045] Procedure score, relying on the AR glasses hardware, recording the 6DOF pose of the virtual puncture needle, calculating the acceleration change rate to evaluate the operation smoothness; comparing the operation trajectory of the virtual puncture needle with the theoretical puncture path to calculate the path similarity; combining the operation smoothness and the path similarity to calculate the procedure score;

[0046] Complication score, based on the real-time state variables of the virtual puncture needle, combining the dangerous area parameters of the lumbar 3D model, calculating the probability of nerve root injury and the probability of epidural vascular injury, and calculating the complication score according to the probability.

[0047] The application has the following beneficial effects: the three-dimensional visual scene constructed by AR technology can present the complex anatomical structure of the lumbar spine part, such as the intervertebral space, the spinal cord, the subarachnoid space and the like, in a dynamic three-dimensional form, so that the user breaks through the cognitive limitation of the traditional two-dimensional image; the user observes the needle insertion process from different angles, simulates the dynamic effect of the change of the cerebrospinal fluid pressure and the like, and helps the user quickly master the key points of the lumbar puncture operation. In the lumbar puncture simulation training, the AR system can feed back the operation data in real time, such as the needle insertion angle, the puncture strength, the puncture depth and the like, so as to help the user accurately master the operation skill. In addition, the AR system can also simulate the special situations that may occur in the lumbar puncture process, such as the cerebrospinal fluid leakage, the nerve root injury and the like, so that the user can be familiar with the processing flow in advance. The system also has an intelligent recommendation function, which pushes personalized learning materials according to the learning progress and professional direction of the user; the hardware device only needs a smart terminal and a simple AR accessory, which can be used in cooperation with the software system, and supports infinite repeated operation, without the need of a large amount of investment for hardware transformation like the traditional device. The interactive characteristics of the AR technology change the learning from passive infusion to active exploration. The user can interact with the virtual scene through gestures, voice and the like, such as simulating the adjustment of the patient's body position, selecting the appropriate puncture point, performing the puncture operation and the like.

[0048] The application will be further described in detail in combination with the drawings and embodiments, but the application is not limited to the embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 It is a system structure diagram of the embodiment of the application;

[0050] Figure 2 It is an internal flowchart of an AR content generation module of the embodiment of the application;

[0051] Figure 3 It is an internal flowchart of an AR display interaction module of the embodiment of the application;

[0052] Figure 4 It is an internal flowchart of an operation evaluation module of the embodiment of the application. DETAILED DESCRIPTION

[0053] Referring to Figure 1 It is a system structure diagram of the embodiment of the application, which comprises:

[0054] The AR content generation module 101 collects the lumbar spine medical image and performs tissue segmentation and lesion identification; based on the tissue segmentation result and the lesion identification result, three-dimensional geometric modeling is performed to generate a lumbar spine 3D model;

[0055] The AR display interaction module 102 develops the interactive function of the lumbar spine 3D model by using the Unity platform and the AR device; the interactive operation of the user is monitored, and real-time feedback is performed;

[0056] The operation evaluation module 103 evaluates the interactive operation of the user.

[0057] Specifically, the internal processing flow of the AR content generation module 101 is described in detail in the following Figure 2 As shown, the following steps are included:

[0058] S101, lumbar puncture special medical data set construction and integration, including:

[0059] (1) Data acquisition standard. To ensure data science, compliance, the whole process follows the medical ethics review process. In terms of image data, it is planned to collect not less than 100 CT / MRI images, covering normal lumbar spine anatomy, L3-L5 intervertebral space variation (such as transitional vertebra), lumbar disc herniation (Pfirrmann classification II-V grade) and spinal stenosis (Torque classification I-III grade). The surgical video requires double-viewing angle of anteroposterior and lateral positions, and complete recording of the operation process. The puncture point positioning is based on the intersection of the iliac crest line and the posterior median line, and standardized labeling is performed; the needle angle adjustment parameter is closely related to the patient's body type BMI index, realizing individualized simulation. At the same time, cases of complications such as nerve root injury (Nurick classification I-II grade), cerebrospinal fluid leakage (pressure <70 mmH2O) are collected to provide rich clinical materials for simulation training.

[0060] In actual collection, AR glasses CT scanning and database integration are used. The infrared depth sensor carried by the AR glasses and the medical image recognition algorithm are used to scan the patient's lumbar CT image in real time. During scanning, the AR system automatically matches the anatomical landmark points (such as iliac crest, spinous process) in the CT image, locates the L3-L5 intervertebral space area through feature matching algorithm (such as SIFT feature extraction), identifies lesions such as disc herniation and spinal stenosis, and marks them with high-light blocks in the AR view (red: severe stenosis, yellow: moderate protrusion). The scanning accuracy is a spatial positioning error ≤0.5 mm, and the lesion recognition accuracy is ≥95% (based on 100 example verification data set). The scanned CT data is standardized processed in DICOM3.0 format, including: patient basic information (desensitization processing, in line with HIPAA compliance); image metadata (layer thickness 0.625-1.25 mm, matrix 512x512); lesion labeling information (coordinates, size, pathological classification). The database uses a distributed storage architecture (such as Hadoop HDFS), and the storage capacity of a single CT data is about 150-300 MB, supporting 10 cases per second parallel loading, and ensuring integrity through data verification interface (MD5 hash value comparison).

[0061] (2) Data integration. CT / MRI image data, surgical video and annotation information are deeply integrated. Medical image processing software is used to pre-process the image data, such as noise reduction and contrast enhancement, to facilitate subsequent three-dimensional reconstruction. Key information is extracted from the surgical video and accurately associated with the image data to build a complete lumbar puncture surgery dataset, laying a solid data foundation for pathological feature modeling and interactive scene design.

[0062] S102, precise modeling of pathological features, including:

[0063] (1) Three-dimensional reconstruction parameters.

[0064] First, image preprocessing and tissue segmentation are performed. Preprocessing includes image noise reduction and tissue segmentation. Non-local mean filtering (NLM) is used to remove CT noise, and the formula is:

[0065]

[0066] where I(x) is the pixel value, Ω(x) is the neighborhood window, h is the smoothing parameter (10-20HU), and C(x) is the normalization constant.

[0067] The tissue segmentation, the threshold method based on CT value (cortical bone: 200-1000HU, cancellous bone: 50-200HU) is used for vertebral segmentation, combined with the U-Net deep learning model, the segmentation accuracy Dice coefficient ≥0.92; the region growing algorithm is used for dural sac and nerve root segmentation, the seed point is set to the posterior edge of L3-L4 intervertebral space (CT value-100 to 0HU), and the growth threshold is ±20HU, to ensure that the boundary extraction error is ≤1mm.

[0068] Second, three-dimensional geometric modeling is performed. Based on the segmentation results, the Marching Cubes algorithm is used to construct three-dimensional grid models of structures such as vertebrae and dural sac, and the formula is:

[0069] ;

[0070] where, represents the three-dimensional basis function, is the vertex weight, and the grid resolution is set to 0.2mm×0.2mm×0.2mm to ensure complete restoration of anatomical details (such as lamina and spinous process);

[0071] Different parts of the model are parameterized, including:

[0072] Vertebrae modeling, set the elastic modulus of vertebral cortical bone 15±2GPa, cancellous bone 0.5±0.1GPa. Introduce BMI self-adaptive adjustment mechanism, when BMI≥30, bone density increases by 15%. According to CT / MRI image data, the geometric shape of the vertebrae is accurately constructed, and the corresponding physical property parameters are assigned.

[0073] Dural sac modeling, the end of the dural sac is positioned at the lower edge of L2 vertebral body (variation range L1-L3), and the puncture point is marked at the area 0.5-1cm away from the midline after L3-L4 / L4-L5 intervertebral space, to ensure the accuracy of the puncture simulation position. During modeling, the shape and position of the dural sac are strictly restored to ensure the true anatomical relationship with the surrounding tissues.

[0074] Dangerous area modeling, detailed labeling of epidural venous plexus (diameter 1-3mm) and L5 nerve root running path (angle 25°-35° with sagittal plane).

[0075] The physical properties of different parts of the model are calculated according to the CT value, and the mapping relationship between bone mineral density (BMD) and elastic modulus (E) is established, including:

[0076] Cortical bone,

[0077] Cancellous bone,

[0078] Yellow ligament, shear modulus (refer to clinical data).

[0079] Adjust the parameters combined with the patient's BMI: when BMI≥28, the soft tissue thickness increases by \(0.8\times(BMI-24)\)mm, and the puncture resistance coefficient is increased by 12%.

[0080] (2) Visualization specification. To improve the visualization effect of AR simulation, the visualization standards of each tissue are clearly defined. The dural sac is semi-transparent blue (RGB0,114,189) with a transparency of 50%, which can clearly show the internal structure without affecting the observation of the surrounding tissues; the nerve root uses yellow (RGB255,204,0), and when highlighted, the transparency is adjusted to 80%, which is highlighted in the surgical simulation to remind the operator to pay attention; the bone tissue is white (RGB240,240,240), which is distinguished from cortical bone and cancellous bone by different textures, enhancing the visual level and realism. In the AR scene, these settings help users intuitively understand the anatomical structure of the surgical site, improving the accuracy and practicality of the simulation.

[0081] Specifically, the internal processing flow of the AR display interaction module 102 is as follows Figure 3As shown, first, with the help of Unity's XR development framework (such as XRInteractionToolkit, OpenXR), the depth adaptation with existing AR glasses (including host and glasses) is realized, and the lumbar 3D model is presented to the user. Since only AR glasses are equipped, without additional controllers, the interaction design will make full use of the 3DoF ray and head movement provided by the AR glasses host, combined with voice instructions, to maximize the potential of hardware interaction. Then based on the existing AR glasses hardware (the host provides 3DoF ray, vibration feedback, and the glasses are responsible for visual and auditory presentation), the interactive physical simulation of the needle insertion path is developed in the Unity engine, including theoretical path calculation and user operation monitoring. The virtual puncture needle is simulated using the host's 3DoF ray, and the ray direction and angle are real-time corresponding to the user's head rotation, making it easier to calculate the puncture angle. When the ray touches different tissues, according to the preset biomechanical properties, the physical material parameters (friction, elasticity, etc.) of the collision body are configured through the Unity physics engine, and different puncture resistance feedback is set for each layer of tissue. For example, when the virtual puncture needle contacts the yellow ligament, the host triggers the "increased resistance" vibration feedback, simulating the real breakthrough feeling; at the same time, the glasses present the visual effect of the puncture needle being blocked, and cooperate with the "click" sound effect to enhance the sense of immersion. During the simulation, the glasses display the position, angle, and depth information of the puncture needle in real time, making it convenient for operation adjustment. The system captures the operation data in real time and performs feedback calculation based on the preset biomechanical model.

[0082] Specifically, the establishment of the theoretical puncture path includes:

[0083] Puncture point positioning, with the intersection of the line connecting the iliac crest and the posterior median line as the origin The puncture point coordinate calculation formula is:

[0084] ;

[0085] Where, is the z-coordinate of the midpoint of the L3-L4 intervertebral space, unit: mm.

[0086] Dynamic calculation of needle insertion angle, including standard angle , BMD is in mg / cm³; when there is spinal stenosis (Torque classification above grade II), the angle is lowered by 5°-10°; when there is lumbar disc herniation (Pfirrmann classification II-V grade), the system automatically lowers the needle insertion angle by 3°-8°; when there is nerve root injury (Nurick classification I-II grade), the needle insertion angle is slightly adjusted to the contralateral side by 2°-5°; when there is cerebrospinal fluid leakage (pressure <70mmH2O), the angle fluctuates too much (such as more than ±3°), the system will determine that "the operation is unstable and may aggravate the leakage".

[0087] Resistance-depth curve establishment, puncture resistance The relationship between (N) and depth d (cm) is expressed as:

[0088] ;

[0089] Complication warning model. When the distance between the puncture needle and the nerve root \(D<2mm\), the warning is triggered, and the risk value R is calculated as:

[0090] ;

[0091] When R>80, the VR interface is red, and the vibration frequency increases with R.

[0092] Model verification and iteration mechanism. Using 100 cases of CT data in the database to generate the model, compared with the real operation video, the path deviation is ≤1.2mm, and the angle error is ≤2.5°, which meets the clinical requirements. Every 50 cases of CT data is added, the U-Net segmentation model is updated through transfer learning to ensure that the recognition accuracy of rare anatomical variations (such as transitional vertebrae) is ≥90%.

[0093] Specifically, the monitoring of user operation includes:

[0094] Complication monitoring. When the puncture needle and the sagittal plane angle is >45° and the depth is >6cm, the nerve root injury complication mechanism is triggered, the cursor in the AR glasses becomes red, the AR glasses host continuously vibrates, and the warning sound effect is played; The glasses screen displays a nerve edema animation, prompting the user's operation error and potential risks; The puncture needle touches the venous plexus (collision volume >0.5mm³), triggering the epidural hemorrhage complication mechanism, presenting a blood exudation particle effect, the AR glasses host is accompanied by vibration feedback, simulating the bleeding scene, and strengthening the user's awareness and response ability to dangerous operation.

[0095] Needle insertion angle monitoring. The system displays the needle insertion angle in real time, and the standard value is set to 30°-45°. When the deviation is 1°-3°, the AR glasses display a yellow text prompt; When the deviation is >5°, the text prompt changes to red and is accompanied by a rapid prompt sound, reminding the user to adjust the angle in time.

[0096] Three-dimensional path deviation monitoring. On the edge of the AR scene, the spatial distance between the puncture trajectory and the standard path (threshold ≤2mm) is displayed in real time with a semi-transparent interface. When the deviation exceeds the threshold, the path line flashes red, directly presenting the operation error and assisting the user to calibrate the puncture path.

[0097] Needle insertion depth monitoring. Real-time monitoring of puncture depth, standard depth is 4-6cm. When it exceeds 7cm, the AR glasses pop up a "needle insertion too deep" warning pop-up window, the AR glasses host vibrates and plays an alarm sound; When the depth is less than 3cm, "puncture too shallow" is displayed to ensure that the user can master the correct needle insertion depth.

[0098] Cerebrospinal fluid monitoring. The cerebrospinal fluid pressure value (normal range 70-180mmH2O) is displayed in real time at a fixed position in the AR interface, and abnormal values are highlighted in red to facilitate observation of physiological index changes. Simulate the normal outflow of cerebrospinal fluid 1-2ml / min, and support pressure fluctuation simulation (±10-20mmH2O during breathing), which is directly presented through the change of particle flow velocity. The relationship curve between needle insertion speed and resistance is dynamically drawn in the corner of the AR interface, and the breakthrough point of the yellow ligament is automatically marked to help users analyze the operation process and summarize experience.

[0099] Specifically, various auditory prompts are also designed for the hardware conditions of AR glasses only: when the needle insertion angle deviates, a sharp prompt sound is played; when approaching the dangerous area, an alarm sound with gradually increasing frequency is sounded. Different clinical scenarios such as lumbar puncture under normal anatomy and pathological variation are also set to improve the user's ability to cope with complex situations. Voice guidance and text prompt functions are added to provide operation guidance at key steps to help beginners quickly master the lumbar puncture technique, fully utilize the advantages of existing hardware, create a high-immersion lumbar puncture surgery simulation environment, and serve medical education and training.

[0100] Specifically, the physical material parameters (friction, elasticity, etc.) of the collision body are configured through the Unity physical engine, and vibration prompts and visual prompts are given according to the set puncture resistance of each layer of tissue, including:

[0101] Skin-subcutaneous tissue, set resistance threshold ≤5N, when the puncture speed >10mm / s, the AR glasses host emits low-frequency vibration (20Hz, amplitude 0.3mm), and at the same time, the "fast needle insertion" text prompt is displayed in the glasses, reminding the operation state with visual and tactile feedback.

[0102] Supraspinous-inter-spinous ligament, as the puncture deepens, the resistance rises to 8-10N in a gradient, the AR glasses host triggers high-frequency small amplitude vibration (50Hz, amplitude 0.5mm), simulating the resistance change during real puncture; The glasses synchronously present the visual animation of ligament fiber stretching, enhancing the operation immersion.

[0103] Yellow ligament, when the puncture needle contacts the yellow ligament, the resistance rises to 12-18N, the AR glasses host releases low-frequency strong vibration (20Hz, amplitude 1mm), and at the same time, a "click" sound effect is played; The glasses show the ligament fiber deformation animation, simulating the real experience at the breakthrough moment, helping users perceive the key operation node.

[0104] The dural sac breaks through the yellow ligament, and the resistance drops to ≤3N, and the AR glasses host vibration is weakened. The system synchronously activates the cerebrospinal fluid pressure simulation (normal 70-180mmH2O), and the cerebrospinal fluid flow effect is presented through the particle system, and the flow rate is positively correlated with the pressure value (flow rate = pressure x 0.05mm³ / s), so as to intuitively display the physiological phenomenon through dynamic visual effect.

[0105] Specifically, as shown in the following table 1:

[0106] Table 1-vibration prompt and visual prompt based on puncture resistance:

[0107]

[0108] Specifically, in Unity, the 3DoF ray emitted by the AR glasses host can also be controlled by head movements to simulate the "gaze selection" operation. When the ray points to the lumbar spine 3D model, a short focus can trigger the model to zoom in, zoom out, and rotate, supporting detailed observation of the internal structure of the lumbar spine from different angles; combined with the voice recognition plug-in, voice commands (such as "show lumbar puncture needle insertion steps" and "enlarge nerve root") are converted into scene switching or model operation instructions, reducing the dependence on additional hardware.

[0109] Specifically, the internal process of the operation evaluation module 103 is shown in Figure 4 By reading the operation data transmitted by the "AR display interaction module" in real time, key information such as puncture point selection, needle insertion sequence, operation time, and error times is recorded in detail, and the real-time angle of the ray, puncture depth, and other data are compared with the pre-set normal anatomical structure and pathological model parameters in the "AR content generation module" to accurately judge whether the operation is in compliance, and individualized improvement suggestions are made according to the evaluation results.

[0110] Specifically, the evaluation adopts the analytic hierarchy process (AHP) to determine the weight of each index: theoretical learning accounts for 20%, operation standardization accounts for 50%, and risk response accounts for 30%. The scoring formula is: total score = theoretical score x 0.2 + (puncture point score + angle score + flow score) x 0.5 + complication handling score x 0.3. The model compares the operation data monitored in real time in the "AR display interaction module" with the standard parameters in the "AR content generation module" to ensure the accuracy and authority of the evaluation results.

[0111] Specifically, the puncture point score is predicted by a reinforcement learning scoring model, which introduces the Q-learning algorithm to dynamically adjust the scoring weight, represented as:

[0112] Q(s,a)=Q(s,a)+α[r+γmaxQ(s',a')-Q(s,a)];

[0113] State s = {operation step, error type, learning stage};

[0114] Action a = {increase / decrease the weight of a certain indicator};

[0115] For example, if the score in the "puncture point positioning" link is less than 70 for 3 times in a row, the system automatically increases the weight of this indicator from 20% to 25%.

[0116] Specifically, the angle score is obtained by comparing the puncture depth of the virtual puncture needle with the angle of the theoretical puncture path.

[0117] Specifically, the process score is obtained by integrating multi-source data for comprehensive evaluation. In terms of motion capture, relying on AR glasses hardware, the 6DOF pose of the puncture needle (simulated by 3DoF ray) is recorded at a frequency of 120Hz, the acceleration change rate (Jerk value) is calculated to evaluate the smoothness of the operation; the operation trajectory generated by the user through the 3DoF ray of the AR glasses host is time-aligned with the theoretical puncture path, and the similarity score is calculated. The core algorithm pseudocode is as follows:

[0118]

[0119] Specifically, the complication handling score is obtained by implementing risk prediction based on Bayes. The embodiment of the present application constructs a probabilistic graphical model to predict the risk of complications in real time, taking the real-time state variables (puncture depth, angle, speed, tissue contact type) of the virtual puncture needle as input, combining the dangerous region parameters of the pathological model in the "AR content generation module", and calculating the "nerve root injury probability" (P(NI)) and "epidural vessel injury probability" (P(EVI)). When P(NI) exceeds the set threshold, the system automatically interacts with the "AR display interaction module", reduces the forward speed of the puncture needle, highlights the dangerous area, and issues a warning sound in the AR glasses to avoid risks in advance.

[0120] Specifically, the personalized improvement suggestion is automatically pushed to the learning content based on the evaluation results. If the puncture point positioning error exceeds the set threshold, the system will retrieve relevant teaching videos from the teaching resource library of the "AR content generation module" and start targeted virtual marker training in the "AR display interaction module". If the needle angle deviation is too large, the angle auxiliary line function is started in the "AR display interaction module", and multiple special angle control training is arranged to help users correct their operation habits.

[0121] Specifically, in the AR content generation module, for the construction of 3D models, in addition to using existing medical image data processing techniques, image recognition and reconstruction techniques based on artificial intelligence can also be used. Specifically, a large number of lumbar CT and MRI image data are trained using deep learning algorithms such as generative adversarial networks (GAN) and convolutional neural networks (CNN), and the model can automatically identify the boundaries and features of each tissue of the lumbar spine, and then quickly generate a high-precision 3D model. Taking a patient with lumbar disc herniation as an example, this technology can accurately capture the morphological changes of the lesion site, and compared with traditional methods, the model generation efficiency is improved by about 60%, and the detail restoration is higher. At the same time, through natural language processing technology, doctors only need to input simple descriptions (such as "generate a lumbar puncture operation model for a patient with L4-L5 intervertebral space stenosis"), and the system can automatically generate a customized 3D model containing pathological features based on the knowledge base, greatly improving the relevance of teaching content.

[0122] Specifically, in the AR display interaction module, in addition to gesture and voice interaction methods, eye tracking technology can also be introduced, allowing doctors to operate the AR scene through eye contact, further improving the convenience and naturalness of interaction. In lumbar puncture simulation teaching, after the doctor wears AR glasses integrated with eye tracking devices, the system can capture the eye movement trajectory in real time. When the doctor gazes at a specific part of the lumbar spine 3D model (such as the intervertebral space), the system automatically enlarges and highlights that area, making it easier for the doctor to observe the details; during the puncture operation, the doctor only needs to confirm the puncture point through eye contact, and the system can automatically mark it without manual operation. In addition, by analyzing the eye movement data of the doctor using machine learning algorithms, the learning concentration and knowledge mastery of the doctor can be determined, for example, if the doctor gazes at a certain operation step for a long time, the system may infer that he or she has difficulty understanding, and automatically push relevant teaching materials or provide voice explanations, realizing intelligent interactive learning.

[0123] Specifically, in the learning evaluation module, in addition to the quantitative scoring method based on preset standards, a big data analysis-based evaluation method can be introduced. A large amount of lumbar puncture operation data of clinical doctors (including operation steps, operation time, complication rate, etc.) is collected to construct a lumbar puncture operation behavior database. Using clustering analysis, decision tree and other algorithms, the difference characteristics of the operation behaviors of doctors at different levels are mined to form a dynamic evaluation model. For the user's simulation operation, the system not only compares it with the standard process, but also matches and analyzes its operation data with the data of doctors at different levels in the database to give an evaluation result that is more in line with the actual clinical situation. For example, if the user's operation mode is similar to that of an experienced doctor, but there is a slight deviation in a certain step, the system will accurately point out that step and provide targeted optimization suggestions to help the user improve to the level required for clinical operation more quickly.

[0124] It can be seen that the three-dimensional visualization scene constructed by the AR technology can present the complex anatomical structure of the lumbar spine part, such as the intervertebral space, the spinal cord, the subarachnoid space and the like, in a dynamic three-dimensional form, so that the user breaks through the cognitive limitation of the traditional two-dimensional image. The AR system can enable the user to observe the needle insertion process from different angles, simulate the dynamic effect of the change of the cerebrospinal fluid pressure and the like, and help the user quickly master the key points of the lumbar puncture operation. In the simulation training of the lumbar puncture, the AR system can feed back the operation data in real time, such as the needle insertion angle, the puncture strength, the puncture depth and the like, to help the user accurately master the operation skills; in addition, the AR system can simulate the special situations that may occur in the lumbar puncture process, such as the cerebrospinal fluid leakage, the nerve root injury and the like, so that the user can be familiar with the processing flow in advance. The application breaks the shackles of time and space in the traditional teaching, relies on the cloud server to integrate a large amount of medical data related to the lumbar puncture, covers 3D models of different patient lumbar anatomical variations, lumbar puncture operation videos, complication treatment cases and the like, and the user can call the resources at any time through the mobile terminal, and is not restricted by the opening time and the region of the hospital training room. The system also has an intelligent recommendation function, which can push personalized learning materials according to the learning progress and professional direction of the user, such as the key points of the lumbar puncture operation for pediatric patients, the matters needing attention for the lumbar puncture of elderly patients and the like, to further improve the pertinence and efficiency of learning. The hardware device of the application only needs an intelligent terminal and a simple AR accessory, and can be used in cooperation with the software system, and the single use cost is less than 1 / 10 of the traditional mode. The system also supports infinite repeated operation, avoids resource waste, and greatly reduces the financial pressure of teaching institutions. The updating and upgrading of the system only needs to optimize the software in the cloud, without the need of a large amount of investment for hardware modification like the traditional device. The interactive characteristics of the AR technology change the learning from passive infusion to active exploration. The user can interact with the virtual scene through gestures, voice and the like, such as simulating adjustment of the patient's position, selection of a suitable puncture point, puncture operation and the like. The system design can further integrate an incentive mechanism such as points and grades, and develop a function of supporting online collaborative learning of multiple people, so that the user can form a team to complete the simulation of the lumbar puncture diagnosis and operation under complex conditions.

[0125] The above is only a preferred embodiment of the application, and is not intended to limit the application. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the application shall be included in the protection scope of the application.

Claims

1. An AR technology-based lumbar puncture simulation teaching system, characterized by, The application relates to a lumbar spine AR model interactive system, which comprises the following modules: An AR content generation module collects lumbar spine medical images and carries out tissue segmentation and lesion identification; Three-dimensional geometric modeling is carried out based on the tissue segmentation result and the lesion identification result to generate a lumbar spine 3D model; An AR display interaction module develops the interactive function of the lumbar spine 3D model by using a Unity platform and AR equipment; the interactive operation of a user is monitored, and real-time feedback is carried out; An operation evaluation module evaluates the interactive operation of the user, and specifies improvement suggestions according to the evaluation result; The interactive function of the lumbar spine 3D model developed by using the Unity platform and the AR equipment comprises the following steps: A display function is used to present the lumbar spine 3D model on AR glasses; A puncture function is used, in which the physical engine of the Unity is used to set differentiated puncture resistance feedback for different parts of the lumbar spine 3D model; a theoretical puncture path is calculated; the user carries out puncture operation by using the 3DoF ray of the AR glasses as a virtual puncture needle, and the puncture operation of the user is monitored in real time to update the state of the lumbar spine 3D model; A prompt function is used to realize various prompts by using the display function and the vibration function of the AR glasses; The puncture resistance feedback set by the physical engine of the Unity for different parts of the lumbar spine 3D model is represented as follows: ； Puncture resistance in N and the depth d in cm; The state of the lumbar spine 3D model is updated by monitoring the puncture operation of the user in real time, which comprises the following steps: 2.The AR technology-based lumbar puncture simulation teaching system according to claim 1, characterized in that, A particle system is used to present the cerebrospinal fluid flow effect, the flow speed is positively correlated with the cerebrospinal fluid pressure value, and the flow speed change of the particle is directly presented. The three-dimensional geometric modeling based on the tissue segmentation result and the lesion identification result to generate the lumbar spine 3D model comprises the following steps: The lumbar spine medical images are segmented into vertebrae, dural sacs and nerve roots, different parts are modeled based on the segmentation result, and a lumbar spine 3D preliminary model is obtained; 3.The AR technology-based lumbar puncture simulation teaching system according to claim 2, characterized in that, The lumbar spine medical images are identified, the identified lesions are marked in the lumbar spine 3D preliminary model, and the lumbar spine 3D model is obtained. Vertebrae modeling, according to the lumbar vertebrae medical image to build the geometric shape of the vertebrae and give the corresponding physical attribute parameters, including the elastic modulus of the vertebral cortex bone 15±2GPa, the cancellous bone 0.5±0.1GPa, the shear modulus of the yellow ligament ; the mapping relationship between bone mineral density BMD and elastic modulus E is established, and a BMI self-adaptive adjustment mechanism is introduced; the color of the bone tissue is white, and different textures are set for the cortical bone and cancellous bone; BMI represents the body mass index; The segmentation of the lumbar spine medical images into vertebrae, dural sacs and nerve roots, and the modeling of different parts based on the segmentation result to obtain the lumbar spine 3D preliminary model comprise the following steps: Dural sac modeling is carried out, the end of the dural sac is positioned at the lower edge of the L2 vertebra, and a puncture point is marked in the region 0.5-1 cm away from the median line after the L3-L4 intervertebral space or the L4-L5 intervertebral space; the color of the dural sac is blue with a transparency of 50%; 4.The AR technology-based lumbar puncture simulation teaching system according to claim 1, characterized in that, Dangerous area modeling is carried out, and the epidural venous plexus and the L5 nerve root path are marked; the color of the nerve root is yellow with a transparency of 80%. The prompt function comprises the following steps: When the puncture speed is greater than 10 mm / s, the AR glasses host machine gives a low-frequency vibration when the puncture reaches the skin-subcutaneous tissue; When the puncture reaches the supraspinous-inter-spinous ligament, the AR glasses host machine gives a high-frequency vibration; When the puncture needle contacts the yellow ligament, the AR glasses host machine gives a strong vibration; When the distance D between the needle and the nerve root is < 2 mm, a complication warning is triggered, and the vibration frequency Increases with increasing risk value R, expressed as: ； 。 5.The AR technology-based lumbar puncture simulation teaching system according to claim 1, wherein, When the puncture reaches the dural sac, the AR glasses host machine gives a weak vibration; Puncture point positioning, with the line connecting iliac crest as With the posterior median line as With the intersection of the line connecting iliac crest and the posterior median line as the origin, the coordinates of the puncture point are represented as: ； wherein, is the z coordinate of the midpoint of the L3-L4 intervertebral space; BMI represents the body mass index; Dynamic model of needle insertion angle, including standard angle , BMD represents bone mineral density, unit: mg / cm³; when there is spinal canal stenosis, the angle is adjusted down by 5°-10°; when there is lumbar disc herniation, the system automatically adjusts the needle insertion angle down by 3°-8°; when there is nerve root injury, the needle insertion angle is fine-tuned to the opposite side by 2°-5°; when there is cerebrospinal fluid leakage, set the angle fluctuation threshold, and the user's puncture operation angle fluctuation exceeding the threshold will issue a warning. 6.The AR technology-based lumbar puncture simulation teaching system according to claim 5, characterized in that, The calculation of the theoretical puncture path comprises the following steps: The prompt function comprises the following steps: The needle insertion angle monitoring is used to display the needle insertion angle in real time, and a prompt is given when the actual angle deviates from the calculation result of the needle insertion angle dynamic model. A three-dimensional path deviation map is used to display the spatial distance between the actual puncture trajectory and the theoretical puncture path in real time, and a prompt is given when the deviation exceeds a threshold value; Depth monitoring is used to monitor the puncture depth in real time, and a prompt is given when the puncture depth exceeds a standard depth range. 7.The AR technology-based lumbar puncture simulation teaching system according to claim 1, wherein, The real-time monitoring of the user's puncture operation is used to update the state of the lumbar 3D model, including: After the puncture needle breaks through the yellow ligament and enters the dural sac, the resistance drops sharply to ≤3N, and the AR glasses host vibration is weakened; When the puncture needle touches the venous plexus, the AR glasses present a blood exudation particle effect with vibration feedback, simulating a bleeding scene. 8.The AR technology-based lumbar puncture simulation teaching system according to claim 1, wherein, The evaluation performed by the operation evaluation module includes: Puncture point scoring, comparing the puncture point of the user's operation with the puncture point of the theoretical puncture path to evaluate the puncture point score; Angle score, comparing the puncture angle of the user's operation with the puncture angle of the theoretical puncture path to evaluate the angle score; Process score, relying on the AR glasses hardware, recording the 6DOF pose of the virtual puncture needle, calculating the acceleration change rate to evaluate the operation smoothness; comparing the operation trajectory of the virtual puncture needle with the theoretical puncture path to calculate the path similarity; combining the operation smoothness and the path similarity to calculate the process score; Complication score, based on the real-time state variables of the virtual puncture needle, combining the dangerous area parameters of the lumbar 3D model, calculating the probability of nerve root injury and the probability of epidural vascular injury, and calculating the complication score according to the probability.

Images