Radial artery puncture catheterization training system based on VR platform

Abstract

The invention discloses a radial artery puncture catheterization training system based on a VR platform, and relates to the technical field of medical trainers, and the system comprises a user side which is used for selecting a radial artery puncture catheterization training scene number; the head-mounted VR wearable terminal is worn on the head of a user to execute radial artery puncture catheterization training; the VR platform comprises a 3D modeling module for constructing a puncture catheterization training scene model; the puncture posture acquisition module is used for acquiring puncture catheterization posture monitoring time sequence information; and the training feedback module is used for comparing the puncture catheterization posture monitoring time sequence information with the puncture catheterization posture target time sequence information, generating voice correction information according to the posture deviation vector information, and sending the voice correction information to the head-mounted VR wearable terminal to remind a user of correction. The technical problem that the radial artery puncture catheterization training effect is poor due to the lack of real patient response information and interactive training schemes in the prior art is solved, and the technical effect of improving the radial artery puncture catheterization training effect is achieved.

Description

Technical Field

[0001] The present application relates to the technical field related to medical trainers, and in particular to a radial artery puncture and catheterization training system based on a VR platform. Background Art

[0002] With the rapid development of modern medical technology, minimally invasive interventional surgery and invasive hemodynamic monitoring technology are increasingly used in clinical practice. Among them, radial artery puncture and catheterization technology is an important operational skill, which is widely used in cardiovascular interventional treatment, angiography, invasive hemodynamic monitoring and other medical fields. As a skill with strong operability, radial artery puncture and catheterization technology places high demands on the clinical operation level and proficiency of medical staff. In order to ensure the success rate of puncture and catheterization and reduce damage to patients, medical staff need to undergo a lot of practice and operation training before actual operation. However, due to many limitations such as training resources and patient safety, traditional arterial puncture and catheterization training mainly relies on operations on patients or the use of human models. The participation of real patients may increase the uncertainty and risk in the training process, thereby causing medical disputes. The simulation model lacks real patient response information, such as the feeling of arterial pulsation, changes in vascular elasticity, and changes in arterial pulsation that occur with changes in the patient's condition. It is also difficult to fully and realistically restore the puncture and catheterization operation scene, which not only affects the training effect, but may also cause operators to lack sufficient confidence and operational proficiency when facing real scenes. Based on this, how to improve the training effect of radial artery puncture and catheterization through a high-precision and controllable virtual environment without relying on real patients has become an urgent problem to be solved in the field of medical training.

[0003] Therefore, in the current radial artery puncture and catheterization training-related technologies, there is a lack of real patient response information and interactive training programs, which leads to technical problems such as poor results in radial artery puncture and catheterization training. Summary of the invention

[0004] The present application provides a radial artery puncture and catheterization training system based on a VR platform, thereby solving the technical problem in the prior art of lacking real patient response information and interactive training programs, resulting in poor results in radial artery puncture and catheterization training, and achieving the technical effect of improving the results of radial artery puncture and catheterization training.

[0005] The present application provides a radial artery puncture and catheterization training system based on a VR platform, the system comprising: a user end, the user end is used to select a radial artery puncture and catheterization training scene number; a head-mounted VR wearable terminal, the head-mounted VR wearable terminal is worn on the user's head to perform radial artery puncture and catheterization training; the VR platform comprises: a 3D modeling module, the 3D modeling module is used to construct a puncture and catheterization training scene model based on a scene database according to the radial artery puncture and catheterization training scene number, and transmit it to the head-mounted VR wearable terminal through a scene output device to perform radial artery puncture and catheterization training; a puncture posture acquisition module, the puncture posture acquisition module is used to collect puncture and catheterization posture monitoring timing information when the user performs radial artery puncture and catheterization training; a training feedback module, the training feedback module is used to compare the puncture and catheterization posture monitoring timing information with the puncture and catheterization posture target timing information to obtain posture deviation vector information, generate voice correction information according to the posture deviation vector information, and send it to the head-mounted VR wearable terminal to remind the user to make corrections.

[0006] In a possible implementation, the radial artery puncture and catheterization training system based on a VR platform further performs the following processing: the user terminal is embedded in the head-mounted VR wearable terminal.

[0007] In a possible implementation, the 3D modeling module further performs the following processing: inputting the radial artery puncture and catheterization training scene number into a feature database to match training features, wherein the training features include model features, medical device features, and scene features; inputting the model features, the medical device features, and the scene features into the scene database to match the puncture target three-dimensional model, the medical device three-dimensional model, and the scene three-dimensional model; and fusing the puncture target three-dimensional model, the medical device three-dimensional model, and the scene three-dimensional model through a VR scene fusion network to generate the puncture and catheterization training scene model.

[0008] In a possible implementation, the 3D modeling module also performs the following processing: collecting puncture target three-dimensional model recording data, medical tool three-dimensional model recording data, scene three-dimensional model recording data and fused three-dimensional model identification data; using the fused three-dimensional model identification data as supervision data, and using the puncture target three-dimensional model recording data, the medical tool three-dimensional model recording data, and the scene three-dimensional model recording data as input variables, training a fully convolutional neural network to obtain the VR scene fusion network.

[0009] In a possible implementation, the 3D modeling module further performs the following processing: using the fused three-dimensional model identification data as supervision data, and the puncture target three-dimensional model record data, the medical device three-dimensional model record data, and the scene three-dimensional model record data as input variables to construct a VR scene fusion network construction data set; after dividing the VR scene fusion network construction data set into k groups, perform k times of extraction with replacement to obtain a first VR scene fusion network construction data set; based on the first VR scene fusion network construction data set, train a fully convolutional neural network to obtain a first VR scene fusion network; loop M times to obtain an MVR scene fusion network; fully connect the first VR scene fusion network until the MVR scene fusion network to obtain the VR scene fusion network.

[0010] In a possible implementation, the 3D modeling module also performs the following processing: constructing a full connection rule: calculating the pairwise similarities of the first VR scene fusion network to the first output VR scene to the Mth output VR scene of the MVR scene fusion network, and obtaining several output VR scene similarities; performing LOF outlier detection based on the several output VR scene similarities to obtain the first outlier factor to the Mth outlier factor; outputting the output VR scene corresponding to the minimum value of the first outlier factor to the Mth outlier factor, and setting it as the output value of the VR scene fusion network; according to the full connection rule, fully connecting the first VR scene fusion network to the MVR scene fusion network to obtain the VR scene fusion network.

[0011] In a possible implementation, the training feedback module also performs the following processing: parsing the motion characteristics of the puncture catheter posture monitoring timing information to obtain the first puncture site, the first puncture angle, the first puncture depth, and the first puncture time; parsing the motion characteristics of the puncture catheter posture target timing information to obtain the second puncture site, the second puncture angle, the second puncture depth, and the second puncture time; comparing the first puncture site, the first puncture angle, the first puncture depth, and the first puncture time with the second puncture site, the second puncture angle, the second puncture depth, and the second puncture time to obtain the puncture deviation vector information.

[0012] In a possible implementation, the training feedback module also performs the following processing: activating a feature extraction model, wherein the feature extraction model is a SLOWFAST neural network; splitting the puncture catheterization posture monitoring timing information according to the first step length to obtain a first puncture catheterization posture sequence; splitting the puncture catheterization posture monitoring timing information according to the second step length to obtain a second puncture catheterization posture sequence, wherein the second step length is one eighth of the first step length; processing the first puncture catheterization posture sequence and the second puncture catheterization posture sequence according to the feature extraction model to obtain the first puncture site, the first puncture angle, the first puncture depth, and the first puncture time.

[0013] The present application also provides an electronic device, comprising: a memory for storing executable instructions; and a processor for executing the executable instructions stored in the memory to implement a radial artery puncture and catheterization training system based on a VR platform.

[0014] The present application also provides a computer-readable storage medium, including: a computer program stored thereon, which, when executed by a processor, implements a radial artery puncture and catheterization training system based on a VR platform.

[0015] The radial artery puncture and catheterization training system proposed in this application is based on a VR platform. The user end is used to select the radial artery puncture and catheterization training scene number; the head-mounted VR wearable terminal is worn on the user's head to perform radial artery puncture and catheterization training; the VR platform includes: a 3D modeling module to build a puncture and catheterization training scene model; a puncture posture acquisition module to collect puncture and catheterization posture monitoring timing information; a training feedback module to compare the puncture and catheterization posture monitoring timing information with the puncture and catheterization posture target timing information, and generate voice correction information based on the posture deviation vector information and send it to the head-mounted VR wearable terminal to remind the user to make corrections. It solves the technical problem in the prior art that the lack of real patient response information and interactive training programs leads to poor results in radial artery puncture and catheterization training, and achieves the technical effect of improving the training effect of radial artery puncture and catheterization. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solution of the embodiment of the present disclosure, the accompanying drawings of the embodiment of the present disclosure will be briefly introduced below. A flow chart is used in the present application to illustrate the operations performed by the system according to the embodiment of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps can be processed in reverse order or simultaneously as needed. At the same time, other operations can also be added to these processes, or one or more operations can be removed from these processes.

[0017] Figure 1A schematic diagram of the structure of a radial artery puncture and catheterization training system based on a VR platform provided in an embodiment of the present application;

[0018] Figure 2 A schematic diagram of the execution process of a 3D modeling module in a radial artery puncture and catheterization training system based on a VR platform provided in an embodiment of the present application;

[0019] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0020] Explanation of the reference numerals: user terminal 10, head-mounted VR wearable terminal 20, VR platform 30, 3D modeling module 31, puncture posture acquisition module 32, training feedback module 33, input device 401, processor 402, memory 403, output device 404. DETAILED DESCRIPTION

[0021] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below.

[0022] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0023] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict, and the terms "first\second" involved are merely to distinguish similar objects and do not represent a specific ordering of objects. The terms "including" and "having" and any variations are intended to cover non-exclusive inclusions, for example, a process, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, products, or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by technicians in the technical field of this application. The terms used herein are for the purpose of describing the embodiments of the present application only.

[0024] The present application embodiment provides a radial artery puncture and catheterization training system based on a VR platform, such as Figure 1 As shown, the system comprises:

[0025] The user terminal 10 is used to select a radial artery puncture and catheterization training scene number.

[0026] Preferably, the user terminal is used to select the radial artery puncture and catheterization training scene number specifically refers to that the user selects different training scene numbers through the user terminal device (such as a control panel, etc.) before the training starts. These training scene numbers correspond to different preset training scenario modes, which may specifically represent different patient conditions, anatomical structures, puncture difficulties, etc. For example, different scene numbers can correspond to different anatomical characteristics of patients, such as the thickness of the artery in the wrist, the position difference, etc., to simulate the conditions of patients with different body shapes, ages or pathological conditions; different scene numbers can correspond to scenes of different difficulty levels, and the puncture difficulty and the degree of refinement of feedback information in each scene may be different, such as arterial puncture for blood gas collection and arterial puncture and catheterization in different scenes such as obese patients and edema patients; some scene numbers may require the user to complete the puncture under specific circumstances, such as simulating the puncture of a shock patient in an emergency situation, and the puncture path that needs to avoid other blood vessels or tissues; different scene numbers can also represent changes in the external environment, such as light intensity, the influence of hand stability, etc., so that users can train in more diverse situations.

[0027] A head-mounted VR wearable terminal 20 is worn on the user's head to perform radial artery puncture and catheterization training.

[0028] Preferably, a head-mounted VR wearable terminal refers to a virtual reality device that a user wears on the head during radial artery puncture and catheterization training. The head-mounted VR wearable terminal is usually a VR helmet or VR glasses, equipped with a display screen and sensors, for displaying realistic 3D virtual training scenes to users, and allowing users to interact immersively with the virtual environment, aiming to make users feel as if they are in a real puncture and catheterization operation environment, thereby enhancing the training effect. Specifically, the VR terminal presents the virtual environment of radial artery puncture and catheterization in front of users through an internal high-resolution display screen and 3D sound effects, allowing users to view and experience the puncture operation process in all directions; many The VR headset is equipped with built-in sensors that can track the direction and position of the user's head movement. During the training process, the user can observe the puncture area at different angles by moving the head, simulating the process of observing the target position in real operation. At the same time, the device can be linked with the puncture posture acquisition module to record and feedback the user's puncture gestures and postures in real time; the head-mounted VR terminal is connected to the training feedback module to receive real-time feedback information, such as whether the operation is correct and whether the posture is standard. When the user's operation is incorrect or deviates from the standard posture, the VR terminal will issue prompts to the user (such as voice or visual prompts) to help them adjust in time and perform more accurate operations. The head-mounted VR wearable terminal provides a highly realistic simulation experience in radial artery puncture and catheterization training. Through virtual reality technology, users can continue to practice and improve their operational proficiency in a safe and risk-free environment.

[0029] The VR platform 30 includes: a 3D modeling module 31, which is used to construct a puncture and catheterization training scene model based on a scene database according to the radial artery puncture and catheterization training scene number, and transmit it to a head-mounted VR wearable terminal through a scene output device to perform radial artery puncture and catheterization training.

[0030] Preferably, the VR platform refers to the core virtual reality platform that supports the entire radial artery puncture and catheterization training process, which is mainly used to create and manage virtual training scenes, so that users can get an immersive training experience in a head-mounted VR terminal. The VR platform includes multiple modules such as a 3D modeling module, a puncture posture acquisition module, and a training feedback module. The 3D modeling module is the key part and is responsible for building a 3D virtual scene model for radial artery puncture and catheterization training.

[0031] Preferably, the 3D modeling module is used to create a virtual training scene model, and based on the selected training scene number, the corresponding data and image resources are extracted from the scene database to construct a high-precision 3D scene, which includes not only a three-dimensional image of the radial artery and surrounding anatomical structures, but also information such as anatomical variations, vascular orientation, skin layers, etc. that may be encountered, so as to ensure the authenticity and detail accuracy of the training scene; wherein the scene database stores preset scene data required for different training scenarios, for example, the scene database may contain patient models of different ages and body shapes, or common anatomical structures and path information, when the user selects a training scene with a certain number, the 3D modeling module retrieves the corresponding model data from the database; based on these data, a specific 3D scene is generated, that is, according to the specific training scenario corresponding to the scene number, an interactive puncture and catheterization training scene model is generated in a virtual environment, and the puncture and catheterization training scene model can simulate the subtle operational details of radial artery puncture, such as the location of the puncture point, the insertion angle and depth of the needle tip, etc. After the scene construction is completed, the scene output device transmits the generated 3D scene model to the user's head-mounted VR terminal, ensuring that the user can enter the constructed puncture and catheterization training scene in real time when wearing the VR device, observe and participate in the virtual operation.

[0032] The puncture posture acquisition module 32 is used to collect puncture and catheterization posture monitoring time sequence information when the user is performing radial artery puncture and catheterization training.

[0033] Preferably, the puncture posture acquisition module is used to monitor and record the user's operation posture in real time during the radial artery puncture and catheterization training, that is, to collect the user's posture information when performing radial artery puncture and catheterization, such as puncture site, puncture angle, puncture depth information, etc., to help analyze and evaluate the accuracy of the operation. Specifically, the puncture posture acquisition module records the user's puncture operation posture in real time during the training process through sensors, controllers or motion capture devices, including dynamic changes in subtle movements such as hand movements, needle tip angles, puncture strength, and insertion depth, to ensure that each user's movement can be accurately captured and restored in the VR environment. While collecting the puncture posture, the puncture posture acquisition module will record the timing information of the operation, that is, the time point or time period of each operation of the user during the training process, such as the moment of needle tip insertion, the pause duration during the puncture process, the interval for adjusting the angle, etc. The puncture posture acquisition module plays the role of data monitoring in radial artery puncture and catheterization training, ensuring that each posture information and time point can be accurately recorded. The functions of real-time posture acquisition and timing monitoring help improve the quality of radial artery puncture and catheterization training.

[0034] The training feedback module 33 is used to compare the puncture catheterization posture monitoring timing information with the puncture catheterization posture target timing information to obtain posture deviation vector information, generate voice correction information according to the posture deviation vector information, and send it to the head-mounted VR wearable terminal to remind the user to make corrections.

[0035] Preferably, the training feedback module is used to analyze and correct the user's operating posture, that is, to compare the puncture and catheterization posture monitoring timing information of the user's actual operation with the preset puncture and catheterization posture target timing information, identify the operation deviation and provide correction feedback, and generate correction information to remind the user to make corrections. Specifically, the training feedback module compares the puncture and catheterization posture monitoring timing information of the user's actual operation with the preset puncture and catheterization posture target timing information, wherein the puncture and catheterization posture target timing information refers to the ideal radial artery puncture and catheterization operation posture data, which is usually a precise action set by an expert. Through posture comparison, the training feedback module calculates the posture deviation vector information. The deviation vector represents the deviation of the user's actual posture from the standard posture, and contains specific error information, such as the offset of the hand position and the error of the needle tip angle. The deviation vector may be multi-dimensional data, which can describe the displacement, angle difference and time deviation in three-dimensional space to quantify the user's operation error; based on the posture deviation vector information, the training feedback module generates corresponding voice correction information. For example, if the user's needle tip insertion angle is incorrect, the system generates a voice prompt to adjust the insertion angle. These voice correction information are dynamically generated and adjusted in real time according to the specific content of the deviation vector to ensure that the feedback information is accurate and targeted. It is then sent to the head-mounted VR terminal worn by the user and played through the speaker or headphones of the VR terminal to remind the user to adjust the posture in time. The real-time feedback allows users to quickly correct their posture during training, ensuring that users can continuously optimize their operating posture in the virtual training environment and improve the accuracy and standardization of operations.

[0036] A radial artery puncture and catheterization training system based on a VR platform according to an embodiment of the present invention is used to solve the technical problem that the prior art lacks real patient response information and interactive training programs, resulting in poor radial artery puncture and catheterization training results, and achieves the technical effect of improving the radial artery puncture and catheterization training results. A radial artery puncture and catheterization training system based on a VR platform includes: a user terminal 10, a head-mounted VR wearable terminal 20, a VR platform 30, a 3D modeling module 31, a puncture posture acquisition module 32, and a training feedback module 33.

[0037] The following describes in detail the specific configuration of a radial artery puncture and catheterization training system based on a VR platform, including: the user terminal is embedded in the head-mounted VR wearable terminal.

[0038] Preferably, the user terminal is embedded in the head-mounted VR wearable terminal, which means that the user terminal function has been integrated into the head-mounted VR device, and no additional equipment or external controller is required for operation selection and control, that is, the head-mounted VR terminal is not only used to display virtual scenes and provide immersive experience, but also undertakes the interaction and control functions of the user terminal, and has a complete control panel operation interface for users to directly operate when wearing the VR device. By embedding the user terminal in the head-mounted VR terminal, the user can directly complete all operations when wearing the device without the need for other additional equipment, thereby improving the convenience of use; the embedded user terminal usually includes a virtual menu or user interface, and the user can operate in the virtual interface through the control handle, gesture recognition, eye tracking, etc. in the head-mounted VR terminal. For example, the user can view different training scene numbers in the VR display interface, select the scene number by gaze or gesture, and complete the selection of the training scene. After selecting the scene, adjusting the settings or starting the training, the user can immediately obtain corresponding visual or voice feedback in the VR environment to ensure the correctness of the operation.

[0039] The specific configuration of the compensation 3D modeling module will be described in detail below. Figure 2 As shown, the 3D modeling module may further include: inputting the radial artery puncture and catheterization training scene number into a feature database to match training features, wherein the training features include model features, medical device features and scene features; inputting the model features, the medical device features and the scene features into the scene database to match the puncture target three-dimensional model, the medical device three-dimensional model and the scene three-dimensional model; and fusing the puncture target three-dimensional model, the medical device three-dimensional model and the scene three-dimensional model through a VR scene fusion network to generate the puncture and catheterization training scene model.

[0040] Preferably, after the user selects a radial artery puncture and catheterization training scene number, the number is input into the feature database to match a specific training feature, wherein the feature database contains various preset training feature information, and the training features include model features, medical device features and scene features. Model features refer to the patient anatomical model features required for training, such as details of the skin, muscles, radial artery and surrounding anatomical structures at the wrist. Medical device features refer to the features of medical devices related to puncture and catheterization, such as needles, catheters, and fixation devices. Scene features refer to scene-related features such as the background environment, light conditions, and operating space in the training scenario. information; then the matching training features (model features, medical device features and scene features) are input into the scene database, and the corresponding three-dimensional models are retrieved, including the puncture target three-dimensional model, the medical device three-dimensional model and the scene three-dimensional model. The puncture target three-dimensional model is a three-dimensional model of the anatomical structure constructed according to the model features, which contains details such as the patient's skin layer and vascular structure. The medical device three-dimensional model is a three-dimensional model created according to the medical device features and displayed in a virtual environment, such as models of puncture needles and catheters. The scene three-dimensional model is a training background generated according to the scene features, including a simulated operating environment, to help users better adapt to the actual operating environment.

[0041] Finally, the 3D model of the puncture target, the 3D model of the medical equipment and the 3D model of the scene are fused through the VR scene fusion network. The VR scene fusion network is constructed and trained based on a fully convolutional neural network (FCN). Specifically, the patient's anatomical structure, medical equipment and scene background are integrated into the same virtual space to make the size, proportion, position and other aspects of each part consistent, forming a seamless virtual environment, and then generating a puncture and catheterization training scene model, which includes realistic anatomical structures, puncture equipment and background settings that conform to the medical operation environment. Users can see the combined effects of all these models in the VR environment and can conduct highly simulated interactive training in a realistic virtual environment.

[0042] The specific configuration of the compensation 3D modeling module will be described in detail below. The 3D modeling module may further include: collecting puncture target 3D model recording data, medical appliance 3D model recording data, scene 3D model recording data and fused 3D model identification data; using the fused 3D model identification data as supervision data, using the puncture target 3D model recording data, the medical appliance 3D model recording data, and the scene 3D model recording data as input variables, training a fully convolutional neural network, and obtaining the VR scene fusion network.

[0043] Preferably, a high-precision VR scene fusion network is constructed by training a fully convolutional neural network (FCN) to fuse the puncture target three-dimensional model, the medical tool three-dimensional model and the scene three-dimensional model. Specifically, the puncture target three-dimensional model recording data, the medical tool three-dimensional model recording data, the scene three-dimensional model recording data and the fusion three-dimensional model identification data are collected. The puncture target three-dimensional model recording data refers to the three-dimensional model data of the patient's anatomical structure (such as skin, blood vessels, etc.) in the radial artery puncture and catheterization scene, which records the spatial position information, shape characteristics, etc. of the puncture target model. The medical tool three-dimensional model recording data refers to the three-dimensional model data of medical tools (such as needles, catheters, etc.) related to the puncture and catheterization operation, which records the size, shape, material and other characteristics of the medical tools. The scene three-dimensional model recording data refers to the three-dimensional model data of the virtual scene background, including scene details such as the operating space and light conditions. The fusion three-dimensional model identification data is the identification data of the fusion scene generated by integrating the puncture target model, the medical tool model and the scene model, which includes the combination state, relative position relationship, proportion, etc. of each model data in the virtual environment.

[0044] Preferably, the puncture target three-dimensional model recording data, the medical tool three-dimensional model recording data, and the scene three-dimensional model recording data are used as input variables, that is, input information when training the network, and the fused three-dimensional model identification data is used as supervision data to supervise and guide the full convolutional neural network (FCN) network for training, and learn how to integrate the puncture target, medical tool and scene model into a complete three-dimensional scene, wherein the full convolutional neural network (FCN) is a neural network structure suitable for image and three-dimensional data processing, and is particularly suitable for tasks such as semantic segmentation and image generation. The trained full convolutional neural network (i.e., VR scene fusion network) can automatically identify and process the input three-dimensional model data, and accurately fuse different models (puncture targets, medical tools and scenes), that is, the input three-dimensional model data can automatically generate an integrated virtual training scene, so that the user can experience the complete puncture catheterization operation environment in the VR environment, including the real anatomical structure, the precise position of the medical tools and the background setting that conforms to the scene, to ensure the accuracy of the virtual scene and improve the user's operation effect in virtual training.

[0045] The specific configuration of the compensation 3D modeling module will be described in detail below. The 3D modeling module may further include: using the fused 3D model identification data as the supervision data, the puncture target 3D model record data, the medical tool 3D model record data, and the scene 3D model record data as input variables to construct a VR scene fusion network construction data set; after the VR scene fusion network construction data set is divided into k groups, k times of extraction with replacement are performed to obtain a first VR scene fusion network construction data set; according to the first VR scene fusion network construction data set, a full convolutional neural network is trained to obtain a first VR scene fusion network; M cycles are performed to obtain an MVR scene fusion network; the first VR scene fusion network is fully connected until the MVR scene fusion network is obtained to obtain the VR scene fusion network; according to the fused 3D model identification data (as the supervision data) and the puncture target 3D model record data, the medical tool 3D model record data and the scene 3D model record data (as the input variables), a data set for training the VR scene fusion network is constructed, which includes multiple groups of input 3D model record data and corresponding fused 3D model identification data, which is used to supervise the network to learn how to integrate different 3D models into a unified virtual scene.

[0046] Preferably, the constructed data set is equally divided into k groups, and then k extractions with replacement are performed, similar to Bootstrap resampling, to generate different training subsets on the data set, wherein Bootstrap resampling is a statistical method for estimating the characteristics of sample distribution by randomly extracting samples, that is, extracting the same data points from the same data set multiple times to increase the diversity of the data, thereby generating a first VR scene fusion network construction data set for training the first VR scene fusion network; then the first VR scene fusion network construction data set is used to train the fully convolutional neural network to obtain the first VR scene fusion network, specifically, by inputting variable The fully convolutional neural network learns how to fuse puncture targets, medical instruments and scene models into a complete three-dimensional scene based on the amount and supervision data; the above process is repeated M times, and the data set is re-extracted each time and a new VR scene fusion network is trained to obtain M VR scene fusion networks from the first VR scene fusion network to the Mth VR scene fusion network, where k and M are positive integers greater than 1, such as k is 10 and M is 3. Finally, the M VR scene fusion networks are integrated in a fully connected manner to obtain the final VR scene fusion network, which can more accurately and comprehensively realize the fusion of puncture targets, medical instruments and scene models, thereby generating high-quality VR training scenes.

[0047] Below, the specific configuration of the compensation 3D modeling module will continue to be described in detail. The 3D modeling module may further include: constructing a full connection rule: calculating the pairwise similarities of the first VR scene fusion network until the first output VR scene until the Mth output VR scene of the MVR scene fusion network, and obtaining several output VR scene similarities; performing LOF outlier detection based on the several output VR scene similarities, and obtaining the first outlier factor until the Mth outlier factor; outputting the output VR scene corresponding to the minimum value of the first outlier factor until the Mth outlier factor, and setting it as the output value of the VR scene fusion network; according to the full connection rule, fully connecting the first VR scene fusion network until the MVR scene fusion network, and obtaining the VR scene fusion network.

[0048] Preferably, the full connection rule is calculated, that is, the pairwise similarities between the outputs of the first VR scene fusion network to the MVR scene fusion network are calculated by similarity algorithms such as Euclidean distance and cosine similarity, that is, the similarities of the outputs (VR scene models) of different VR scene fusion networks in terms of structure, features, position, content, etc. are calculated, so as to obtain multiple output VR scene similarities, and then LOF (local outlier factor) outlier detection is used to identify the abnormality of the output VR scene similarity, that is, the degree of outlier of each VR scene output compared with other outputs is evaluated and calculated, and represented by the first outlier factor to the Mth outlier factor respectively, the smaller the outlier factor, the higher the similarity between the output and other outputs; according to the result of LOF outlier detection, the output VR scene corresponding to the minimum value from the first outlier factor to the Mth outlier factor is selected as the output value of the final VR scene fusion network; according to the full connection rule, the output result of the final VR scene fusion network is determined by integrating the outputs of the first VR scene fusion network to the MVR scene fusion network, and the VR scene fusion network is obtained.

[0049] The specific configuration of the training feedback module will be described in detail below. The training feedback module may further include: parsing the action characteristics of the puncture catheter posture monitoring timing information to obtain the first puncture site, the first puncture angle, the first puncture depth, and the first puncture time; parsing the action characteristics of the puncture catheter posture target timing information to obtain the second puncture site, the second puncture angle, the second puncture depth, and the second puncture time; comparing the first puncture site, the first puncture angle, the first puncture depth, and the first puncture time with the second puncture site, the second puncture angle, the second puncture depth, and the second puncture time to obtain the puncture deviation vector information.

[0050] Preferably, the puncture and catheterization posture monitoring timing information generated by the user during the actual operation is analyzed to extract key action features, including the first puncture site (the puncture site position data in the user's actual operation), the first puncture angle (the actual puncture angle of the user, such as the insertion angle relative to the skin surface), the first puncture depth (the actual insertion depth of the user, representing the distance the needle enters the tissue), and the first puncture time (the time taken to complete the puncture action, reflecting the speed and rhythm of the operation); then the preset puncture and catheterization posture target timing information is analyzed to extract action features, including the second puncture site (the puncture site required by the standard operation), the second puncture angle (the standard puncture angle, which is the optimal operating angle), the second puncture depth (the standard puncture depth, ensuring that the needle reaches the target position but not too deep or too shallow), and the second puncture time (the standard operating time, used to standardize the puncture speed and rhythm).

[0051] Preferably, the user's actual operation data (first puncture site, angle, depth, time) is compared with the standard target data (second puncture site, angle, depth, time) one by one, and the difference is calculated to quantify the deviation between the user's actual operation and the standard operation, and generate puncture deviation vector information. The deviation vector information may include position deviation (position difference between the user's puncture site relative to the standard site), angle deviation (deviation between the actual puncture angle and the standard angle), depth deviation (error between the user's puncture depth and the standard depth), and time deviation (difference in operation time relative to the standard time), so that users can make corrections and improvements during training to improve the accuracy and safety of radial artery puncture and catheterization.

[0052] The specific configuration of the training feedback module will be described in detail below. The training feedback module may further include: activating a feature extraction model, wherein the feature extraction model is a SLOWFAST neural network; splitting the puncture catheterization posture monitoring timing information according to the first step length to obtain a first puncture catheterization posture sequence; splitting the puncture catheterization posture monitoring timing information according to the second step length to obtain a second puncture catheterization posture sequence, wherein the second step length is one eighth of the first step length; processing the first puncture catheterization posture sequence and the second puncture catheterization posture sequence according to the feature extraction model to obtain the first puncture site, the first puncture angle, the first puncture depth, and the first puncture time.

[0053] Preferably, the feature extraction model is activated to extract key action features from the posture monitoring time series data of radial artery puncture and catheterization training, wherein the feature extraction model is a SLOWFAST neural network, which is a dual-stream neural network for video analysis, particularly suitable for processing data of multiple time steps, and is composed of two parallel network streams, a slow stream and a fast stream, wherein the slow stream processes low-frequency information and is suitable for features that change over a long period of time, while the fast stream processes high-frequency information and focuses on capturing the details of fast actions, and can extract features of posture changes from different time scales, thereby capturing complete action information; the posture monitoring time series information is split according to the first step length (longer time interval) to form a first puncture and catheterization posture sequence, which is mainly used for the slow stream branch to extract slowly changing features in the action, such as the overall trend of the posture or the stability over a longer period of time, and then the same posture monitoring time series information is split according to the second step length (shorter time interval, about one eighth of the first step length) to form a second puncture and catheterization posture sequence, which is used for the fast stream branch to capture the slow changing features in the action. The first puncture and catheterization posture sequence and the second puncture and catheterization posture sequence are processed according to the feature extraction model, that is, the two sequences of different step lengths are processed in parallel using the SLOWFAST network, so that the network can capture the user's puncture and catheterization action features from both global and local time scales. By combining the feature outputs of the slow stream and the fast stream, the SLOWFAST network can generate a high-dimensional description of the posture information and extract the key action features of the user's puncture and catheterization operation, including the first puncture site, the first puncture angle, the first puncture depth, and the first puncture time.

[0054] Based on the foregoing embodiments, the embodiments of the present application further provide an electronic device and a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor of the electronic device, the system described in any of the previous embodiments can be implemented.

[0055] Figure 3 1 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, showing a block diagram of an exemplary electronic device suitable for implementing an implementation of the present invention. Figure 3The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention. The electronic device is in the form of a general computing device, and its components may include but are not limited to an input device 401, a processor 402, a memory 403, and an output device 404. Among them, the processor 402 may be one or more; the memory 403 may include a computer-readable medium and at least one program product, and the program product has a set (at least one) of program modules, which are configured to perform the functions of each embodiment of the present application.

[0056] The memory 403 shown in the embodiment of the present invention may adopt any combination of one or more computer-readable media; the computer-readable storage medium may be, but is not limited to, an infrared, semiconductor system, device or component, or any combination of the above, for storing software programs, computer executable programs and modules, such as the program instructions / modules corresponding to a radial artery puncture and catheterization training system based on a VR platform in the embodiment of the present invention. The processor 402 executes various functional applications and data processing of the computer device by running the software programs, instructions and modules stored in the memory 403, thereby realizing the above-mentioned radial artery puncture and catheterization training system based on a VR platform.

[0057] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules may be used and run on the user terminal and / or server, and the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0058] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A radial artery puncture and catheterization training system based on a VR platform, characterized in that: include: A user terminal, the user terminal is used to select a radial artery puncture and catheterization training scene number; A head-mounted VR wearable terminal, which is worn on the user's head to perform radial artery puncture and catheterization training; VR platforms, including: A 3D modeling module, wherein the 3D modeling module is used to construct a puncture and catheterization training scene model based on a scene database according to the radial artery puncture and catheterization training scene number, and transmit the model to a head-mounted VR wearable terminal through a scene output device to perform radial artery puncture and catheterization training; A puncture posture acquisition module, which is used to collect puncture and catheterization posture monitoring time sequence information when the user is performing radial artery puncture and catheterization training; A training feedback module is used to compare the puncture catheterization posture monitoring timing information with the puncture catheterization posture target timing information to obtain posture deviation vector information, generate voice correction information based on the posture deviation vector information, and send it to the head-mounted VR wearable terminal to remind the user to make corrections.

2. A radial artery puncture and catheterization training system based on a VR platform as claimed in claim 1, characterized in that: The user terminal is embedded in the head-mounted VR wearable terminal.

3. A radial artery puncture and catheterization training system based on a VR platform as claimed in claim 1, characterized in that: According to the radial artery puncture and catheterization training scenario number, a puncture and catheterization training scenario model is constructed based on the scenario database, including: Inputting the radial artery puncture and catheterization training scene number into a feature database to match training features, wherein the training features include model features, medical appliance features, and scene features; Inputting the model features, the medical tool features and the scene features into the scene database, and matching the puncture target three-dimensional model, the medical tool three-dimensional model and the scene three-dimensional model; The puncture target three-dimensional model, the medical device three-dimensional model and the scene three-dimensional model are fused through a VR scene fusion network to generate the puncture catheterization training scene model.

4. A radial artery puncture and catheterization training system based on a VR platform as claimed in claim 3, characterized in that: The VR scene fusion network construction step includes: Collect puncture target three-dimensional model record data, medical appliance three-dimensional model record data, scene three-dimensional model record data and fused three-dimensional model identification data; The fused three-dimensional model identification data is used as supervision data, and the puncture target three-dimensional model recording data, the medical tool three-dimensional model recording data, and the scene three-dimensional model recording data are used as input variables to train a fully convolutional neural network to obtain the VR scene fusion network.

5. A radial artery puncture and catheterization training system based on a VR platform as claimed in claim 4, characterized in that: The fused three-dimensional model identification data is used as supervision data, and the puncture target three-dimensional model record data, the medical appliance three-dimensional model record data, and the scene three-dimensional model record data are used as input variables to train a fully convolutional neural network to obtain the VR scene fusion network, including: The fused three-dimensional model identification data is used as supervision data, and the puncture target three-dimensional model record data, the medical appliance three-dimensional model record data, and the scene three-dimensional model record data are used as input variables to construct a VR scene fusion network construction data set; After dividing the VR scene fusion network construction data set into k groups, performing k extractions with replacement to obtain a first VR scene fusion network construction data set; Constructing a data set according to the first VR scene fusion network, training a fully convolutional neural network, and obtaining a first VR scene fusion network; Cycle M times to obtain the MVR scene fusion network; The first VR scene fusion network is fully connected until the MVR scene fusion network to obtain the VR scene fusion network.

6. A radial artery puncture and catheterization training system based on a VR platform as claimed in claim 5, characterized in that: Fully connecting the first VR scene fusion network until the MVR scene fusion network to obtain the VR scene fusion network, including: Constructing full connection rules: Calculate the pairwise similarities of the first VR scene fusion network to the first output VR scene to the Mth output VR scene of the Mth VR scene fusion network to obtain a plurality of output VR scene similarities; Performing LOF outlier detection according to the similarities of the plurality of output VR scenes to obtain a first outlier factor up to an Mth outlier factor; Output the first outlier factor until the output VR scene corresponding to the minimum value of the Mth outlier factor, and set it as the output value of the VR scene fusion network; According to the full connection rule, the first VR scene fusion network is fully connected until the MVR scene fusion network to obtain the VR scene fusion network.

7. A radial artery puncture and catheterization training system based on a VR platform as claimed in claim 1, characterized in that: The puncture catheterization posture monitoring time sequence information is compared with the puncture catheterization posture target time sequence information to obtain posture deviation vector information, including: Analyze the action characteristics of the puncture catheter placement posture monitoring time series information to obtain a first puncture site, a first puncture angle, a first puncture depth, and a first puncture time; Analyze the action characteristics of the puncture catheter placement posture target time sequence information to obtain a second puncture site, a second puncture angle, a second puncture depth, and a second puncture time; The puncture deviation vector information is obtained by comparing the first puncture site, the first puncture angle, the first puncture depth, and the first puncture time with the second puncture site, the second puncture angle, the second puncture depth, and the second puncture time.

8. A radial artery puncture and catheterization training system based on a VR platform as claimed in claim 7, characterized in that: Analyzing the action characteristics of the puncture catheter posture monitoring time series information to obtain a first puncture site, a first puncture angle, a first puncture depth, and a first puncture time includes: activating a feature extraction model, wherein the feature extraction model is a SLOWFAST neural network; According to the first step, the puncture and catheter placement posture monitoring time sequence information is split to obtain a first puncture and catheter placement posture sequence; Splitting the puncture and catheter placement posture monitoring time sequence information according to the second step length to obtain a second puncture and catheter placement posture sequence, wherein the second step length is one eighth of the first step length; The first puncture catheter placement posture sequence and the second puncture catheter placement posture sequence are processed according to the feature extraction model to obtain the first puncture site, the first puncture angle, the first puncture depth, and the first puncture time.

9. An electronic device, characterized in that: The electronic device comprises: A memory for storing executable instructions; The processor is used to implement the radial artery puncture and catheterization training system based on a VR platform as described in any one of claims 1 to 8 when executing the executable instructions stored in the memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, a radial artery puncture and catheterization training system based on a VR platform is implemented as described in any one of claims 1 to 8.

Images