Dissection teaching system and method based on VR

Abstract

The invention discloses an anatomy teaching system and method based on VR, relates to the technical field of VR anatomy teaching, solves the problem that in anatomy teaching based on the VR technology, it is difficult to observe human anatomy during movement and reveal the complex movement law of the human body, integrates a motion capture system, a computer system and a projector, and improves the teaching efficiency. Firstly, the system utilizes an infrared camera module, a tracking module and a high-precision motion capture device to accurately capture kinematics data of a human body, and anatomical positioning points are positioned one by one and projected according to human bony marker positioning points and anatomical picture resources needing to be displayed; the function not only improves the dynamic expressive force of teaching content, but also ensures the accuracy of anatomical structure display, avoids the limitation that a static model cannot display complex dynamic information in traditional teaching, and secondly, the system can realize projection following the human body position in movement.

Description

Technical Field

[0001] The present invention relates to the technical field of VR anatomy teaching, and in particular to an anatomy teaching system and method based on VR. Background Art

[0002] As a core course in medical education, anatomy requires students to have a deep understanding of the human body structure and the ability to imagine and identify anatomical structures in three-dimensional space. However, the cultivation of this spatial understanding ability is still a big challenge for many students. Traditional anatomy teaching mainly relies on books, two-dimensional images and text descriptions. Although these methods can convey basic information, it is difficult to fully help students transform two-dimensional information into a three-dimensional mental model.

[0003] With the advancement of technology, a variety of technologies and tools to assist in anatomy teaching have appeared on the market. These technologies attempt to solve the difficulties students encounter when learning anatomy by providing more intuitive three-dimensional models and interactive experiences. However, although these technologies have improved the learning experience to a certain extent, they also have certain defects and limitations.

[0004] The defects of the prior art include: Screen: The small screen of mobile devices may limit the observation of details. The large-screen 3D interactive anatomical model is still detached from the real environment and lacks the combination of images and subjective movement feelings.

[0005] VR technology: VR technology may cause motion sickness and other discomfort to some users, and it is often difficult to teach multiple people at the same time.

[0006] AR technology: Existing AR technology often relies on smartphones or tablets, which is often limited by the accuracy of the device's camera and sensors, and is divorced from the actual human body, affecting the user experience.

[0007] The main disadvantages of the above-mentioned prior art are as follows: it is difficult to observe human anatomy and reveal the complex movement patterns of the human body during movement. Summary of the invention

[0008] In view of the deficiencies in the prior art, the present invention provides a VR-based anatomical teaching system and method, which solves the problem that it is difficult to observe the purpose of human anatomy in motion and reveal the complex movement laws of the human body in anatomical teaching based on VR technology.

[0009] To achieve the above objectives, the present invention is implemented through the following technical solutions: The VR-based anatomy teaching system of the present invention comprises: A motion capture system, used to collect and capture the user's movements or the position of the model. The motion capture system includes an infrared camera module, a tracking module and a motion capture device; A computer system, used for processing data received from the motion capture module through a high-performance computer, performing two key computing tasks, and rendering a three-dimensional image and controlling a projector. After receiving a video signal, the real-time projection projector projects a real-time image onto a target area of ​​the human body to achieve dynamic visualization of human anatomy, wherein the computer system includes a software platform and a three-dimensional anatomy model database; Projector: The projector integrates tracking sensors, motion capture devices and projection components. It identifies and tracks human targets through tracking sensors, obtains human kinematic information through motion capture devices, and processes data with computers to identify joint movement angles and call related image or video resources. Finally, the image is projected into the target area of ​​the human body in real time through the projector.

[0010] Preferably, in the motion capture system, the collection method of the infrared camera module, the tracking module and the motion capture device is: Tracking module: Use tracking sensors to identify human-shaped objects and track their positions, and use computer vision libraries to implement image recognition in tracking sensors and tracking of human body shapes and movements; Motion capture equipment: Motion capture equipment is equipped with high-precision sensors, specially designed to capture the human body's kinematic data in detail, including joint angles and limb positioning; data transmission technology quickly and accurately transmits large amounts of motion data to subsequent computer systems; data processing and analysis involves software and hardware to make necessary adjustments to the raw signal, calculate the motion path, and combine the data with the three-dimensional model; Infrared camera module: connect the 3D infrared camera to the computer, use the 3D infrared camera to capture the 3D model of the human body, and import the image data into the computer; connect the infrared sensors to the computer and attach them to the joints of the human body so that the computer can detect these sensors; use the 3D infrared camera again to capture the human body model, compare it with the previously imported model, and mark the joint positions; use the computer to generate 3D coordinates; map the measured 3D position to the corresponding image of the human body model, combine the preset joint point position information and the actual captured joint point position information, track the joint points, and finally obtain the motion trajectory of each joint point in the video.

[0011] Preferably, in the computer system: The software platform, as a central control unit, integrates and coordinates the functions of all hardware components to control motion capture, image rendering, and projection control; The 3D anatomical model database contains detailed 3D models of human anatomical structures for display in projections.

[0012] Preferably, in the computer system, the two key computing tasks are: The first is to clean and filter the captured data, remove noise, obtain and calculate the joint motion angle, and provide accurate motion parameters for subsequent image rendering. The programming language is combined with relevant mathematical libraries to achieve data cleaning, filtering and calculation of joint motion angles; EasyTracker: It uses the SMK series of intelligent calculation dedicated cameras to directly calculate the two-dimensional coordinates of the marking points; the SMK series of intelligent calculation dedicated cameras support real-time online or offline motion capture and analysis, and have high-performance automatic recognition functions. It automatically matches the marking points according to the template and calculates and outputs the spatial position and degree of freedom data of people or objects in real time; Secondly, based on the captured bony landmarks and joint positions, three-dimensional modeling technology is used to create three-dimensional models of human body parts according to the captured data; based on the captured bony landmarks and joint position information, the three-dimensional anatomical model database is queried, and the corresponding image or video resources are retrieved from the three-dimensional anatomical model database, and a relational database or a non-relational database is used to store and retrieve the images and video resources in the three-dimensional anatomical model database; the three-dimensional model is rendered and images are generated to display the human anatomical structure; its dedicated software platform integrates user interface, data processing, image rendering and projection control functions, allowing users to intuitively operate and control the projection process.

[0013] Preferably, in the computer system, the image rendering process is: The computer system performs real-time rendering based on the joint motion angles and retrieved anatomical structures to generate images synchronized with human body movements; a professional 3D graphics library is used to create and render 3D models of human body parts; at the same time, the development interface of the 3D modeling software is used in combination with the corresponding rendering engine to generate high-quality images.

[0014] Preferably, it also includes: The video output interface uses HDMI, DVI or other high-speed video output interfaces to transmit the rendered image from the computer processing module to the projector; after receiving the video signal, the real-time projection projector projects the real-time image onto the target area of ​​the human body to achieve dynamic visualization of human anatomy; users can select different anatomical parts, adjust the viewing angle and zoom ratio through the interactive interface to achieve a personalized learning experience, and use a cross-platform UI framework to create an intuitive user interaction interface.

[0015] Preferably, the projector integrates three core components: a tracking sensor, a motion capture device and a projection component. The projector has three main functions: human tracking, kinematic information acquisition and real-time projection. The implementation method is that the tracking sensor identifies the human-shaped object and tracks its position, the motion capture device collects the kinematic information of the human body, and the relative position relationship of the relevant structures of the human body is determined by capturing the bony landmarks. After computer processing, the projector finally projects the real-time image to the target area of ​​the human body.

[0016] Preferably, it also includes: Kinematic information is transmitted to the computer system via wired or wireless means. The computer processing system needs to complete two main calculation tasks. After the operator determines the displayed part, the first is to identify the joint movement angle in real time, and the second is to retrieve relevant static and dynamic images or video resources from the three-dimensional anatomical model database according to the bony landmarks and corresponding joints, load them into the computer system through the software platform for rendering and projection, and connect to the projector through HDMI, DVI or other video output interfaces to project them onto the human body.

[0017] The VR-based anatomy teaching method of the present invention comprises the following specific steps: Step 1: Use the all-in-one dual-purpose human tracking projector to start locating and capturing the human body; Step 2: Use the tracking module to perform rough positioning through tracking sensors and corresponding algorithms, and start the infrared camera module to perform motion capture to achieve fine positioning; Step 3: The signal converter converts the optical signal into an electrical signal and transmits it to the computer. The computer processes the received signal, eliminates noise, filters out errors, confirms the position of the joint point, builds a three-dimensional model, and analyzes the angle of the joint point. Step 4: Store the confirmed joint point data into the 3D anatomical model database for subsequent use; Step 5: Use the rendering algorithm to generate the corresponding three-dimensional image on the computer. The electrical signal is converted into a video signal through another signal converter, and the output is controlled by the software platform and projected.

[0018] The beneficial effects of the present invention are as follows: The present invention integrates a motion capture system, a computer system and a projector. First, the system uses an infrared camera module, a tracking module and a high-precision motion capture device to accurately capture the kinematic data of the human body, and locates and projects the anatomical positioning points one-to-one according to the human body's bony landmark positioning points and the anatomical image resources that need to be displayed; this function not only improves the dynamic expressiveness of the teaching content, but also ensures the accuracy of the anatomical structure display, and avoids the limitation that static models in traditional teaching cannot display complex dynamic information. Secondly, the system processes and renders the captured data through a high-performance computer, and can realize projection that follows the position of the human body during movement, and the image can change in real time according to the joint angle measurement, generating a realistic three-dimensional anatomical image, and projecting it to the target area of ​​the human body in real time through a projector; this real-time dynamic projection function enables students to better understand the changes in the human body structure under different motion states while observing the anatomical structure, thereby enhancing the intuitiveness and memory effect of learning, and avoiding the impact of head-mounted tools on the body. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a structural framework diagram of the VR-based anatomy teaching system of the present invention.

[0020] Figure 2 This is a flow chart of the VR-based anatomy teaching method of the present invention. DETAILED DESCRIPTION

[0021] Example 1 like Figure 1 As shown, the VR-based anatomy teaching system includes: A motion capture system, used to collect and capture the user's movements or the position of the model. The motion capture system includes an infrared camera module, a tracking module and a motion capture device; The acquisition method of the infrared camera module, tracking module and motion capture device is: Tracking module: Use tracking sensors to identify human-shaped objects and track their positions, and use computer vision libraries such as OpenCV to implement image recognition in tracking sensors and tracking of human body shapes and movements; Motion capture equipment: This equipment is equipped with high-precision sensors and is specially designed to capture the human body's kinematic data in detail, including the angles of joints and the positioning of limbs; data transmission technology quickly and accurately transmits this large amount of motion data to the subsequent computer system; the processing and analysis of the data involves software and hardware to make necessary adjustments to the raw signal, calculate the motion path, and combine the data with the three-dimensional model to create smooth and natural animation effects; the entire process relies on the fast processing power of the computer system to ensure the accuracy of data processing and the authenticity of animation performance; Specifically, the data collected by high-precision sensors is converted into human motion data as follows: S1: Data collection and preprocessing Choose the right sensor: According to the research purpose and needs, choose sensors that can measure physical quantities related to human motion, such as accelerometers, gyroscopes, magnetometers, pressure sensors, etc.; ensure that the accuracy, sampling frequency and dynamic range of the sensor meet the requirements; Sensor installation: Install the sensor correctly on the corresponding part of the human body, ensuring that the sensor is firmly fixed to the human body and does not affect the normal movement of the human body; for example, accelerometers and gyroscopes can be installed at key joints of the body, such as wrists, ankles, waist, etc.; Data collection: Use sensor equipment to collect data and record sensor data during human motion. The collection time should be long enough to cover different types of motion and actions. At the same time, ensure that the collected data has a sufficient sampling frequency to accurately reflect the details of human motion. Data filtering: The data collected by the sensor may be interfered by noise, so filtering is required; commonly used filtering methods include low-pass filtering, high-pass filtering, band-pass filtering, etc., which can remove noise in different frequency ranges; in this embodiment, low-pass filtering can remove high-frequency noise and retain the low-frequency components of human body movement; Data calibration: Calibrate the sensor to ensure that the collected data is accurate and reliable; the calibration process includes zero calibration, sensitivity calibration, etc.; for example, zero calibration can eliminate the output error of the sensor in a static state; S2: Kinematic parameter calculation Position and displacement calculation: By integrating the data collected by the accelerometer twice, the position and displacement information of the object can be obtained; in human motion, the position and displacement of the joints can be calculated by installing accelerometers in different parts of the body; for example, by installing an accelerometer on the wrist, the position and displacement changes of the wrist can be calculated; Speed ​​and acceleration calculation: By performing one and two differentiations on the position data, we can get speed and acceleration information. The gyroscope can directly measure the angular velocity, and the angle information can be obtained by integration. For example, by calculating the speed and acceleration of the wrist, we can understand the speed and change trend of the wrist movement. The relationship between velocity and acceleration is: ; The relationship between position and velocity is: ; Joint angle calculation: The joint angle can be calculated by sensors installed on both sides of the joint (such as accelerometers, gyroscopes, etc.); commonly used methods include Euler angle method, quaternion method, etc.; for example, the joint angle between the arm and the shoulder can be calculated by sensors installed on the arm and shoulder; Among them, the Euler angle method: usually uses three angles (yaw angle, pitch angle, roll angle) to describe the posture of an object; the angular velocity data collected by the gyroscope can be integrated to obtain the angle change, and then combined with the initial angle to obtain the current Euler angle; Quaternion method: , w, x, y, z are the four components of the quaternion; Muscle activity analysis: The muscle electrical activity signals collected by the electromyography sensor can be processed and analyzed to obtain parameters such as muscle activation degree and contraction time, so as to understand the role of muscles in movement; for example, by analyzing the electromyography signals of leg muscles, the degree of involvement of different muscles during walking can be understood; S3: Data Analysis and Visualization Data analysis: Analyze the calculated kinematic parameters to extract useful information; for example, analyze the range of motion, velocity, acceleration and other parameters of the joints to evaluate the performance and injury risk of human movement; compare the differences in kinematic parameters between different individuals or under different movement conditions to study the laws and characteristics of human movement; Visualization: Present kinematic data in a visual way, such as drawing charts and animations, so as to understand the characteristics of human movement more intuitively; for example, 3D animation software can be used to visualize the human movement process, or to draw a curve of joint angle changes over time, to help researchers better understand the process of human movement; S4: Verification and Evaluation Data verification: Verify the calculated kinematic parameters to ensure the accuracy and reliability of the data; compare with other measurement methods (such as optical motion capture systems, force sensors, etc.) or conduct repeatable experiments to verify the accuracy of the conversion of sensor data into kinematic data; Evaluation method: Establish an evaluation index system to evaluate human kinematics data; for example, the standard deviation and root mean square error of kinematic parameters can be used to evaluate the stability and accuracy of data; the quality and efficiency of human movement can also be understood by evaluating the symmetry and coordination of human movement. Infrared camera module: connect the 3D infrared camera to the computer, use the 3D infrared camera to capture the 3D model of the human body, and import the image data into the computer; connect the infrared sensors to the computer and stick them to the joints of the human body so that the computer can detect these sensors; use the 3D infrared camera to capture the human body model again, compare it with the previously imported model, and mark the joint positions; use the computer to generate 3D coordinates; map the measured 3D position to the corresponding image of the human body model, combine the preset joint point position information and the actual captured joint point position information, track the joint points, and finally obtain the movement trajectory of each joint point in the video; this process can accurately capture and record detailed data of human body movement; tracking sensor: using advanced image recognition technology, the tracking sensor can identify and track the human body shape and movement to ensure that the projected content matches the actual position of the human body; It should be noted that the previously imported human body models come from a variety of sources. One common source is to use professional 3D modeling software, such as Maya, 3ds Max or Blender, to create 3D models of the human body. These software can build accurate human body models by manual modeling or using scanned data. Another source is to obtain existing standard human body models from medical databases or anatomical resources. For example, some medical research institutions or organizations may provide human body models that have been accurately measured and annotated for use in related applications. In actual cases, in the field of medical education, some anatomical software uses human body models obtained from professional databases, which have been detailedly annotated and verified. For example, the human body models used by software such as Visible Body are created based on medical image data and anatomical knowledge. The specific method of comparing the marked joint positions is: AS1: Use infrared cameras or other sensors to capture images or data of the human body; AS2: Preprocess the captured images, including noise removal, contrast enhancement, etc.; AS3: Use image recognition algorithms to detect the contours and key features of the human body, such as joints; AS4: Match and compare the detected joint positions with the previously imported model; AS5: Mark and record the joint positions based on the matching results; It should be noted that in actual applications, in order to more accurately adapt to people of different body shapes, some methods can be used to deal with this differentiation; for example, using adjustable parameters or models to adjust the display of anatomical structures and the calculation of kinematic data according to human body measurement data (such as height, weight, body fat percentage, etc.); or using machine learning algorithms to train a large amount of human body data of different body shapes to generate a model that can adapt to individual differences; in actual cases, some advanced motion capture systems use multiple sensors to obtain detailed data of the human body, and automatically adapt to people of different body shapes through algorithms; for example, in virtual reality games, full-body motion capture equipment can be used to capture the player's body movements, and real-time animation rendering can be performed according to the player's body shape and movement characteristics to provide a more realistic experience; The way the computer generates 3D coordinates is: By using multiple sensors or cameras to capture information about the human body from different angles, the coordinates of the object in three-dimensional space are calculated through triangulation or other related algorithms; for example, a depth camera can be used to obtain the depth information of an object, and the 3D coordinates of the object can be calculated by combining the position and posture data of the camera; in actual cases, in augmented reality (AR) or virtual reality (VR) applications, multiple cameras or depth sensors are usually used to capture the user's body movements and positions, and then the user's 3D coordinates in the virtual space are generated through real-time calculation; for example, Microsoft's Kinect sensor is a device that can capture the 3D coordinates of the human body, which can be used in games, sports training and other applications; The process of mapping the acquired 3D coordinates to the previous human body model usually involves coordinate transformation and registration algorithms. First, the coordinate system of the previous human body model and the coordinate system of the acquired 3D coordinates need to be determined. Then, the acquired coordinates are converted to the coordinate system of the model by calculating the coordinate transformation matrix. Finally, the converted coordinates are matched and aligned with the model using the registration algorithm to determine information such as joint positions. In medical image analysis of actual cases, it is often necessary to register the patient's medical images (such as CT and MRI scans) with standard human body models for disease diagnosis and treatment planning. For example, the registration and coordinate mapping of medical images can be achieved using open source libraries such as ITK (Insight Segmentation and Registration Toolkit). Specifically, the motion capture system is a high-tech device used to accurately measure the motion of a moving object in three-dimensional space. It is based on the principle of computer graphics and records the motion of a moving object in the form of an image through several motion capture devices arranged in space. The image data is then processed by a computer to obtain the spatial coordinates of different tracking sensors in different time units. From a technical perspective, the essence of motion capture is to measure, track, and record the motion trajectory of an object in three-dimensional space. A computer system, used for processing data received from the motion capture module through a high-performance computer, performing two key computing tasks, and rendering a three-dimensional image and controlling a projector. After receiving a video signal, the real-time projection projector projects a real-time image onto a target area of ​​the human body to achieve dynamic visualization of human anatomy, wherein the computer system includes a software platform and a three-dimensional anatomy model database; The software platform, which acts as a central control unit, integrates and coordinates the functions of all hardware components to control motion capture, image rendering, and projection control; 3D anatomical model database, containing detailed 3D models of human anatomical structures for display in projections; It should be noted that the 3D anatomical model database is an existing medical database; The two key computational tasks are: First, it cleans and filters the captured data, removes noise, obtains and calculates joint motion angles, and provides accurate motion parameters for subsequent image rendering. Programming languages ​​such as Python are combined with relevant mathematical libraries (such as NumPy and SciPy) to achieve data cleaning, filtering and calculation of joint motion angles; EasyTracker: Adopts SMK series of intelligent calculation dedicated cameras to directly calculate the two-dimensional coordinates of the marker points; SMK series of intelligent calculation dedicated cameras support real-time online or offline motion capture and analysis, and have high-performance automatic recognition functions. It can automatically match marker points according to templates, and calculate and output the spatial position and degree of freedom data of people or objects in real time. At the same time, the software platform also has powerful editing functions such as automatic interpolation, filtering and manual modification of post-data, as well as general data format conversion functions, which can fully support the combination of third-party software and use interfaces; The specific algorithm for calculating the joint motion angle is to use a kinematic analysis algorithm, combined with the joint position data and time information collected by the sensor, to calculate the angle change of the joint; for example, Euler angles or quaternions can be used to represent the rotation angle of the joint, and the rate of change of the angle can be obtained by differential or integral calculation; in actual cases, in sports science research, motion capture systems are often used to collect athletes' joint motion data, and special software is used to calculate the joint motion angle; for example, using motion capture systems such as OptiTrack combined with related analysis software can calculate and display the changes in joint motion angles in real time; Secondly, based on the captured bony landmarks and joint positions, 3D modeling technology is used to create 3D models of human body parts based on the captured data; based on the captured bony landmarks and joint position information, the 3D anatomical model database is queried, and the corresponding image or video resources are retrieved from the 3D anatomical model database, and a relational database (such as MySQL, Oracle) or a non-relational database (such as MongoDB) is used to store and retrieve images and video resources in the 3D anatomical model database; 3D models are rendered and images are generated to display the anatomical structure of the human body; its dedicated software platform integrates functions such as user interface, data processing, image rendering and projection control, allowing users to intuitively operate and control the projection process; Among them, the specific algorithm for creating a three-dimensional model of human body parts is: The specific algorithm for creating a 3D model of a human body part can use a variety of 3D modeling techniques, such as polygonal modeling, surface modeling, or voxel modeling. Usually, it is necessary to construct the geometric shape and structure of the human body part based on human anatomical knowledge and data. In actual cases: in the field of medical visualization, professional 3D modeling software such as Maya or 3dsMax can be used to create accurate 3D models of human body parts by manual modeling or using scanned data. For example, when creating a heart model, the shape of the heart, blood vessel distribution and other details can be constructed based on medical image data and anatomical knowledge. The image rendering process is as follows: the computer system performs real-time rendering based on the joint motion angles and retrieved anatomical structures to generate images synchronized with human body motion; professional 3D graphics libraries such as OpenGL or DirectX can be used to create and render 3D models of human body parts; at the same time, the development interface of 3D modeling software (such as Blender, Maya, etc.) is combined with the corresponding rendering engine (such as V-Ray, Arnold, etc.) to generate high-quality images; Finally, the video output interface uses HDMI, DVI or other high-speed video output interfaces to transmit the rendered image from the computer processing module to the projector; after receiving the video signal, the real-time projection projector projects the real-time image onto the target area of ​​the human body to achieve dynamic visualization of human anatomy; users can select different anatomical parts, adjust the viewing angle and zoom ratio through the interactive interface to achieve a personalized learning experience, and use cross-platform UI frameworks (such as Qt, GTK+) to create intuitive user interaction interfaces for easy operation and control; system synchronization ensures that each link such as motion capture, data processing, image rendering and projection display is synchronized to avoid delays; Projector: The projector integrates tracking sensors, motion capture equipment and projection components. The tracking sensor identifies and tracks the human target. The motion capture equipment obtains human kinematic information. The computer processes the data, identifies the joint movement angle and calls related image or video resources. Finally, the projector projects the image to the target area of ​​the human body in real time. Specifically, the projector integrates three core components: tracking sensors, motion capture equipment and projection components. Therefore, the projector has three main functions: human tracking, kinematic information acquisition and real-time projection. The implementation method is that the tracking sensor identifies the human object and tracks its position, the motion capture equipment collects the human body's kinematic information, and determines the relative position relationship of the relevant structures of the human body by capturing the bony landmarks. After computer processing, the projector finally projects the real-time image to the target area of ​​the human body. Among them, kinematic information is transmitted to the computer system via wired or wireless means. The computer processing system needs to complete two main calculation tasks. After the operator determines the displayed part, the first is to identify the joint movement angle in real time, and the second is to retrieve relevant static and dynamic pictures or video resources from the three-dimensional anatomical model database according to the bony landmarks and corresponding joints, load them into the computer system through the software platform, render and project them, and connect them to the projector through HDMI, DVI or other video output interfaces to project them onto the human body.

[0022] like Figure 2 As shown in the figure, the steps of the VR-based anatomy teaching method are as follows: Step 1: Use the all-in-one dual-purpose human tracking projector to start locating and capturing the human body; Step 2: Use the tracking module to perform rough positioning through tracking sensors and corresponding algorithms, and start the infrared camera module to perform motion capture to achieve fine positioning; Step 3: The signal converter converts the optical signal into an electrical signal and transmits it to the computer. The computer processes the received signal, eliminates noise, filters out errors, confirms the position of the joint point, builds a three-dimensional model, and analyzes the angle of the joint point. Step 4: Store the confirmed joint point data into the 3D anatomical model database for subsequent use; Step 5: Use the rendering algorithm to generate the corresponding three-dimensional image on the computer. The electrical signal is converted into a video signal through another signal converter, and the output is controlled by the software platform and projected.

Claims

1. A VR-based anatomy teaching system. It is characterized in that include: A motion capture system, used to collect and capture the user's movements or the position of the model. The motion capture system includes an infrared camera module, a tracking module and a motion capture device; A computer system, used for processing data received from the motion capture module through a high-performance computer, performing two key computing tasks, and rendering a three-dimensional image and controlling a projector. After receiving a video signal, the real-time projection projector projects a real-time image onto a target area of ​​the human body to achieve dynamic visualization of human anatomy, wherein the computer system includes a software platform and a three-dimensional anatomy model database; Projector: The projector integrates tracking sensors, motion capture devices and projection components. It identifies and tracks human targets through tracking sensors, obtains human kinematic information through motion capture devices, and processes data with computers to identify joint movement angles and call related image or video resources. Finally, the image is projected into the target area of ​​the human body in real time through the projector.

2. The VR-based anatomy teaching system according to claim 1, It is characterized in that In the motion capture system, the acquisition method of the infrared camera module, tracking module and motion capture device is: Tracking module: Use tracking sensors to identify human-shaped objects and track their positions, and use computer vision libraries to implement image recognition in tracking sensors and tracking of human body shapes and movements; Motion capture equipment: Motion capture equipment is equipped with high-precision sensors to capture the human body's kinematic data in detail, including joint angles and limb positioning; the motion data is transmitted to subsequent computer systems through data transmission; the processing and analysis of the data involves software and hardware to make necessary adjustments to the raw signal, calculate the motion path, and combine the data with the three-dimensional model; Infrared camera module: connect the 3D infrared camera to the computer, use the 3D infrared camera to capture the 3D model of the human body, and import these image data into the computer; Connect infrared sensors to the computer and stick them to the joints of the human body so that the computer can detect them; use the 3D infrared camera again to capture the human body model and compare it with the previously imported model to mark the joint positions; The 3D coordinates are generated by a computer; the measured 3D position is mapped to the corresponding image of the human body model, and the joints are tracked by combining the preset joint position information and the actually captured joint position information, and finally the motion trajectory of each joint in the video is obtained.

3. The VR-based anatomy teaching system according to claim 1, It is characterized in that In a computer system: The software platform, which acts as a central control unit, integrates and coordinates the functions of all hardware components to control motion capture, image rendering, and projection control; The 3D anatomical model database contains 3D models of human anatomical structures for display in projections.

4. The VR-based anatomy teaching system according to claim 1, It is characterized in that In a computer system, two key computing tasks are: The first is to clean and filter the captured data, remove noise, obtain and calculate the joint motion angle, and provide accurate motion parameters for subsequent image rendering. The programming language is combined with relevant mathematical libraries to achieve data cleaning, filtering and calculation of joint motion angles; EasyTracker: It uses the SMK series of intelligent calculation dedicated cameras to directly calculate the two-dimensional coordinates of the marking points; the SMK series of intelligent calculation dedicated cameras support real-time online or offline motion capture and analysis, and have high-performance automatic recognition functions. It automatically matches the marking points according to the template and calculates and outputs the spatial position and degree of freedom data of people or objects in real time; Second, based on the captured bony landmarks and joint positions, using 3D modeling technology, create a 3D model of the human body part based on the captured data; based on the captured bony landmarks and joint position information, query the 3D anatomical model database, and retrieve the corresponding image or video resources from the 3D anatomical model database, and use a relational database or a non-relational database to store and retrieve the image and video resources in the 3D anatomical model database; Render 3D models and generate images to illustrate human anatomy; Its dedicated software platform integrates user interface, data processing, image rendering and projection control functions, allowing users to intuitively operate and control the projection process.

5. The VR-based anatomy teaching system according to claim 1, It is characterized in that In a computer system, the image rendering process is: The computer system performs real-time rendering based on the joint motion angles and retrieved anatomical structures to generate images synchronized with human body movements; the 3D graphics library is used to create and render 3D models of human body parts; at the same time, the development interface of the 3D modeling software is combined with the corresponding rendering engine to generate high-quality images.

6. The VR-based anatomy teaching system according to claim 5, It is characterized in that Also includes: The video output interface uses HDMI, DVI or other high-speed video output interfaces to transmit the rendered image from the computer processing module to the projector; After receiving the video signal, the real-time projection projector projects the real-time image onto the target area of ​​the human body to achieve dynamic visualization of human anatomy; users can select different anatomical parts, adjust the viewing angle and zoom ratio through the interactive interface to achieve a personalized learning experience, and use a cross-platform UI framework to create an intuitive user interaction interface.

7. The VR-based anatomy teaching system according to claim 1, It is characterized in that The projector integrates three core components: tracking sensors, motion capture devices and projection components. The projector has three main functions: human tracking, kinematic information acquisition and real-time projection. The implementation method is that the tracking sensor identifies the human-shaped object and tracks its position, the motion capture device collects the human body's kinematic information, and determines the relative position relationship of related structures of the human body by capturing bony landmarks. After computer processing, the projector finally projects the real-time image to the target area of ​​the human body.

8. The VR-based anatomy teaching system according to claim 7, It is characterized in that Also includes: Kinematic information is transmitted to the computer system via wired or wireless means. The computer processing system needs to complete two calculation tasks. After the operator determines the displayed part, the first is to identify the joint movement angle in real time, and the second is to retrieve relevant static and dynamic images or video resources from the three-dimensional anatomical model database according to the bony landmarks and corresponding joints, load them into the computer system through the software platform for rendering and projection, and connect to the projector through HDMI, DVI or other video output interfaces to project them onto the human body.

9. A VR-based anatomy teaching method. It is characterized in that The method is implemented by the VR-based anatomy teaching system according to any one of claims 1 to 8, and the specific steps are: Step 1: Use the all-in-one dual-purpose human tracking projector to start locating and capturing the human body; Step 2: Use the tracking module to perform rough positioning through tracking sensors and corresponding algorithms, and start the infrared camera module to perform motion capture to achieve fine positioning; Step 3: The signal converter converts the optical signal into an electrical signal and transmits it to the computer. The computer processes the received signal, eliminates noise, filters out errors, confirms the position of the joint point, builds a three-dimensional model, and analyzes the angle of the joint point. Step 4: Store the confirmed joint point data into the 3D anatomical model database for subsequent use; Step 5: Use the rendering algorithm to generate the corresponding three-dimensional image on the computer. The electrical signal is converted into a video signal through another signal converter, and the output is controlled by the software platform and projected.

Images