Method, device, medium and program product for implementing human acupoint teaching

Abstract

This invention provides a method, apparatus, medium, and program product for teaching acupoints on the human body, relating to the field of augmented reality technology. The method includes: using an augmented reality head-mounted display device worn by a user to identify a real human body model via a depth camera; superimposing a pre-constructed digital human body model onto the real human body model and spatially aligning it so that the virtual acupoints of the digital human body model coincide with the real acupoints of the real human body model; calculating the distance between the user's index fingertip and any virtual acupoint by capturing the user's hand movements in real time; and displaying acupoint teaching information for the virtual acupoint through a UI information panel when the distance is less than a predetermined distance threshold. This invention solves the problem that existing technologies cannot simultaneously satisfy realistic tactile sensation and real-time interactive teaching feedback, resulting in poor teaching effectiveness.

Description

Technical Field

[0001] This invention relates to the field of augmented reality technology, and in particular to a method, apparatus, medium, and program product for teaching acupoints on the human body. Background Technology

[0002] Currently, the following two techniques are mainly used in the teaching and practical training of meridians and acupoints: One type is the traditional physical acupuncture model, such as acupuncture bronze figures or silicone models. These models typically have fixed meridian lines and acupoints drawn or marked on the body surface, providing a realistic physical feel, allowing students to perceive the location of acupoints by touch. However, traditional physical models have extremely limited information capacity, only displaying the location and name of acupoints. They cannot provide real-time recognition and feedback on students' acupoint selection actions, nor can they intuitively demonstrate the therapeutic effects of acupoints, the depth of needling, or other rich teaching information, resulting in poor interactivity.

[0003] Another type is teaching software based on digital technology, such as mobile apps and PC-based 3D anatomy software. This type of software is rich in information, dynamically displaying acupoint locations and related knowledge, and providing interactive feedback through clicks and other operations. However, its display medium is usually a two-dimensional flat screen, meaning users cannot simultaneously experience real human tactile sensations while learning. Their gaze needs to frequently switch between the screen and the object being manipulated, making it difficult to develop accurate spatial intuition.

[0004] In summary, existing technologies cannot simultaneously meet the needs of realistic tactile sensation and real-time interactive teaching feedback, making it difficult for students to effectively combine virtual information with physical operation during acupoint teaching, resulting in poor teaching effectiveness. Summary of the Invention

[0005] The embodiments of the present invention provide a method, device, medium and program product for teaching human acupoints, aiming to solve the problem that the existing technology cannot simultaneously satisfy the requirements of realistic touch and real-time interactive teaching feedback, resulting in poor teaching effect.

[0006] To achieve the above objectives, in a first aspect, the present invention provides a method for teaching acupoints, comprising the following steps: After a user puts on an augmented reality head-mounted display device, a pre-set real human body model is identified by a depth camera set on the augmented reality head-mounted display device; the real human body model is identified by extracting geometric feature points of the real human body model or by auxiliary positioning marks pre-set on the base of the real human body model. The pre-built digital human body model and its included acupoints are superimposed and rendered onto the surface of the real human body model, so that the virtual acupoints of the digital human body model coincide with the coordinates of the real acupoints of the real human body model. The system captures the user's hand movements using a depth camera and sensors, and outputs the real-time coordinates of the user's index fingertip based on the hand movements using a pre-built hand skeletal model. It also calculates the distance between the index fingertip coordinates and the coordinates of each virtual acupoint. When the distance between the index fingertip coordinate and the coordinate of a virtual acupoint is less than a predetermined distance threshold, the acupoint teaching information is displayed through the UI information panel.

[0007] Furthermore, the real human body model is made of medical-grade silicone material with a Shore hardness of A10-20.

[0008] Furthermore, the step of overlaying and rendering the pre-constructed digital human body model and its associated acupoint coordinates onto the surface of the real human body model, so that the virtual acupoints of the digital human body model coincide with the real acupoint coordinates of the real human body model, includes: In response to the user's gesture drag, the digital human body model is moved to a predetermined range of the real human body model to complete the initial alignment; In response to the user's fine-tuning control interface, the translation and rotation angles of the human digital model along the X, Y, and Z axes are adjusted in steps until the virtual acupoints of the human digital model completely coincide with the surface markers of the real acupoints of the real human model, thereby obtaining the transformation matrix of the human digital model relative to the real human model; the fine-tuning refers to adjusting the human digital model by less than a predetermined adjustment range. Create a world anchor point at the currently perfectly overlapping physical location, and bind the transformation matrix to the world anchor point.

[0009] Furthermore, the step-by-step adjustment of the translation and rotation angle of the human digital model along the X, Y, and Z axes is as follows: the step value of the translation is 1 mm, and the step value of the rotation angle is 0.5 degrees.

[0010] Furthermore, the predetermined distance threshold is 5 millimeters.

[0011] Furthermore, the acupoint teaching information includes the acupoint name, functions and indications, anatomical layers, acupuncture depth, needle insertion angle, and / or teaching videos.

[0012] Secondly, the present invention provides an apparatus for teaching acupoints on the human body, comprising a memory and a processor, wherein the memory stores at least one program, and the at least one program is executed by the processor to implement the method for teaching acupoints on the human body as described above.

[0013] Thirdly, the present invention provides a computer-readable storage medium storing at least one program, which is executed by a processor to implement the method for teaching acupoints as described above.

[0014] Fourthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method for teaching acupoints as described above.

[0015] The above technical solution has the following technical effects: This invention solves the problem of poor teaching effectiveness caused by existing technologies that cannot simultaneously satisfy realistic tactile sensation and real-time interactive teaching feedback. It involves a user wearing an augmented reality head-mounted display device, using a depth camera to identify a real human body model; superimposing a pre-built digital human body model onto the real human body model and spatially aligning it so that the virtual acupoints on the digital human body model coincide with the real acupoints on the real human body model; and calculating the distance between the user's index fingertip and any virtual acupoint by capturing the user's hand movements in real time. When this distance is less than a predetermined threshold, the acupoint's teaching information is displayed on a UI information panel. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a method for teaching acupoints in the human body according to an embodiment of the present invention.

[0017] Figure 2 This is a schematic diagram illustrating the principle of spatial alignment between a digital human body model and a real human body model in one embodiment of the present invention.

[0018] Figure 3 This is a schematic diagram of acupoint teaching information prompts in one embodiment of the present invention.

[0019] Figure 4 This is a schematic diagram of a device for teaching acupoints in the human body according to an embodiment of the present invention. Detailed Implementation

[0020] To further illustrate the various embodiments, the present invention provides accompanying drawings. These drawings are part of the disclosure of the present invention, primarily used to illustrate the embodiments and to explain the operating principles of the embodiments in conjunction with the relevant descriptions in the specification. With reference to these drawings, those skilled in the art should be able to understand other possible implementations and the advantages of the present invention. Components in the drawings are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0021] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments.

[0022] Example 1: Figure 1 This is a flowchart illustrating a method for teaching acupoints according to an embodiment of the present invention, as shown below. Figure 1 As shown, the method of this embodiment includes the following steps: After a user puts on an augmented reality (AR) head-mounted display, a depth camera on the device identifies a pre-set real human model. In one specific implementation, the AR head-mounted display uses HoloLens 2 glasses, but other head-mounted displays with depth sensing and gesture tracking capabilities can also be used. The device has a built-in depth camera and inertial measurement unit (IMU). Upon startup, the depth camera performs simultaneous localization and mapping (SLAM) of the environment, scanning the surroundings. When the user aligns the device's field of view with the pre-placed real human model, the system uses depth data collected by the depth camera, combined with an object recognition algorithm, to scan the physical silicone model in the field of view, identifying its spatial position and outline. The object recognition algorithm identifies the physical model by extracting geometric feature points such as facial features, nipples, and navels, or by recognizing auxiliary positioning markers pre-placed on the model's base.

[0023] In this embodiment, the real human body model is a digital model constructed based on a standard digital twin module, and a medical silicone solid model made using a 1:1 mold manufacturing process. Specifically, a high-precision 3D human body mesh model is constructed using 3D scanning or modeling software, and N key acupoints P(x, y, z) are marked on the digital model, defining meridian paths; then, a negative mold is made using 3D printing technology, and a silicone material with a Shore A hardness close to that of human skin is used for casting to create a 1:1 solid model. The Shore A hardness is 10-20 degrees, giving the solid model a tactile feel similar to a real human body, and its geometry is completely consistent with the digital model, with errors controlled within millimeters. In one specific implementation, the real human body model is 160cm tall.

[0024] Figure 2 This is a schematic diagram illustrating the principle of spatial alignment between a digital human body model and a real human body model in one embodiment of the present invention. Figure 2 As shown, a pre-constructed digital human body model and its included acupoints are superimposed and rendered onto the surface of the real human body model, so that the virtual acupoints of the digital human body model coincide with the coordinates of the real acupoints of the real human body model. In one specific implementation, the system calls a pre-constructed human digital model. This human digital model is a three-dimensional human digital model conforming to the national standard (GB / T 12346-2006). The model has multiple pre-defined virtual acupoints, each with predefined three-dimensional coordinates (x, y, z) on the digital model. The system also establishes an acupoint spatial database, storing information such as the three-dimensional coordinates, anatomical layers, and indications for each acupoint.

[0025] The system overlays a digital human model as a virtual image onto the location of the identified real human model. Then, through spatial alignment, it adjusts the posture and position of the digital human model to visually perfectly overlap with the real human model.

[0026] In this embodiment, a three-level registration strategy is adopted for spatial alignment: First, the human digital model is initialized and coarsely registered. After the system starts, the human digital model is loaded in front of the user's field of vision, at a distance of 100cm from the center of the user's head. The user activates the human digital model's follow mode or displacement mode through specific gestures such as pinching or grabbing, and drags the human digital model near the physical silicone model to visually align it with the physical model in terms of position and posture, thus completing the initial alignment.

[0027] Then, six-degree-of-freedom fine-tuning is performed. After coarse registration, the user can bring up the holographic fine-tuning control panel via voice command or gesture. This panel includes translation control buttons for the X, Y, and Z axes, including up, down, left, right, forward, and backward fine-tuning, as well as control buttons for rotation angles for the three axes, such as pitch, yaw, and roll fine-tuning. Fine-tuning refers to adjusting the human digital model to a less than predetermined range. When a user clicks a virtual button, the system incrementally updates the coordinates of the human digital model and its associated acupoints with preset minimum step values, such as a displacement step of 1 mm and a rotation step of 0.5 degrees, until the virtual acupoints completely coincide with the marked points on the surface of the physical model, thus obtaining the transformation matrix M of the human digital model relative to the real human model.

[0028] Finally, spatial anchoring and transformation locking are performed. After the user confirms the alignment is correct, they click the lock button. The system utilizes the spatial mapping capabilities of the AR device to create a world anchor point at the current physical location. The system binds the final transformation matrix M to this anchor point, ensuring that during subsequent teaching, regardless of how the user moves their viewpoint, the virtual model remains relatively fixed on the surface of the physical model and does not drift. The system continuously tracks the position of the physical model, updating the transformation matrix in real time as the model moves to ensure that the virtual acupoints do not drift. After alignment, each virtual acupoint on the digital human body model corresponds one-to-one with its real acupoint on the real human body model in space.

[0029] The system captures the user's hand movements using depth cameras and sensors, and outputs the real-time coordinates of the user's index fingertip based on the hand movements using a pre-built hand skeletal model. It also calculates the distance between the index fingertip coordinates and the coordinates of each virtual acupoint. In one specific implementation, while wearing the device, the user points to acupoints on a real human body model with their finger. The augmented reality head-mounted display device captures the user's hand movements in real time through its depth camera and sensors, constructs a hand skeletal model, and outputs the real-time coordinates P1(x,y,z) of the user's index fingertip in the world coordinate system.

[0030] The system simultaneously acquires the coordinates P2(x,y,z) of all virtual acupoints within the current field of view. For each virtual acupoint, the system calculates in real time the distance between its coordinates and the coordinates of the index fingertip, such as the Euclidean distance D.

[0031] When the distance between the index fingertip coordinate and the coordinate of a virtual acupoint is less than a predetermined distance threshold, the acupoint teaching information is displayed through the UI information panel. In one specific implementation, the system pre-sets a distance threshold, such as 5mm, which is used to determine whether the user's finger has touched or pointed to a certain acupoint. The system compares each calculated Euclidean distance with the distance threshold.

[0032] If the distance between a virtual acupoint and the tip of the index finger is greater than or equal to a distance threshold, the acupoint is determined not to be selected, and the system will not display any information.

[0033] If the distance between a virtual acupoint and the tip of the index finger is less than a distance threshold, the acupoint is considered to have been selected by the user. The system then triggers the information display function and provides the following feedback: Visual feedback: The selected virtual acupoints are highlighted, such as by changing color or enlarging.

[0034] UI Generation: Instantiate a UI information panel, load the acupoint's name, indications, anatomical description, and video data, and display it in the augmented reality view, such as next to or near the virtual acupoint. Figure 3 This is a schematic diagram illustrating acupoint teaching information prompts in one embodiment of the present invention. Figure 3 As shown, the teaching information for acupoints may include, but is not limited to, the acupoint name, the meridian to which it belongs, its functions and indications, the acupuncture method, and the description of its anatomical location, in the form of text, images, or videos. The information content can be customized and may also be accompanied by voice broadcast.

[0035] Intelligent following: The algorithm controls the normal direction of the UI panel to always point in the direction of the user's line of sight, and automatically adds position offset according to the hand position to avoid the UI panel being blocked by the user's arm.

[0036] This invention differs significantly from existing methods for teaching acupoints in traditional Chinese medicine in terms of function and structure, mainly in the following aspects: Precise Overlay of Virtual and Physical Models: Existing teaching methods for TCM acupoints typically rely on illustrations, books, or two-dimensional models. While these methods can convey acupoint information, they lack a sense of depth and precision, making it difficult for students to accurately understand the spatial location of acupoints. This invention utilizes augmented reality technology to precisely overlay digital 3D models with physical teaching models. Students wearing HoloLens can see the perfect combination of virtual and physical models in real time. This method of overlaying virtual and physical models allows students to more intuitively understand the spatial relationships of acupoints, improving learning effectiveness.

[0037] Real-time Hand Tracking and Interactive Feedback: Unlike existing solutions, this invention utilizes real-time hand tracking technology to enable students to interact with the model during the learning process. Students simply bring their index finger close to an acupoint, and the Hololens system calculates the distance between the finger and the acupoint, triggering the display of that acupoint information. This interactive feedback method breaks away from the passive learning model of traditional teaching, providing immediate and personalized teaching information, significantly enhancing the interactivity and engagement of learning.

[0038] Automated Information Display: In existing TCM teaching programs, students typically rely on teacher explanations or static teaching tools to acquire acupoint information. This invention, however, sets a distance threshold; when a student's finger approaches a target acupoint, the system automatically displays detailed information such as the acupoint's name, function, and common treatment methods. This automated information display method not only improves teaching efficiency but also reduces reliance on teachers, enabling learners to acquire knowledge independently and quickly.

[0039] Enhanced Personalized Learning Experience: Unlike standardized teaching methods in existing technologies, this invention utilizes HoloLens devices for personalized learning, providing customized feedback and information based on each student's learning progress and needs. This augmented reality-based personalized learning approach helps improve students' memory of acupoints and enhances their practical skills.

[0040] The beneficial technical effects of the present invention also include: Improving teaching accuracy and operability: By overlaying virtual and physical models, students can more accurately grasp the location and positioning methods of each acupoint. Compared with traditional teaching methods, it has higher accuracy, especially in locating complex acupoints, and can significantly reduce student misunderstandings or positioning errors.

[0041] Enhance learning interactivity and engagement: Through real-time hand tracking and feedback, students can directly interact with the teaching model, enhancing their sense of participation and initiative in the learning process and helping them better understand and master the location of acupoints in Traditional Chinese Medicine.

[0042] Personalized, real-time feedback: This provides personalized, real-time information displays, enabling students to quickly receive feedback based on their learning progress. This approach not only accelerates the learning process but also enhances students' interest and motivation.

[0043] In summary, this invention, by combining augmented reality technology and hand tracking technology, breaks through the limitations of traditional Chinese medicine acupoint teaching methods, providing a more precise, interactive, and personalized learning approach, and greatly improving the effectiveness of Chinese medicine acupoint teaching and students' learning experience.

[0044] Example 2: Figure 4 This is a schematic diagram of the structure of a device for teaching acupoints in one embodiment of the present invention, as shown below. Figure 4 As shown, the device includes a processor 401, a memory 402, a bus 403, and a computer program stored in the memory 402 and executable on the processor 401. The processor 401 includes one or more processing cores. The memory 402 is connected to the processor 401 via the bus 403. The memory 402 is used to store program instructions. When the processor executes the computer program, it implements the steps in the above-described method embodiment of Embodiment 1 of the present invention.

[0045] Furthermore, as an executable solution, the human acupoint teaching device can be a computer unit, which can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer unit may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the above-described structure of the computer unit is merely an example and does not constitute a limitation on the computer unit. It may include more or fewer components, or combine certain components, or use different components. For example, the computer unit may also include input / output devices, network access devices, buses, etc., and this embodiment of the invention does not limit this.

[0046] Furthermore, as an executable solution, the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The processor is the control center of the computer unit, connecting various parts of the entire computer unit via various interfaces and lines.

[0047] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the computer unit by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital card (SD card), flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0048] Example 3: The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described in the embodiments of the present invention.

[0049] If the modules / units integrated in the computer unit are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0050] Example 4: The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method for teaching acupoints as described above.

[0051] Although the invention has been specifically shown and described in conjunction with preferred embodiments, those skilled in the art should understand that various changes in form and detail may be made to the invention without departing from the spirit and scope of the invention as defined in the appended claims, all of which shall be within the scope of protection of the invention.

Claims

1. A method for teaching acupoints on the human body, characterized in that, Includes the following steps: After a user puts on an augmented reality head-mounted display device, a pre-set real human body model is identified by a depth camera set on the augmented reality head-mounted display device. The pre-built digital human body model and its included acupoints are superimposed and rendered onto the surface of the real human body model, so that the virtual acupoints of the digital human body model coincide with the coordinates of the real acupoints of the real human body model. The system captures the user's hand movements using a depth camera and sensors, and outputs the real-time coordinates of the user's index fingertip based on the hand movements using a pre-built hand skeletal model. It also calculates the distance between the index fingertip coordinates and the coordinates of each virtual acupoint. When the distance between the index fingertip coordinate and the coordinate of a virtual acupoint is less than a predetermined distance threshold, the acupoint teaching information is displayed through the UI information panel.

2. The method for teaching acupoints according to claim 1, characterized in that, The real human body model is made of medical-grade silicone material with a Shore hardness of A10-20.

3. The method for teaching acupoints according to claim 1, characterized in that, The step of overlaying and rendering a pre-constructed digital human body model and its included acupoints onto the surface of the real human body model, so that the virtual acupoints of the digital human body model coincide with the coordinates of the real acupoints of the real human body model, includes: In response to the user's gesture drag, the digital human body model is moved to a predetermined range of the real human body model to complete the initial alignment; In response to the user's fine-tuning control interface, the translation and rotation angles of the human digital model along the X, Y, and Z axes are adjusted in steps until the virtual acupoints of the human digital model completely coincide with the surface markers of the real acupoints of the real human model, thereby obtaining the transformation matrix of the human digital model relative to the real human model; the fine-tuning refers to adjusting the human digital model by less than a predetermined adjustment range. Create a world anchor point at the currently perfectly overlapping physical location, and bind the transformation matrix to the world anchor point.

4. The method for teaching acupoints according to claim 3, characterized in that, The step-by-step adjustment of the translation and rotation angle of the human digital model along the X, Y, and Z axes is as follows: the step value of the translation is 1 mm, and the step value of the rotation angle is 0.5 degrees.

5. The method for teaching acupoints according to claim 1, characterized in that, The predetermined distance threshold is 5 millimeters.

6. The method for teaching acupoints according to claim 1, characterized in that, The acupoint teaching information includes the acupoint name, functions and indications, anatomical layers, acupuncture depth, needle insertion angle, and / or teaching videos.

7. A device for teaching acupoints on the human body, characterized in that, The method includes a memory and a processor, the memory storing at least one program, which is executed by the processor to implement the method for teaching acupoints as described in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The storage medium stores at least one program, which is executed by a processor to implement the method for teaching acupoints as described in any one of claims 1-6.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method for teaching human acupoints as described in any one of claims 1-6.

Images