Intelligent auxiliary acupuncture system based on AR technology

Abstract

The invention discloses an intelligent auxiliary acupuncture system based on the AR technology, and the system comprises the following modules: a multi-modal sensing module which is used for collecting a multi-modal sensing data set containing the surface information of a patient; the visual SLAM guiding module is used for outputting a relative pose between the camera and the skin of the patient; the three-dimensional reconstruction module is used for generating a three-dimensional skin grid model; the candidate acupoint generation module is used for generating a candidate acupoint set under the condition that the density threshold value and the minimum spacing are met; the stable matching module is used for outputting a stable matching result set; the trajectory planning module is used for binding the needle insertion trajectory to the three-dimensional skin grid model to form a needle insertion guide set; and the augmented reality rendering module is used for carrying out real-time visual rendering on the virtual acupuncture point target and the needle inserting track through an augmented reality display terminal. The invention constructs an intelligent acupuncture guide system which is oriented to a real clinical scene and has a real-time correction capability and a medical significance interpretation capability.

Description

Technical Field

[0001] The present invention relates to the technical field of acupuncture, and particularly to an intelligent assisted acupuncture system based on AR technology. Background Art

[0002] With the development of augmented reality and computer vision technologies, the teaching and clinical operation scenarios of traditional Chinese medicine acupuncture have gradually tried to introduce auxiliary systems based on visual SLAM and AR guidance to improve the intuitiveness of acupoint positioning and the standardization of the acupuncture process. Such systems usually reconstruct the three-dimensional model of the patient's surface through a camera and a depth perception module, and superimpose and display the preset standard acupoint template or path planning result in a visual form on the patient's body surface to help doctors identify acupuncture points and perform needle insertion operations. However, there are still significant deficiencies in the actual application scenarios of acupuncture in the prior art.

[0003] First of all, most current AR acupuncture systems use fixed templates or static image overlay methods for acupoint display, lacking the dynamic adaptation ability to the spatial differences of the individual body surface, its own posture changes, or skin bending and deformation. When facing slight displacement of the body surface or hand occlusion, the acupoint presentation is prone to problems such as dislocation, floating, or occlusion overlap, lacking stability and practicality. Secondly, existing systems mostly use the nearest neighbor strategy based on Euclidean distance for standard acupoint matching and acupoint-body registration, failing to fully introduce meridian topology, physiological structure constraints, and individualized needle-avoidance information, and it is difficult to achieve a stable and one-to-one matching relationship in the medical sense, resulting in the need for frequent manual intervention and adjustment during the teaching process, with low efficiency and accuracy.

[0004] In addition, in terms of needle insertion path planning, the prior art often only provides simple arrow-style direction prompts, lacking the ability to model the actual needle insertion angle, depth, and physiological obstacle avoidance, and also lacking a correction mechanism linked to the SLAM mapping error. Once the scene drifts or is occluded, the rendered path does not match the skin surface seriously, thereby affecting the accuracy and safety of the actual operation.

[0005] In summary, there is an urgent need to innovate and integrate in the algorithm structure and spatial mapping mechanism to improve the intelligence and clinical adaptability of the overall system. Summary of the Invention

[0006] An object of the present invention is to propose an intelligent assisted acupuncture system based on AR technology. The present invention constructs an intelligent acupuncture guidance system facing real clinical scenarios, with real-time correction ability and medical significance interpretability, and has broad application potential in acupuncture teaching, assisted treatment, and personalized diagnosis and treatment.

[0007] An intelligent assisted acupuncture system based on AR technology according to an embodiment of the present invention includes the following modules: A multi-modal perception module for collecting a multi-modal perception data set containing patient surface information; A visual SLAM guidance module for performing visual SLAM initialization, constructing a unified spatial reference between the world coordinate system, the camera coordinate system, and the skin coordinate system, and outputting the relative pose between the camera and the patient's skin; A 3D reconstruction module for reprojection of the point cloud in a multi-frame depth image sequence to the skin coordinate system based on the relative pose to generate a 3D skin mesh model; A candidate acupoint generation module for mapping a standard meridian path to the skin surface in the skin space topological coordinate domain based on the 3D skin mesh model and the bony landmark coordinate set to form a meridian mapping path, and generating a set of candidate acupoint points under the conditions of meeting the density threshold and the minimum distance; A stable matching module for constructing an initial two-way preference list based on the set of candidate acupoint points and the standard acupoint knowledge base, constructing a dynamic stable matching preference scoring function, executing an improved Gale-Shapley stable matching algorithm, and outputting a set of stable pairing results; A trajectory planning module for planning a 3D acupuncture insertion trajectory that meets the acupuncture insertion angle limit, the acupuncture insertion depth limit, and the vessel avoidance condition in the local subcutaneous space of the candidate acupoint according to the set of stable pairing results, and binding the acupuncture insertion trajectory to the 3D skin mesh model to form an acupuncture insertion guidance set; An augmented reality rendering module for real-time overlaying the acupuncture insertion guidance set on the patient's skin surface and performing real-time visualization rendering of virtual acupoint targets and acupuncture insertion trajectories through an augmented reality display terminal.

[0008] An intelligent assisted acupuncture method based on AR technology for implementing a blast shock wave front distribution prediction system based on particle swarm optimization, including the following steps: S1. Collecting a multi-modal perception data set containing patient surface information; S2. Performing visual SLAM initialization according to the multi-modal perception data set, establishing a unified spatial reference of the world coordinate system - camera coordinate system - skin coordinate system, and outputting the initial relative pose between the camera and the patient's skin; S3. Generating a 3D skin mesh model based on the visual SLAM initialization result, and extracting a bony landmark coordinate set based on the 3D skin mesh model; S4. Automatically generating a set of candidate acupoint points according to the 3D skin mesh model and the bony landmark coordinate set in combination with the meridian topology rules; S5. Reading meridian names, disease relevance, and acupuncture contraindicated area information from a pre-set standard acupoint knowledge base, constructing a two-way preference list corresponding to the set of candidate acupoint points, inputting the set of candidate acupoint points and the standard acupoint knowledge base into the Gale-Shapley stable matching algorithm, and outputting a pairing result; S6. Calculate the needle insertion trajectory that meets the needle insertion angle limit, needle insertion depth limit, and vessel avoidance condition based on the pairing result, and associate the needle insertion trajectory with the three-dimensional skin mesh model; S7. Render the semi-transparent virtual acupoint target and the visible needle insertion path in the augmented reality display terminal according to the pairing result and the needle insertion trajectory, realize the real-time overlay display of the virtual content and the real skin surface, and refresh the virtual acupoint target and the needle insertion trajectory in real time according to the updated pairing result until the whole acupuncture process is completed.

[0009] Optionally, the S2 includes the following steps: S21. Input the multi-modal perception data set into the visual SLAM guidance module. The multi-modal perception data set includes the color image frame sequence , depth image frame sequence and the attitude data stream output by the inertial measurement unit , where is the time frame index. The color image frame sequence, depth image frame sequence, and the attitude data stream output by the inertial measurement unit are used to synchronously generate the initial image reference frame set of the acupuncture scene ; S22. Extract the feature point set from the initial image reference frame set of the acupuncture scene , and construct the skin area weighted image feature mask according to the curvature of the acupuncture skin area. The skin area weighted image feature mask sets the skin area as the priority area for feature point selection, forming a skin area feature point subset ; S23. Perform robust registration on the skin area feature point subset , and calculate the skin stable pose estimation matrix : ; Among them, represents the spatial pose of the patient's skin surface at the th frame moment in the acupuncture scene in the world coordinate system , is the skin stable pose estimation matrix, represents the spatial rigid body transformation matrix from the three-dimensional Euclidean space to the affine transformation group, including rotation and translation components, , respectively represent the image coordinate positions of the th paired skin feature points in the current frame and the previous frame image, used to construct the inter-frame feature correspondence, , respectively represent the depth values at the corresponding image point positions in the th frame and the th frame, , are the unit normal vectors corresponding to the th skin points in the current frame and the previous frame respectively, is the M-estimator loss function in robust estimation, which is used to optimize the influence of abnormal errors caused by hand occlusion and light reflection on pose estimation, is the number of the priority feature point subsets of the skin area, representing the number of effective registration points available for pose calculation in the current acupuncture reference area; S24. Jointly constrain the skin stable pose estimation matrix and the camera-inertial fusion pose to obtain the camera-skin relative pose transformation matrix . The camera-skin relative pose transformation matrix represents the dynamic spatial relationship between the camera and the skin in the acupuncture scene S25. Use the camera-skin relative pose transformation matrix to construct the main reference frame of the skin coordinate system.

[0010] Optionally, the S3 includes the following steps: S31. Based on the camera-skin relative pose transformation matrix and the main reference frame of the skin coordinate system, re-project the skin points in the acupuncture area included in the multi-frame depth image frame sequence, and uniformly map the depth information of each frame to the skin coordinate system to form the skin point cloud data in the continuous space. On the basis of the skin point cloud data, perform meshing processing to construct a three-dimensional skin mesh model ; S32. In the three-dimensional skin mesh model , by analyzing the curvature gradient and normal consistency of the mesh vertices, identify the positions with significant geometric protrusion changes and extract them as the set of bone marker candidate points . For each bone marker candidate point in the set of bone marker candidate points, calculate its principal curvature response value, and find the point pair with the largest curvature difference in its neighborhood. If the largest curvature difference exceeds the set curvature threshold, then this point pair is considered as a bone marker point; S33. Screen the extracted set of bone marker candidate points to form the bone marker coordinate set , and the screening conditions are: the spatial distance between the bone marker points must be greater than the preset minimum distance threshold; the included angle between the normal vectors corresponding to the bone marker points must be less than the set included angle threshold; S34. Map the bone marker coordinate set to the camera coordinate system to obtain the corresponding set of observed projection points , perform consistent tracking on the observed projection points in consecutive image frames, construct a bone alignment error function based on the projection error, and optimize the camera inertial fusion pose of the current frame by minimizing the bone alignment error function to obtain the optimized bone constraint pose. .

[0011] Optionally, the three-dimensional skin mesh model is composed of a vertex set, an edge set, and a corresponding normal vector set. The vertex set represents the position of each reconstructed point on the skin surface, and the normal vector set represents the curvature direction change of each vertex on the skin surface.

[0012] Optionally, S4 includes the following steps: S41. Based on the three-dimensional skin mesh model and the bone marker coordinate set , construct a skin space topological coordinate domain in the main reference frame of the skin coordinate system. The skin space topological coordinate domain refers to a continuous space region composed of all grid vertices within the skin region centered on all bone marker points and limited within the bone control radius range. The continuous space region serves as the basic space range for candidate acupoint spreading. S42. Represent each standard meridian path in the meridian topological information database as a point sequence composed of several standard acupoints, and project the standard meridian path within the skin space topological coordinate domain to generate a meridian mapping path on the surface of the three-dimensional skin mesh model. The meridian mapping path maintains the same orientation as the original meridian but is nested within the actual skin mesh surface structure due to local skin geometry changes. S43. Calculate the skin curvature response value and the bone guiding strength value of each vertex in the skin mesh on the meridian mapping path. The skin curvature response value reflects the degree of local skin curvature change, and the bone guiding strength value represents the reciprocal relationship of the distance between the current vertex and the bone marker coordinate set. Weightedly sum the skin curvature response value and the bone guiding strength value to form a candidate acupoint density function for quantifying the potential of each skin point to generate candidate acupoints. S44. According to the confidence level of the camera pose estimation of the current frame adaptive adjustment is performed on the candidate acupoint density function to form an adaptive spreading density function . When the confidence level of the camera pose estimation is higher than the threshold, the adaptive spreading density function approaches the candidate acupoint density function itself. When the confidence level of the camera pose estimation decreases, the candidate density is increased through a magnification factor . S45. On each meridian mapping path, according to the adaptive spreading density function Perform point sampling on the surface of the skin mesh to generate a set of candidate acupoint locations , and only when the sampling density of a certain point location is higher than the set density lower threshold , and the distance from the already sampled points is greater than the minimum acupoint spacing threshold , will the candidate acupoint location be retained as a legal candidate acupoint location.

[0013] Optionally, the S5 includes the following steps: S51. Based on the set of candidate acupoint locations and the set of standard acupoints in the standard acupoint knowledge base, construct an initial two-way preference list between the candidate acupoint locations and the standard acupoints. The initial two-way preference list respectively defines the initial matching tendencies of the candidate acupoint locations and the standard acupoints towards each other; S52. Improve the single Euclidean distance preference calculation method in the Gale-Shapley stable matching algorithm based on the acupuncture assistance space guidance scenario, and introduce the skin geometric consistency measure under spatial constraints and the dynamic safety constraint penalty function , to form an improved space-guided multi-dimensional preference scoring function : ; wherein, represents the geometric Euclidean distance between the candidate acupoint location and the standard acupoint in the main reference frame of the skin coordinate system , represents using the local curvature change of the three-dimensional skin mesh model as an evaluation index to quantify the local skin geometric consistency measure between the candidate acupoint location and the standard acupoint on the skin surface, represents the dynamic safety constraint penalty function established based on the needling restriction area information. When the candidate acupoint location is located in or near the needling restriction area, the safety penalty coefficient for pairing is increased, represents the standard acupoint 's treatment relevance score for a specific disease, is the weighting coefficient for different preference indicators; S53. Based on the improved space-guided multi-dimensional preference scoring function , introduce the stability criterion function to dynamically correct the two-way preference list between the candidate acupoint locations and the standard acupoints: ; wherein, is the standard deviation of the position fluctuation of the candidate acupoint location output by visual SLAM in consecutive frames, reflecting the stability of the acupoint location in space, is the standard deviation of the normal vector direction fluctuation of the candidate acupoint location in consecutive frames, reflecting the direction stability of the acupoint location, , is the threshold parameter for position stability and orientation stability; S54. According to the stability criterion function Adjust the spatial guidance multi-dimensional preference scoring function to obtain a dynamic stable matching preference scoring function , ; Under the action of the dynamic stable matching preference scoring function, when the spatial stability of the candidate acupoint decreases, the matching priority of this point with the corresponding standard acupoint is automatically reduced; S55. The candidate acupoint set , the standard acupoint set and the dynamic stable matching preference scoring function are input into the improved Gale-Shapley stable matching algorithm for iterative matching calculation, and the final stable pairing result set is output. For all matching pairs already in the final stable pairing result set , there does not exist another unpaired combination such that the following two conditions are simultaneously satisfied: The candidate acupoint tends to the standard acupoint in its dynamic stable preference , and outperforms its matched object ; The standard acupoint tends to the candidate acupoint in its dynamic stable preference , and outperforms its matched object ; If the pairing pair does not exist, it means that the current matching pair is stable and there is no destructive preference recombination: ; Among them, , respectively represent the priority ranking relationship between the candidate acupoint and the standard acupoint determined by the dynamic stable matching preference scoring function. The dynamic stable condition ensures that each stable pairing result not only has no cross obstruction in the preference dimension, but also meets the matching requirements of the real-time position stability of visual SLAM.

[0014] Optionally, the S6 includes the following steps: S61. Based on the stable pairing result set in the main reference frame of the skin coordinate system , with the candidate acupoint as the acupuncture target point, from the three-dimensional skin mesh model Method for extracting the normal vector of candidate acupoint points and define the normal vector of the candidate acupoint point as the reference for the default needle insertion direction; S62. At each candidate acupoint point establish a local subcutaneous three-dimensional space search area , and the three-dimensional space search area is a spherical volume space with the candidate acupoint point as the center of the sphere and the maximum needle insertion depth as the radius, serving as the search range for needle insertion path planning; S63. In the local subcutaneous three-dimensional space search area , evaluate all feasible needle insertion paths; S64. Among the candidate paths that meet all the constraint conditions, select the needle insertion path that is most consistent with the normal direction of the candidate acupoint point as the target needle insertion path , and the target needle insertion path represents a three-dimensional vector line segment extending from the acupoint on the skin surface to the target depth along a predetermined direction; S65. Map and attach the target needle insertion path to the vertex structure on the three-dimensional skin mesh model , and establish a trajectory correspondence with the standard acupoints in the pairing result to form a set of needle insertion guides for augmented reality visualization overlay.

[0015] Optionally, the path evaluation is screened according to the following three constraint rules: If the included angle between the needle insertion path and the normal vector of the candidate acupoint point meets , it is considered to meet the needle insertion angle limit; If the length of the needle insertion path meets , it is considered to meet the needle insertion depth limit; If the minimum distance between the needle insertion path and any blood vessel point in the three-dimensional subcutaneous vascular mesh model on the path segment meets , it is considered to meet the vascular avoidance condition; where, represents the included angle between the candidate needle insertion path and the normal vector of the candidate acupoint point, used to measure whether the needle insertion path deviates from the skin surface normal, represents the maximum allowable deviation angle threshold of the needle insertion direction, which is the set tolerance range for the needle deviating from the skin normal. A path exceeding the maximum deviation angle threshold is considered not to meet the needle insertion angle limit, represents the actual path length of the candidate needle insertion path, defined as the spatial distance from the candidate acupoint point along the candidate path direction to the termination point, Represents the minimum needle insertion depth threshold allowed for the needle insertion path, which is the set lower limit value used to avoid insufficient needle insertion depth and inability to achieve the treatment effect. Represents the maximum needle insertion depth threshold allowed for the needle insertion path. Represents the minimum distance between the needle insertion path and the nearest blood vessel point in the three-dimensional subcutaneous vascular grid model, which is used to measure whether the needle insertion path is close to the blood vessel. Represents the minimum safe distance threshold between the needle insertion path and the blood vessel, which is the lower limit standard for vascular avoidance set by the system.

[0016] The beneficial effects of the present invention are as follows: (1) The present invention proposes a dynamic stable Gale-Shapley matching model for the acupuncture scenario to solve the real-time one-to-one pairing problem under complex preference constraints. By introducing a spatial-guided multi-dimensional preference scoring function and a dynamic stability adjustment function, the matching between standard acupoints and candidate acupoints is extended from static Euclidean distance to an integrated optimization index that comprehensively considers skin geometric consistency, acupuncture point prohibition area penalty, disease relevance, and SLAM scenario stability. During the iterative matching process, the priority ranking is dynamically adjusted, effectively overcoming the dependence of traditional stable matching methods on static preference assumptions in the visual dynamic environment, and achieving a one-to-one stable pairing of global unobstructed pairs on the premise of meeting medical semantic registration, thereby improving the spatial accuracy and clinical applicability of registration.

[0017] (2) The present invention constructs a three-dimensional acupoint scattering mechanism based on skin geometry and visual stability to enhance the distribution rationality and robustness of the matching space. When generating the candidate acupoint point set, an adaptive density function is constructed using the skin mesh curvature response and bony guiding strength, and further combined with the pose confidence degree output by SLAM for dynamic adjustment, so that the scattering result can automatically increase the distribution density in areas with poor visual stability, thereby enhancing the system's robustness to actual scene disturbances such as occlusion, light changes, or slight patient movement at the spatial level, and realizing a joint constraint scattering strategy between skin anatomical structure and mapping accuracy.

[0018] (3) The present invention realizes a real-time needle insertion path planning method that combines three elements: angle limitation, depth boundary, and blood vessel avoidance. Based on the pairing result, a three-dimensional path planning mechanism is proposed, comprehensively considering three constraint conditions: the included angle of the needle insertion angle, the depth range of the needle instrument, and the blood vessel distance threshold, to construct a controllable and obstacle-avoiding needle insertion space search area. The finally output needle insertion trajectory is real-time associated with the three-dimensional skin mesh model and a trajectory mapping relationship is established with the standard acupoint, significantly improving the safety and accuracy of clinical operations. Description of the Drawings

[0019] The drawings are used to provide a further understanding of the present invention and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the present invention and do not constitute a limitation to the present invention. In the drawings: Figure 1 Flow chart of an intelligent assisted acupuncture system based on AR technology proposed by the present invention; Figure 2 Schematic diagram of the acupoint matching process for realizing multi-dimensional preference and dynamic stability regulation by improving the Gale-Shapley algorithm in an intelligent assisted acupuncture system based on AR technology proposed by the present invention. Specific implementation manner

[0020] Now, the present invention will be further described in detail with reference to the accompanying drawings. These drawings are all simplified schematic diagrams, only illustrating the basic structure of the present invention in a schematic manner, so they only show the components related to the present invention.

[0021] Refer to Figure 1 - Figure 2 , an intelligent assisted acupuncture system based on AR technology, includes the following modules: A multi-modal perception module, used to collect a multi-modal perception data set including the surface information of the patient; A visual SLAM guidance module, used to perform visual SLAM initialization, construct a unified spatial reference between the world coordinate system, the camera coordinate system and the skin coordinate system, and output the relative pose between the camera and the patient's skin; A three-dimensional reconstruction module, used to re-project the point cloud in a multi-frame depth image sequence to the skin coordinate system based on the relative pose, and generate a three-dimensional skin mesh model; A candidate acupoint generation module, used to map the standard meridian path to the skin surface to form a meridian mapping path in the skin space topological coordinate domain based on the three-dimensional skin mesh model and the bony marker coordinate set, and generate a set of candidate acupoint points under the conditions of meeting the density threshold and the minimum distance; A stable matching module, used to construct an initial two-way preference list based on the set of candidate acupoint points and the standard acupoint knowledge base, construct a dynamic stable matching preference scoring function, execute the improved Gale-Shapley stable matching algorithm, and output a set of stable pairing results; A trajectory planning module, used to plan a three-dimensional acupuncture insertion trajectory that meets the acupuncture insertion angle limit, the acupuncture insertion depth limit and the vessel avoidance condition in the local subcutaneous space of the candidate acupoint according to the set of stable pairing results, and bind the acupuncture insertion trajectory to the three-dimensional skin mesh model to form an acupuncture insertion guidance set; An augmented reality rendering module, used to overlay the acupuncture insertion guidance set on the patient's skin surface in real time, and perform real-time visualization rendering of the virtual acupoint target and the acupuncture insertion trajectory through an augmented reality display terminal.

[0022] An intelligent assisted acupuncture method based on AR technology, used to execute a system for predicting the front distribution of explosion shock waves based on particle swarm optimization, includes the following steps: S1. Collect a multi-modal perception data set containing patient surface information; S2. Perform visual SLAM initialization based on the multi-modal perception data set, establish a unified spatial reference for the world coordinate system - camera coordinate system - skin coordinate system, and output the initial relative pose between the camera and the patient's skin; S3. Generate a three-dimensional skin mesh model based on the results of visual SLAM initialization, and extract a set of bony landmark coordinates based on the three-dimensional skin mesh model; S4. Automatically generate a set of candidate acupoint points according to the three-dimensional skin mesh model and the set of bony landmark coordinates in combination with the meridian topology rules; S5. Read meridian names, disease relevance, and information on acupuncture needle prohibition areas from a pre-set standard acupoint knowledge base, construct a two-way preference list corresponding to the set of candidate acupoint points, input the set of candidate acupoint points and the standard acupoint knowledge base into the Gale-Shapley stable matching algorithm, and output the matching result; S6. Calculate the acupuncture needle insertion trajectory that meets the acupuncture needle insertion angle limit, acupuncture needle insertion depth limit, and vascular avoidance conditions based on the matching result, and associate the acupuncture needle insertion trajectory with the three-dimensional skin mesh model; S7. Render semi-transparent virtual acupoint targets and visible acupuncture needle insertion paths in the augmented reality display terminal according to the matching result and the acupuncture needle insertion trajectory, realize real-time overlay display of virtual content and the real skin surface, and refresh the virtual acupoint targets and the acupuncture needle insertion trajectory in real time according to the updated matching result until the entire acupuncture process is completed.

[0023] In this embodiment, S2 includes the following steps: S21. Input the multi-modal perception data set into the visual SLAM guidance module. The multi-modal perception data set includes a sequence of color image frames , a sequence of depth image frames , and the attitude data stream output by the inertial measurement unit , where is the time frame index. The sequence of color image frames, the sequence of depth image frames, and the attitude data stream output by the inertial measurement unit are used to synchronously generate an initial image reference frame set for the acupuncture scene ; S22. Extract a set of feature points from the initial image reference frame set for the acupuncture scene , and construct a skin region weighted image feature mask based on the curvature of the acupuncture skin region . The skin region weighted image feature mask sets the skin region as the priority region for feature point selection, forming a subset of skin region feature points ; ; S23. Perform robust registration on the subset of skin region feature points , and calculate the skin stable pose estimation matrix : ; Among them, represents the spatial pose of the patient's skin surface at the th frame moment in the acupuncture scenario in the world coordinate system and is the skin stable pose estimation matrix. represents the spatial rigid body transformation matrix from three-dimensional Euclidean space to the affine transformation group, including rotation and translation components. , respectively represent the image coordinate positions of the th paired successful skin feature points in the current frame and the previous frame image, which are used to construct the inter-frame feature correspondence relationship. , respectively represent the depth values at the corresponding image point positions in the th frame and the th frame. , respectively are the unit normal vectors corresponding to the th skin point in the current frame and the previous frame. is the M-estimator loss function in robust estimation, which is used to optimize the influence of abnormal errors caused by hand occlusion and light reflection on pose estimation. is the number of the priority feature point subset of the skin area, indicating the number of effective registration points available for pose calculation in the current acupuncture reference area; S24. Jointly constrain the skin stable pose estimation matrix and the camera inertial fusion pose to obtain the camera-skin relative pose transformation matrix . The camera-skin relative pose transformation matrix represents the dynamic spatial relationship between the camera and the skin in the acupuncture scenario. S25. Use the camera-skin relative pose transformation matrix to construct the main reference frame of the skin coordinate system.

[0024] The camera inertial fusion pose refers to the more stable and disturbance-resistant spatial pose estimation result obtained by fusing the visual pose information extracted by the camera through images with the acceleration and angular velocity data provided by the inertial measurement unit during the visual SLAM mapping process.

[0025] The camera inertial fusion pose is in the image sequence of consecutive frames. The visual SLAM algorithm first extracts and matches image feature points to calculate the visual estimated pose transformation between the current frame and the previous frame; meanwhile, the IMU provides the three-axis acceleration and the angular velocity ; then, through the extended Kalman filter (EKF), nonlinear optimization, or factor graph optimization state estimation method, the visual and inertial information is fused in a unified state space, and the camera-inertial fusion pose of the camera in the world coordinate system is output , which is the camera-inertial fusion pose. The fusion pose has the ability to resist short-term occlusion, drift in weak texture areas, and slight blur interference, and is an important spatial reference benchmark for accurate skin registration and trajectory planning in the acupuncture scene.

[0026] In this embodiment, S3 includes the following steps: S31. Based on the camera-skin relative pose transformation matrix and the main reference frame of the skin coordinate system , re-project the acupuncture area skin points included in the multi-frame depth image frame sequence, and uniformly map the depth information of each frame to the skin coordinate system to form skin point cloud data in a continuous space. On the basis of the skin point cloud data, perform meshing processing to construct a three-dimensional skin mesh model ; S32. In the three-dimensional skin mesh model , by analyzing the curvature gradient and normal consistency of the mesh vertices, identify the positions with significant geometric protrusion changes, and extract them as the set of bone marker candidate points . For each bone marker candidate point in the set of bone marker candidate points, calculate its principal curvature response value, and find the point pair with the largest curvature difference in its neighborhood. If the maximum curvature difference exceeds the set curvature threshold, then this point pair is considered a bone marker point; S33. Screen the extracted set of bone marker candidate points to form a set of bone marker coordinates , and the screening conditions are: the spatial distance between bone marker points must be greater than the preset minimum distance threshold; the angle between the normal vectors corresponding to bone marker points must be less than the set angle threshold; S34. Map the set of bone marker coordinates to the camera coordinate system to obtain the corresponding set of observed projection points . Consistently track the observed projection points in the continuous image frames, and construct a bone alignment error function based on the projection error. Optimize the camera-inertial fusion pose of the current frame by minimizing the bone alignment error function to obtain the optimized bone constraint pose .

[0027] In this embodiment, the three-dimensional skin mesh model is composed of a vertex set, an edge set, and a corresponding normal vector set. The vertex set represents the position of each reconstructed point on the skin surface, and the normal vector set represents the change in the curvature direction of each vertex on the skin surface.

[0028] In this embodiment, S4 includes the following steps: S41. Based on the three-dimensional skin mesh model and the set of bone marker coordinates , a skin space topological coordinate domain is constructed in the main reference frame of the skin coordinate system . The skin space topological coordinate domain refers to a continuous space area composed of all grid vertices within the skin area centered on all bone marker points and limited within the bone control radius range. The continuous space area serves as the basic space range for candidate acupoint scattering . S42. Each standard meridian path in the meridian topological information database is represented as a point sequence composed of several standard acupoints, and the standard meridian path is projected within the skin space topological coordinate domain to generate a meridian mapping path on the surface of the three-dimensional skin mesh model. The meridian mapping path maintains the same orientation as the original meridian but is nested within the actual skin mesh surface structure due to local skin geometry changes S43. Calculate the skin curvature response value and the bone guiding intensity value of each vertex in the skin mesh on the meridian mapping path. The skin curvature response value reflects the degree of local skin curvature change, and the bone guiding intensity value represents the reciprocal relationship of the distance between the current vertex and the set of bone marker coordinates. The skin curvature response value and the bone guiding intensity value are weighted and summed to form a candidate acupoint density function , which is used to quantify the potential of each skin point to generate candidate acupoints S44. According to the confidence level of the camera pose estimation in the current frame , the candidate acupoint density function is adaptively adjusted to form an adaptive scattering density function . When the confidence level of the camera pose estimation is higher than the threshold, the adaptive scattering density function tends to the candidate acupoint density function itself. When the confidence level of the camera pose estimation decreases, the candidate density is increased through a magnification factor . S45. On each meridian mapping path, point sampling is performed on the surface of the skin mesh according to the adaptive scattering density function to generate a set of candidate acupoint points . Only when the scattering density of a certain point is higher than the set density lower threshold and the distance from the already scattered points is greater than the minimum acupoint spacing threshold , the candidate acupoint point is retained as a legal candidate acupoint point

[0029] In this embodiment, S5 includes the following steps S51. Based on the candidate acupoint set and the standard acupoint set in the standard acupoint knowledge base, construct an initial two-way preference list between candidate acupoints and standard acupoints. The initial two-way preference list respectively defines the initial matching tendencies of candidate acupoints and standard acupoints towards each other. S52. Improve the single Euclidean distance preference calculation method in the Gale-Shapley stable matching algorithm based on the acupuncture assistance space guidance scenario, and introduce the skin geometric consistency measure under spatial constraints and the dynamic safety constraint penalty function to form an improved space-guided multi-dimensional preference scoring function : ; where represents the geometric Euclidean distance between the candidate acupoint and the standard acupoint in the main reference frame of the skin coordinate system , represents using the local curvature change of the three-dimensional skin mesh model as an evaluation index to quantify the local skin geometric consistency measure between the candidate acupoint and the standard acupoint on the skin surface, represents the dynamic safety constraint penalty function established according to the acupuncture-forbidden area information. When the candidate acupoint is located in or near the acupuncture-forbidden area, the safety penalty coefficient for pairing is increased, represents the treatment relevance score of the standard acupoint for a specific disease, is the weighting coefficient of different preference indicators; In this embodiment, the acquisition method of the skin geometric consistency measure is to use the skin geometric consistency measure to measure whether the local geometric structures of the candidate acupoint and the standard acupoint are consistent on the three-dimensional skin surface. Based on the three-dimensional skin mesh model and its vertex normal vector set for the candidate acupoint , extract the local neighborhood of the skin mesh where it is located; for the standard acupoint, extract its predefined local skin topological structure in the standard meridian and acupoint knowledge base as a reference geometric template; Matching method: Calculate the similarity between the local meshes of the two in terms of normal vector distribution, curvature gradient, and surface texture; Output form: Adopt methods based on cosine similarity, curvature difference integral, or surface rigid mapping error, and normalize to obtain a numerical similarity index , and the smaller the value, the closer the geometric shapes are.

[0030] Dynamic safety constraint penalty function To quantify whether a candidate acupoint is located in a needle - forbidden area or there are anatomical structures near it that pose a risk to needle insertion. Specifically, by combining the three - dimensional skin model and the bony landmark coordinate set obtained from visual SLAM and depth map reconstruction; the coordinate mask of the needle - forbidden area preset in the standard acupoint knowledge base or the medical needle - forbidden area rules; map the candidate acupoint to the skin coordinate system and judge the distance from the boundary of the needle - forbidden area; if it falls into the needle - forbidden area or its distance is less than the set safety threshold ; if the distance is less than the safety threshold, it is increased linearly or exponentially to 1; if it is completely outside the safe area, then set ; The dynamic safety constraint penalty function is used as a weighting term to participate in the calculation of the dynamic matching cost, effectively suppressing the pairing risk of the needle - forbidden part.

[0031] Treatment relevance score represents the adaptability degree of a standard acupoint to the current target disease, which is the clinical preference data for personalized acupuncture decision - making. In the acquisition process, there are multiple disease adaptability labels corresponding to each standard acupoint stored in the standard acupoint knowledge base; the physician inputs through the interface or the system automatically calls the patient's electronic medical record, chief complaint symptoms or AI - assisted diagnosis results; set a disease vector for the current disease, match the acupoint entries related to this disease in the acupoint label library; extract the historical association frequency, efficacy literature score or expert scoring between the standard acupoint and the target disease; after comprehensively calculating the weights from different sources and normalizing the score, obtain the treatment relevance score Dynamically correct the two - way preference list of candidate acupoints and standard acupoints: ; Among them, is the standard deviation of the position fluctuation of the candidate acupoint output by visual SLAM in consecutive frames, reflecting the stability of the spatial position of the acupoint, is the standard deviation of the normal vector direction fluctuation of the candidate acupoint in consecutive frames, reflecting the direction stability of the acupoint, 、 are the threshold parameters for position stability and direction stability; S54. According to the stability criterion function adjust the spatial guidance multi - dimensional preference scoring function to obtain the dynamic stable matching preference scoring function : ; Under the action of the dynamic stable matching preference scoring function, when the spatial stability of the candidate acupoint decreases, the matching priority of this point with the corresponding standard acupoint is automatically reduced; S55. Input the set of candidate acupoint points , the set of standard acupoints and the dynamic stable matching preference scoring function into the improved Gale-Shapley stable matching algorithm for iterative matching calculation, and output the set of final stable pairing results . For all the matching pairs in the set of final stable pairing results , there does not exist another unpaired combination such that the following two conditions are simultaneously satisfied: The candidate acupoint prefers the standard acupoint in its dynamic stable preference , over its matched object ; The standard acupoint prefers the candidate acupoint in its dynamic stable preference , over its matched object ; If the pairing pair does not exist, it means that the current matching pair is stable and there is no destructive preference recombination: ; Among them, , respectively represent the priority ranking relationship between the candidate acupoint and the standard acupoint determined by the dynamic stable matching preference scoring function. The dynamic stable condition ensures that each stable pairing result not only has no cross obstruction in the preference dimension, but also meets the matching requirements for the real-time position stability of visual SLAM.

[0032] In this embodiment, S6 includes the following steps: S61. Based on the set of stable pairing results in the main reference frame of the skin coordinate system , taking the candidate acupoint as the needle insertion target point, extract the normal vector of the candidate acupoint from the three-dimensional skin mesh model , and define the normal vector of the candidate acupoint as the default needle insertion direction reference; S62. Establish a local subcutaneous three-dimensional space search area at each candidate acupoint . The three-dimensional space search area is a spherical volume space with the candidate acupoint as the center of the sphere and the maximum needle insertion depth as the radius, serving as the search range for needle insertion path planning; S63. In the local subcutaneous three-dimensional space search area Among them, path evaluation is performed on all feasible needle insertion paths; S64. Among the candidate paths that meet all the constraint conditions, select the needle insertion path that is most consistent with the normal direction of the candidate acupoint as the target needle insertion path , where the target needle insertion path represents a three-dimensional vector line segment extending from the acupoint on the skin surface to the target depth along a predetermined direction; S65. Map and attach the target needle insertion path to the vertex structure on the three-dimensional skin mesh model , and establish a trajectory correspondence with the standard acupoint in the pairing result to form a set of needle insertion guides that can be used for augmented reality visualization overlay.

[0033] In this embodiment, the path evaluation is screened based on the following three constraint rules: If the angle between the needle insertion path and the normal vector of the candidate acupoint meets , it is considered to meet the needle insertion angle limit; If the length of the needle insertion path meets , it is considered to meet the needle insertion depth limit; If the minimum distance between the needle insertion path and any blood vessel point in the three-dimensional subcutaneous vascular mesh model on the path segment meets , it is considered to meet the vascular avoidance condition; where represents the angle between the candidate needle insertion path and the normal vector of the candidate acupoint and is used to measure whether the needle insertion path deviates from the skin surface normal, represents the maximum allowable deviation angle threshold of the needle insertion direction, which is the tolerance range set for the needle deviating from the skin normal. A path exceeding the maximum deviation angle threshold is considered not to meet the needle insertion angle limit, represents the actual path length of the candidate needle insertion path, defined as the spatial distance from the candidate acupoint along the candidate path direction to the termination point, represents the minimum allowable needle insertion depth threshold of the needle insertion path, which is the set lower limit value to avoid insufficient needle insertion depth and inability to achieve the treatment effect, represents the maximum allowable needle insertion depth threshold of the needle insertion path,

[0034] Example 1: In the teaching demonstration room of the Acupuncture and Tuina Department of Dongzhimen Hospital of A University of Traditional Chinese Medicine, Professor Zhang, the clinical instructor, is organizing 7 postgraduate students to conduct clinical practical teaching. The practical task is "Rapid Location of Acupoints in the Scapular Region and Planning of Safe Needling Paths". The teaching object is a female volunteer (number P-032) who is thin, with clear scapulas, but has loose superficial muscles and is prone to sliding and dislocation.

[0035] The first student, Li, used the traditional manual comparison method to align the acupoints. He pasted a static scapular template print on the patient's back, and after manual measurement, aligned the Jianzhongshu and Tianzong acupoints. The operation time was 92 seconds. According to the subsequent MRI-assisted verification, in this operation, the acupoint deviated more than 7.6 mm from the center of the standard acupoint, and the deviation angle between the needling direction and the normal direction was 12.3°, causing the volunteer to feedback "obvious local muscle twitching after needling", and the subjective pain score was 7 points.

[0036] Another student, Wang, used the present invention to assist in completing the same task. Wang started the augmented reality head-mounted device, collected data on the patient's right shoulder through the RGB-D camera and the IMU module, and the SLAM system completed skin mesh mapping within 2.1 seconds, identifying the bony landmark points of the scapular spine and scapular angle.

[0037] The system automatically generated a three-dimensional skin model, and projected a path conforming to the topological structure of the Large Intestine Meridian in the patient's scapular region under the reference of the bony landmarks. Initially, 24 candidate acupoint points were generated, the SLAM confidence was 0.93, and the Gale-Shapley matching module was started, and the matching result was output within 137 milliseconds through five rounds of iteration. Among them, the candidate point was successfully matched with the standard acupoint (i.e., Tianzong), the matching preference score was 3.82, the stability factor, and there was no obstructive pair.

[0038] The system planned a needling path with an incident angle of 18.4°, a depth of 29.7 mm, and a blood vessel avoidance distance of 5.4 mm based on the normal direction of the candidate point, the stability of the SLAM pose sequence, and the needle-free area map. The AR display superimposed a transparent guiding path in real time, and Wang performed needling according to the system prompt. During the operation, the system detected a small movement of the patient's scapula (the attitude change angle was 2.6°), triggered the recalculation of the trajectory deviation and completed the automatic matching update within 380 milliseconds, and the path was refreshed.

[0039] Wang successfully completed the needling. The system recorded that the actual error of the needling path was 2.1 mm and the angle deviation was 3.7°. The volunteer feedback "the needling point is comfortable and there is no discomfort", and the subjective pain score was 1 point. The total operation time was 51 seconds, the number of automatic corrections was 1 time, there was zero manual calibration, and the path drift did not exceed the limit.

[0040] In this practical training course, for 7 groups of comparative practical tasks in the same area and under the same body surface conditions as above, the system recorded in total: Table 1 Data Comparison between the Invention and the Traditional Comparison Method

[0041] Meanwhile, the system also recorded the stable matching calculation time (average 0.13 seconds), SLAM mapping initialization time (average 2.5 seconds), and path refresh response time (maximum 0.42 seconds) during the complete operation process. The data shows that the system of the present invention demonstrates excellent speed, stability, and precision control capabilities in real dynamic body surface scenarios. Under common clinical environment disturbances such as slight patient movement and operation interruption, the system can still quickly adjust the matching results and visible paths, significantly reducing misoperations and repeated calibrations.

[0042] The successful application of the embodiment not only improves the standardization level of acupuncture teaching but also provides an empirical sample and technical model for subsequent clinical acupuncture navigation in complex regions (neck, waist, sacrum). More data will be used to train the dynamic stability model and acupoint personalized parameter recommendation module in the system to further expand the model's capabilities.

[0043] The present invention proposes a dynamic stable Gale-Shapley matching model for acupuncture scenarios to solve the real-time one-to-one pairing problem under complex preference constraints. By introducing a space-guided multi-dimensional preference scoring function and a dynamic stability adjustment function, the matching between standard acupoints and candidate acupoints is extended from static Euclidean distance to an integrated optimization index that comprehensively considers skin geometry consistency, acupuncture point prohibition area penalty, disease relevance, and SLAM scene stability. The priority ranking is dynamically adjusted during the iterative matching process, effectively overcoming the dependence of traditional stable matching methods on static preference assumptions in visual dynamic environments, and achieving one-to-one stable pairing of globally unobstructed pairs while meeting medical semantic registration requirements, improving the spatial accuracy and clinical applicability of registration.

[0044] The present invention constructs a three-dimensional acupoint scattering mechanism based on skin geometry and visual stability to enhance the distribution rationality and robustness of the matching space. When generating a set of candidate acupoint points, an adaptive density function is constructed using skin mesh curvature response and bone-guided intensity, and further combined with the pose confidence degree output by SLAM for dynamic adjustment, enabling the scattering result to automatically increase the distribution density in areas with poor visual stability, thereby enhancing the system's robustness to actual scene disturbances such as occlusion, light changes, or slight patient movement at the spatial level, and realizing a joint constraint scattering strategy between skin anatomical structure and mapping accuracy.

[0045] The present invention realizes a real-time needle insertion path planning method that combines the three elements of angle limitation, depth boundary, and blood vessel avoidance. Based on the pairing result, a three-dimensional path planning mechanism is proposed, which comprehensively considers the three constraint conditions of the included angle of the needle insertion angle, the depth range of the needle tool, and the blood vessel distance threshold, constructs a search area for the needle insertion space that can be controlled and avoid obstacles, and finally the output needle insertion trajectory is real-time associated with the three-dimensional skin mesh model and establishes a trajectory mapping relationship with the standard acupoints, significantly improving the safety and accuracy of clinical operations.

[0046] As described above, the above are only the preferred specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, makes equivalent substitutions or changes, and all should be covered by the protection scope of the present invention.

Claims

1. An intelligent assisted acupuncture system based on AR technology, characterized in that: Includes the following modules: A multimodal perception module, used to collect a multimodal perception data set containing patient surface information; The visual SLAM guidance module is used to perform visual SLAM initialization, build a unified spatial reference between the world coordinate system, the camera coordinate system and the skin coordinate system, and output the relative pose between the camera and the patient's skin; A 3D reconstruction module, used to reproject the point cloud in the multi-frame depth image frame sequence to the skin coordinate system based on the relative pose to generate a 3D skin mesh model; A candidate acupuncture point generation module is used to map the standard meridian path to the skin surface to form a meridian mapping path in the skin space topological coordinate domain based on the three-dimensional skin mesh model and the bone marker coordinate set, and generate a candidate acupuncture point set under the conditions of satisfying the density threshold and the minimum spacing; A stable matching module is used to construct an initial two-way preference list based on the candidate acupoint set and the standard acupoint knowledge base, construct a dynamic stable matching preference scoring function, execute the improved Gale-Shapley stable matching algorithm, and output a stable pairing result set; A trajectory planning module is used to plan a three-dimensional needle insertion trajectory that meets the needle insertion angle restriction, needle insertion depth restriction and vascular avoidance conditions in the local subcutaneous space of the candidate acupuncture points according to the stable pairing result set, and bind the needle insertion trajectory to the three-dimensional skin mesh model to form a needle insertion guide set; The augmented reality rendering module is used to superimpose the needle guide set on the patient's skin surface in real time, and perform real-time visual rendering of virtual acupuncture point targets and needle insertion trajectories through the augmented reality display terminal.

2. An intelligent assisted acupuncture method based on AR technology, used to execute the intelligent assisted acupuncture system based on AR technology according to claim 1, characterized in that: The steps include: S1. Collect a multimodal perception dataset containing patient surface information; S2. Perform visual SLAM initialization based on the multimodal perception dataset, establish a unified spatial reference of the world coordinate system-camera coordinate system-skin coordinate system, and output the initial relative pose between the camera and the patient's skin; S3. Generate a three-dimensional skin mesh model based on the visual SLAM initialization result, and extract a bone marker coordinate set based on the three-dimensional skin mesh model; S4. Automatically generate a set of candidate acupuncture points based on the three-dimensional skin mesh model and the bone marker coordinate set combined with meridian topology rules; S5. Read the meridian name, disease relevance, and acupuncture-prohibited area information from the preset standard acupoint knowledge base, construct a bidirectional preference list corresponding to the candidate acupoint set, input the candidate acupoint set and the standard acupoint knowledge base into the Gale-Shapley stable matching algorithm, and output the matching result; S6. Calculate the needle insertion trajectory that meets the needle insertion angle limit, needle insertion depth limit and vessel avoidance conditions based on the pairing results, and associate the needle insertion trajectory with the three-dimensional skin mesh model; S7. Render a semi-transparent virtual acupuncture point target and a visible needle insertion path in the augmented reality display terminal according to the pairing result and the needle insertion trajectory, so as to realize the real-time superposition display of the virtual content and the real skin surface, and refresh the virtual acupuncture point target and the needle insertion trajectory in real time according to the updated pairing result until the whole acupuncture process is completed.

3. The intelligent assisted acupuncture method based on AR technology according to claim 2, characterized in that: The S2 comprises the following steps: S21. Input the multimodal perception dataset into the visual SLAM guidance module, the multimodal perception dataset includes a color image frame sequence , depth image frame sequence And the attitude data stream output by the inertial measurement unit ,in The time frame index, color image frame sequence, depth image frame sequence and attitude data stream output by the inertial measurement unit are used to synchronously generate the initial image reference frame set of the acupuncture scene ; S22. Initial image reference frame set in acupuncture scene Extract feature point set , and construct the skin area weighted image feature mask based on the curvature of the acupuncture skin area , the skin area weighted image feature mask sets the skin area as the priority area for feature point selection, forming a subset of skin area feature points ; S23. Subset of skin region feature points Perform robust registration and compute the skin-stable pose estimate matrix : ; in, Indicates the first The patient's skin surface at the frame moment is in the world coordinate system The spatial pose under is the skin stable pose estimation matrix, Represents the spatial rigid body transformation matrix from three-dimensional Euclidean space to affine transformation group, including rotation and translation components, , Respectively represent the first pairing that is successful in the current frame and the previous frame. The image coordinate positions of skin feature points are used to construct feature correspondences between frames. , Respectively represent Frame and The depth value of the frame at the corresponding image point position, , are the first and last frames in the current frame and the The unit normal vector corresponding to the skin point is It is the M-estimator loss function in the robust estimation, which is used to optimize the influence of abnormal errors caused by hand occlusion and light reflection on the pose estimation. is the number of priority feature point subsets in the skin area, indicating the number of valid registration points that can be used for pose calculation in the current acupuncture reference area; S24. Stabilize the skin pose estimation matrix Fusion pose with camera inertia Combine constraints to get the camera-skin relative pose transformation matrix , the camera-skin relative pose transformation matrix represents the dynamic spatial relationship between the camera and the skin in the acupuncture scene S25. Using the camera-skin relative pose transformation matrix Construct the main reference frame of the skin coordinate system .

4. The intelligent assisted acupuncture method based on AR technology according to claim 3 is characterized in that: The S3 comprises the following steps: S31. Based on the camera-skin relative pose transformation matrix Skin coordinate system main reference frame The skin points of the acupuncture area contained in the multi-frame depth image frame sequence are reprojected, and the depth information of each frame is uniformly mapped to the skin coordinate system to form skin point cloud data in a continuous space. Grid processing is performed on the basis of the skin point cloud data to construct a three-dimensional skin mesh model. ; S32. In 3D skin mesh model In the above method, by analyzing the curvature gradient and normal consistency of the mesh vertices, the locations with significant geometric protrusion changes are identified and extracted as a set of bone marker candidate points. , for each bone marker candidate point in the bone marker candidate point set, calculate its principal curvature response value, and find the point pair with the largest curvature difference in its neighborhood. If the maximum curvature difference exceeds the set curvature threshold, the point pair is considered to be a bone marker point; S33. Screen the extracted bone marker candidate point set to form a bone marker coordinate set ,The screening conditions are: the spatial distance between the bony markers must be greater than the preset minimum distance threshold; the angle between the normal vectors corresponding to the bony markers must be less than the set angle threshold; S34. Set the bone marker coordinates Mapped to the camera coordinate system, we get the corresponding set of observation projection points , the observed projection points are tracked consistently in continuous image frames, and the bone alignment error function is constructed according to the projection error. The camera inertial fusion pose of the current frame is optimized by minimizing the bone alignment error function to obtain the optimized bone constraint pose .

5. The intelligent assisted acupuncture method based on AR technology according to claim 4 is characterized in that: The three-dimensional skin mesh model is composed of a vertex set, an edge set and a corresponding normal vector set. The vertex set represents the position of each reconstructed point on the skin surface, and the normal vector set represents the change in the curvature direction of each vertex on the skin surface.

6. The intelligent assisted acupuncture method based on AR technology according to claim 5, characterized in that: The S4 comprises the following steps: S41. Based on 3D skin mesh model and bony landmark coordinate sets , in the skin coordinate system main reference frame Construct skin space topological coordinate domain The skin space topological coordinate domain refers to the continuous space region composed of all mesh vertices in the skin area centered on all bony markers and limited to the bony control radius. The continuous space region serves as the basic space range for scattering candidate acupoints. S42. Represent each standard meridian path in the meridian topology information library as a point sequence composed of a number of standard acupuncture points, and project the standard meridian path in the skin space topological coordinate domain to generate a meridian mapping path located on the surface of the three-dimensional skin mesh model. The meridian mapping path remains consistent with the original meridian direction, but is limited by local geometric changes of the skin and is embedded in the actual skin mesh surface structure; S43. Calculate the skin curvature response value and the bone guidance strength value of each vertex in the skin mesh on the meridian mapping path. The skin curvature response value reflects the degree of change of the curvature of the local skin. The bone guidance strength value represents the inverse relationship between the distance between the current vertex and the bone marker coordinate set. Perform weighted summation on the skin curvature response value and the bone guidance strength value to form a candidate acupoint density function. , used to quantify the potential of each skin point to generate candidate acupuncture points; S44. Estimate confidence based on the camera pose of the current frame Density function of candidate acupoints Perform adaptive adjustment to form an adaptive scattering density function When the camera pose estimation confidence is higher than the threshold, the adaptive point density function tends to the candidate acupoint density function itself. When the camera pose estimation confidence decreases, the adaptive point density function tends to the candidate acupoint density function itself. Increase candidate density; S45. On each meridian mapping path, according to the adaptive point density function Perform point sampling on the skin mesh surface to generate a set of candidate acupuncture points , only when the scattering density of a certain point is higher than the set density lower limit threshold , and the distance between the scattered points is greater than the minimum acupoint spacing threshold The candidate acupoints are retained as legal candidate acupoints.

7. The intelligent assisted acupuncture method based on AR technology according to claim 6, characterized in that: The S5 comprises the following steps: S51. Based on the candidate acupoint set and the standard acupoint set in the standard meridian acupoint knowledge base, construct an initial two-way preference list between the candidate acupoints and the standard acupoints, the initial two-way preference list respectively defining the initial matching tendency of the candidate acupoints and the standard acupoints to each other; S52. Based on the acupuncture-assisted spatial guidance scenario, the single Euclidean distance preference calculation method in the Gale-Shapley stable matching algorithm is improved, and the skin geometric consistency measurement under spatial constraints is introduced With dynamic safety constraint penalty function , forming an improved spatial guided multidimensional preference scoring function : ; in, Indicates the main reference frame of the skin coordinate system between the candidate acupoints and the standard acupoints The geometric Euclidean distance under Represented as a 3D skin mesh model The local curvature change of is used as the evaluation index to quantify the local skin geometry consistency between the candidate acupoints and the standard acupoints on the skin surface. It represents the dynamic safety constraint penalty function established according to the forbidden needle area information. When the candidate acupuncture point is located in or near the forbidden needle area, the safety penalty coefficient of the pairing is increased. Indicates standard acupuncture points Treatment relevance scores for specific conditions, is the weighting coefficient of different preference indicators; S53. Bootstrapping multidimensional preference scoring functions in improved space Based on this, the stability criterion function is introduced Dynamically modify the two-way preference list of candidate acupoints and standard acupoints: ; in, Candidate acupuncture points output in real time by visual SLAM The standard deviation of position fluctuation in consecutive frames reflects the stability of the spatial position of the acupuncture points. Candidate acupuncture points The standard deviation of the normal vector direction fluctuation in consecutive frames reflects the stability of the acupoint direction. , is the threshold parameter of position stability and directional stability; S54. According to the stability criterion function Adjusting the spatial bootstrap multidimensional preference scoring function , and obtain the dynamic stable matching preference scoring function ,Under the action of the dynamic stable matching preference scoring function, when the spatial stability of the candidate acupuncture point decreases, the matching priority of the point with the corresponding standard acupuncture point is automatically reduced; S55. Set the candidate acupuncture points , Standard acupoint collection And dynamic stable matching preference scoring function Input to the improved Gale-Shapley stable matching algorithm for iterative matching calculation, and output the final stable pairing result set , for all the final stable pairing result sets The matching pairs in , there is no other pair that has not been paired , so that the following two conditions are satisfied at the same time: Candidate acupuncture points Prefers standard acupuncture points in its dynamic stability preference , outperforms its matched counterpart ; Standard acupuncture points Favoring candidate acupuncture sites in their dynamic stability preferences , outperforms its matched counterpart ; If the matching pair does not exist, it means that the current matching is stable, without disruptive preference reorganization.

8. The intelligent assisted acupuncture method based on AR technology according to claim 7, characterized in that: The S6 comprises the following steps: S61. Based on stable pairing result set In the skin coordinate system, the main reference frame Next, candidate acupuncture points For the needle insertion target point, the 3D skin mesh model Extract candidate acupoint normal vectors from , and define the normal vector of the candidate acupuncture point as the default needle insertion direction reference; S62. At each candidate acupuncture point Establish a local subcutaneous three-dimensional space search area The three-dimensional space search area is centered on the candidate acupuncture point and the maximum needle insertion depth. The spherical volume space with a radius of 2 is used as the search range for needle path planning; S63. Search area in local subcutaneous three-dimensional space In the experiment, all traversable needle paths were evaluated; S64. Among the candidate paths that satisfy all constraints, select the needle insertion path that is most consistent with the normal direction of the candidate acupuncture point as the target needle insertion path , the target needle insertion path represents a three-dimensional vector line segment extending from the acupuncture point on the skin surface to the target depth along a predetermined direction; S65. Target needle path Map and attach to 3D skin mesh model The vertex structure on the right and the standard acupuncture points in the matching results Trajectory correspondence is established to form a needle insertion guide set that can be used for augmented reality visualization overlay.

9. The intelligent assisted acupuncture method based on AR technology according to claim 8, characterized in that: The path evaluation is filtered according to the following three constraints: If the needle insertion path and the normal vector of the candidate acupoint Angle satisfy , it is considered to meet the needle insertion angle limit; If the needle path length satisfy , it is considered to meet the needle insertion depth limit; If the shortest distance between the needle insertion path and any blood vessel point in the three-dimensional subcutaneous vascular mesh model on the path segment satisfy , then the vascular avoidance condition is considered to be met; in, Represents the candidate needle insertion path and the candidate acupoint normal vector The angle between the two is used to measure whether the needle insertion path deviates from the normal direction of the skin surface. Indicates the maximum deviation angle threshold allowed for the needle insertion direction, which is the set tolerance range for the needle to deviate from the skin normal. Paths exceeding the maximum deviation angle threshold are considered to not meet the needle insertion angle limit. The actual path length of the candidate needle insertion path is defined as the distance from the candidate acupoint to the The spatial distance between the candidate path and the end point. Indicates the minimum needle insertion depth threshold allowed by the needle insertion path. The lower limit is set to avoid the needle being inserted too shallowly and failing to achieve the treatment effect. Indicates the maximum needle insertion depth threshold allowed by the needle insertion path. It represents the minimum distance between the needle insertion path and the nearest blood vessel point in the three-dimensional subcutaneous vascular mesh model, and is used to measure whether the needle insertion path is close to the blood vessel. It indicates the minimum safe distance threshold between the needle insertion path and the blood vessel, and is the lower limit standard for vascular avoidance set by the system.

Images