Three-dimensional acupoint recognition interaction method and terminal based on end-side calculation and bone degree division method

Abstract

The invention discloses a three-dimensional acupoint recognition interaction method and terminal based on end-side calculation and a bone degree division method, and belongs to the technical field of traditional Chinese medicine acupoint recognition. The method comprises the steps of collecting video stream data and performing preprocessing, performing 3D face grid reconstruction according to image frame data based on a lightweight deep neural network to obtain 3D face grid data, performing bone degree reference calculation based on anatomical mark points based on the 3D face grid data to obtain a personalized bone degree unit value of a user and an anatomical reference coordinate system, and obtaining a personalized bone degree unit value of the user and an anatomical reference coordinate system. The method comprises the following steps: acquiring image frame data of a target acupoint, performing three-dimensional coordinate calculation of the target acupoint based on the image frame data and a preset acupoint positioning rule to obtain a current frame three-dimensional coordinate of the target acupoint, performing coordinate time sequence smoothing and AR rendering processing based on the current frame three-dimensional coordinate to obtain an AR rendered image superposed with an acupoint identifier, and performing gesture recognition and interactive feedback based on the image frame data and the AR rendered image superposed with the acupoint identifier. According to the method, the traditional Chinese medicine bone degree division method and AR interaction are fused, and accurate and real-time personalized recognition and visual guidance of acupuncture points are achieved.

Description

Technical Field

[0001] This invention relates to the field of TCM acupoint recognition technology, specifically to a three-dimensional acupoint recognition interactive method and terminal based on end-side calculation and bone measurement method. Background Technology

[0002] Currently, technologies using computer vision to assist in the identification of acupoints in Traditional Chinese Medicine can be mainly divided into two categories: 1. Static image overlay type: Directly overlays general acupoint images onto the user's face image.

[0003] 2. Cloud-based image recognition: The client collects images and uploads them to the cloud server. The server then uses a 2D image recognition algorithm (such as minigooglenet) to reconstruct the acupoint coordinates and returns them to the client.

[0004] Specific technical problems with existing solutions: 1. High latency and privacy leakage risks: Existing technology relies on cloud servers for processing, and video stream uploads result in high network latency, making AR tracking unresponsive; moreover, uploading facial biometric data to the cloud poses serious privacy and compliance risks.

[0005] 2. Limited recognition dimensions and insufficient accuracy: Existing technologies are mostly based on 2D planar image analysis. When the user's head rotates (turns to the side, looks up), the planar coordinates cannot be accurately mapped to the three-dimensional face, resulting in the drift and inaccuracy of acupoint marking positions.

[0006] 3. Lack of personalized adaptation (a thousand faces for a thousand people): Existing technologies mostly use fixed model positions and do not incorporate the core TCM theory of "bone measurement" (i.e., using the distance of the user's own anatomical features as the unit), which cannot adapt to the differences of people with different face shapes such as long face and round face.

[0007] 4. Lack of interactive feedback: Existing technology can only display the location on the screen and cannot determine whether the user's finger has actually pressed the exact location, lacking closed-loop guidance.

[0008] To address the aforementioned issues, there is an urgent need for a three-dimensional acupoint recognition and interaction method based on end-side calculation and bone measurement to solve the problems existing in traditional methods. Summary of the Invention

[0009] The purpose of this invention is to provide a three-dimensional acupoint recognition and interaction method and terminal based on end-side calculation and bone measurement method. The method is completed locally on the terminal, protecting user privacy, and integrates traditional Chinese medicine bone measurement method with AR interaction to achieve accurate, real-time personalized recognition and visual guidance of acupoints.

[0010] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A three-dimensional acupoint recognition interactive method based on end-side calculation and bone measurement includes: Step 1: Acquire video stream data and preprocess it to obtain image frame data; Step 2: Reconstruct 3D face meshes based on image frame data using a lightweight deep neural network to obtain 3D face mesh data containing depth information. The lightweight deep neural network includes a feature extraction backbone, a feature fusion layer, a 3D coordinate regression head, and a coordinate mapping module. The feature extraction backbone is a Backbone-improved MobileNetV3 structure. Step 3: Calculate the bone datum based on anatomical landmarks using 3D face mesh data to obtain the user's personalized bone datum unit value and anatomical reference coordinate system; Step 4: Based on the user's personalized bone unit value, anatomical reference coordinate system and preset acupoint positioning rules, calculate the three-dimensional coordinates of the target acupoint to obtain the current frame three-dimensional coordinates of the target acupoint. Step 5: Perform coordinate temporal smoothing and AR rendering based on the current frame's 3D coordinates to obtain an AR rendered image with acupoint markers superimposed. Step 6: Perform gesture recognition and interactive feedback based on the AR rendered image with image frame data and overlaid acupoint markers.

[0011] Further, in step 1, video stream data is acquired and preprocessed to obtain image frame data, specifically as follows: Image frame data is acquired in real time by the image acquisition module, and histogram equalization and resolution adjustment are performed on it to obtain image frame data.

[0012] Furthermore, the specific improvements to the Backbone-Improved MobileNetV3 architecture are as follows: Based on the traditional MobileNetV3 architecture, some global average pooling layers are replaced with a lightweight attention mechanism.

[0013] Furthermore, in step 3, a bone datum calculation based on anatomical landmarks is performed using 3D face mesh data to obtain the user's personalized bone datum unit value and anatomical datum coordinate system, specifically: Extract the coordinates of anatomical landmarks from 3D face mesh data and construct an anatomical reference coordinate system; Based on the theory of bone measurement, the Euclidean distance or geodesic distance between the marker points is calculated, and then divided by the standard measurement unit to obtain the user's personalized bone measurement unit value.

[0014] Further, in step 4, based on the user's personalized bone unit value, anatomical reference coordinate system, and preset acupoint positioning rules, the three-dimensional coordinates of the target acupoint are calculated to obtain the current frame's three-dimensional coordinates of the target acupoint, specifically: Based on preset acupoint positioning rules, in the anatomical reference coordinate system, starting from the anatomical landmark point, the distance of multiple personalized bone unit values ​​is moved along the normal direction of the facial surface to calculate the current frame three-dimensional coordinates of the target acupoint.

[0015] Further, in step 5, coordinate temporal smoothing and AR rendering processing are performed based on the current frame's 3D coordinates to obtain an AR rendered image with overlaid acupoint markers, specifically: Based on the Kalman filter algorithm, combined with the coordinate state of the previous frame, the current frame's three-dimensional coordinates of the target acupoint are denoised and smoothed to eliminate detection jitter. The smoothed three-dimensional coordinates are then mapped back to the screen's two-dimensional pixel coordinates to obtain an AR rendered image with acupoint labels superimposed.

[0016] Furthermore, in step 6, gesture recognition and interactive feedback are performed based on the AR rendered image with image frame data and overlaid acupoint markers, specifically as follows: Based on a parallel-running hand detection model, the system identifies the three-dimensional coordinates of the user's index fingertip, calculates the spatial Euclidean distance between the fingertip coordinates and the acupoint coordinates, and determines whether the Euclidean distance is less than a preset threshold. If it is less, a feedback command is generated; if it is greater than the threshold, no trigger is given.

[0017] This invention also provides a three-dimensional acupoint recognition interactive terminal based on end-side calculation and bone measurement method, applied to the above-mentioned three-dimensional acupoint recognition interactive method based on end-side calculation and bone measurement method, including: The image acquisition and preprocessing module is used to acquire video stream data and preprocess it to obtain image frame data. The 3D face mesh reconstruction module is connected to the image acquisition and preprocessing module. It is used to reconstruct the 3D face mesh based on the image frame data using an integrated lightweight deep neural network, so as to obtain 3D face mesh data containing depth information. The bone datum calculation module is connected to the 3D face mesh reconstruction module and is used to perform bone datum calculation based on anatomical landmarks based on the 3D face mesh data to obtain the user's personalized bone datum unit value and anatomical datum coordinate system. The acupoint coordinate calculation module is connected to the bone reference calculation module. It is used to calculate the three-dimensional coordinates of the target acupoint based on the personalized bone unit value, the anatomical reference coordinate system and the preset acupoint positioning rules, so as to obtain the current frame three-dimensional coordinates of the target acupoint. The coordinate optimization and AR rendering module is connected to the acupoint coordinate calculation module. It is used to perform coordinate temporal smoothing on the three-dimensional coordinates of the current frame and perform AR rendering on the smoothed coordinates to generate an AR rendering image with acupoint markers superimposed. The gesture recognition and interactive feedback module is connected to the image acquisition and preprocessing module and the coordinate optimization and AR rendering module, respectively. It is used to recognize user gestures based on the image frame data and generate interactive feedback instructions according to the spatial relationship between the gestures and acupoint markers.

[0018] Furthermore, the image acquisition and preprocessing module specifically includes: RGB cameras and depth cameras are used to acquire video stream data containing color and depth information in real time; The preprocessing unit performs histogram equalization and resolution adjustment on the acquired video stream data to obtain standardized image frame data.

[0019] In summary, the present invention has at least one of the following beneficial technical effects: 1. Privacy and data security protection: It adopts an edge computing architecture, in which all image acquisition, feature extraction and inference processes are completed in the user's local memory. Data is not uploaded to the server, which solves the risk of leakage of facial biometric data and complies with data security regulations.

[0020] 2. High precision and anti-interference: It adopts 3D face mesh reconstruction technology instead of traditional 2D image recognition. Even when the face is turned to the side, the lighting changes or the expression changes, the acupoints can follow the skin as accurately as if they are "attached" to it, achieving precise positioning with 6 degrees of freedom.

[0021] 3. Personalized and precise acupoint selection: Combining the adaptive algorithm of bone measurement in traditional Chinese medicine, the cun is defined by calculating the proportion of anatomical landmarks. It can automatically adapt to the differences in facial shapes of different groups such as adults, children, long faces, and round faces, and restore the scientific acupoint selection method of traditional Chinese medicine body measurement.

[0022] 4. Interactive guidance loop: The introduction of a 3D distance judgment mechanism between fingertips and acupoints allows users to not only see the acupoints, but also perceive whether they have pressed them accurately through vibration feedback, which greatly reduces the learning threshold for non-professionals. Attached Figure Description

[0023] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0025] like Figure 1 As shown, this invention provides a three-dimensional acupoint recognition and interaction method based on end-side calculation and bone measurement, including: Step 1: Acquire video stream data (Raw Data) and preprocess it to obtain image frame data; Step 2: Reconstruct 3D face meshes based on image frame data using a lightweight deep neural network to obtain 3D face mesh data containing depth information. The lightweight deep neural network includes a feature extraction backbone, a feature fusion layer, a 3D coordinate regression head, and a coordinate mapping module. The feature extraction backbone is a Backbone-improved MobileNetV3 structure. Step 3: Calculate the bone datum based on anatomical landmarks using 3D face mesh data to obtain the user's personalized bone datum unit value and anatomical reference coordinate system; Step 4: Based on the user's personalized bone unit value, anatomical reference coordinate system and preset acupoint positioning rules, calculate the three-dimensional coordinates of the target acupoint to obtain the current frame three-dimensional coordinates of the target acupoint. Step 5: Perform coordinate temporal smoothing and AR rendering based on the current frame's 3D coordinates to obtain an AR rendered image with acupoint markers superimposed. Step 6: Perform gesture recognition and interactive feedback based on the AR rendered image with image frame data and overlaid acupoint markers.

[0026] In step 1, video stream data is acquired and preprocessed to obtain image frame data, specifically as follows: Image frame data is acquired in real time by the image acquisition module and then subjected to histogram equalization to enhance image features under uneven lighting conditions. The resolution is then adjusted to the model input standard (e.g., 224x224 or 192x192) to obtain preprocessed image frame data. I pre .

[0027] In step 2, a 3D face mesh is reconstructed based on the image frame data using a lightweight deep neural network to obtain 3D face mesh data containing depth information, specifically as follows: Will I preThe input is a lightweight deep neural network pre-installed on the terminal. This network does not rely on the cloud and directly outputs the dense keypoint topology of the face surface, resulting in 3D face mesh data containing depth information. M face (A set of X, Y, Z coordinates containing 468 vertices); Next, we will introduce lightweight deep neural networks: It mainly consists of the following four modules, which realize the transformation from image to 3D coordinates through "value transfer": 1. Feature extraction backbone (Backbone - Improved MobileNetV3): Its input is the preprocessed image frame. I pre (e.g. 224) 224 3); The processing involves using depthwise separable convolution and a Squeeze-and-Excitation (SE) module. To improve edge efficiency, some global average pooling layers are replaced with a lightweight attention mechanism.

[0028] Used to extract multi-scale semantic features from images, such as the contour features of anatomical key points like the corners of the eyes and the tip of the nose.

[0029] 2. Feature Fusion Layer (Neck): By fusing deep high-level semantic features with shallow spatial detail features, the stability of the features can be maintained even when the head rotates significantly.

[0030] 3. Coordinate Regression Head: Input: The fused feature vector.

[0031] Processing: The relative coordinate offsets of the 468 vertices are directly regressed through a fully connected layer (FC).

[0032] The core of the "face mesh" is used to directly output the three-dimensional values ​​of X, Y, and Z without going through an intermediate heatmap.

[0033] 4. Coordinate Mapping Module: The regressed relative coordinates are combined with the camera's intrinsic parameter matrix and converted into dense grid data in three-dimensional space. M face .

[0034] This invention describes the calculation process of depthwise separable convolution, which is used to reduce the number of parameters, as follows: (1) This formula describes the computation process of depthwise separable convolution in the feature extraction backbone.

[0035] In the formula, : Output feature map in coordinates ( The pixel value or feature value at that location. The depthwise kernel is located at... The weight parameters at that location. : Input feature map. : Output spatial coordinate index of the feature map. i , j : Spatial offset index of the convolution kernel.

[0036] The network structure of this invention employs a Hard-swish activation function to optimize the computation time at the edge, as follows: (2) This formula is used to optimize the computation time on the edge side and is a non-linear activation function used in the MobileNetV3 architecture.

[0037] In the formula, Input feature values. : Modified linear unit function, i.e. min(max(0,x),6) . : The calculated activation output value.

[0038] The network structure of this invention uses a weighted mean squared error (MSE) loss function, assigning higher weights to key regions such as the eyes and mouth. w i ,for: (3) This formula is used to train the network, assigning higher weights to key areas such as the eyes and mouth.

[0039] In the formula, : Total loss function value. i Key point index, traversing vertices 1 to 468.

[0040] : No. i The weighting coefficients of key points (giving higher weight to key areas such as the eyes and mouth). , , : No. i The true coordinate labels of each key point. , , The network predicted the first i The three-dimensional coordinates of the key points.

[0041] In step 3, a bone datum calculation based on anatomical landmarks is performed using 3D face mesh data to obtain the user's personalized bone datum unit value and anatomical reference coordinate system, specifically: From 3D face mesh data M face Extract the coordinates of anatomical landmarks, such as the glabella. P glabella The midpoint of the front hairline P hairline An anatomical reference coordinate system was constructed by taking measurements of the bony points between the two mastoid processes. S ref ; Based on the theory of bone measurement, calculate the Euclidean distance or geodesic distance between landmarks. For example, calculate... D head =Distance( P glabella , P hairline ),Will D head Divide by the standard measurement unit (3 inches) to obtain the user's personalized bone measurement unit value. L unit ; The Euclidean distance (the basic unit of bone measurement) is: (4) This formula is used to calculate the Euclidean distance between specific anatomical landmarks.

[0042] In the formula, The Euclidean distance between the center of the forehead and the midpoint of the hairline. : the center of the forehead The three-dimensional spatial coordinates. : Midpoint of the anterior hairline ( The three-dimensional spatial coordinates of ).

[0043] Personalized bone measurement unit values L unit The definition of is: (5) This formula is based on the theory of bone measurement and converts the measured distance into personalized units.

[0044] In the formula, The user's personalized bone measurement unit value (i.e., the "1 inch" length unique to the user's face). : The calculated Euclidean distance from the center of the eyebrows to the midpoint of the hairline. 3 The standard bone measurement corresponding to this anatomical distance (i.e., the standard is set at 3 inches).

[0045] In step 4, based on the user's personalized bone unit value, anatomical reference coordinate system, and preset acupoint positioning rules, the three-dimensional coordinates of the target acupoint are calculated to obtain the current frame's three-dimensional coordinates of the target acupoint, specifically: From 3D face mesh data M face Pre-defined anatomical landmarks (such as pupil center coordinates) are used for positioning. P pupil Determine the reference starting point. P start ; Calculate the reference starting point P start The normal vectors of its neighboring vertices n Determine the tangent plane on the skin surface at that location, and use it as a reference for movement to construct a local tangent plane; The location of acupoints follows the contours of the face (such as the curvature from the wing of the nose to the corner of the mouth). The system does not move directly in a straight line, but instead searches for geodesic distances along the edges of a 3D mesh. N×L unit The distance; The calculated initial coordinates may deviate slightly from the skin surface due to mesh discretization. The system projects them back onto the nearest mesh triangular facet along the normal direction to finally obtain the current frame 3D coordinates of the target acupoint. C target (x, y, z) ; Target acupoint coordinates C target The calculation is as follows: Set the movement direction vector as (Determined by the positioning rules of Traditional Chinese Medicine, such as downward or outward), the preliminary coordinates are: (6) This formula calculates the preliminary three-dimensional coordinates of acupoints based on preset rules and bone units.

[0046] In the formula, : The preliminary three-dimensional coordinates of the target acupoint calculated. : The three-dimensional coordinates of the reference starting point (such as an anatomical landmark). : The direction vector of movement determined by the positioning rules of traditional Chinese medicine (such as downward or outward). : The number of bone measurements (inches) between the starting point and the target acupoint. : The user's personalized bone density unit value.

[0047] The geodesic path iteration (simplified representation) is as follows: To move on a curved surface, the following conditions must be met. ,in All in Mface In the set of vertices.

[0048] In step 5, coordinate temporal smoothing and AR rendering are performed based on the current frame's 3D coordinates to obtain an AR rendered image with overlaid acupoint markers, specifically: Using the Kalman filter algorithm, combined with the coordinate state of the previous frame, the... C target Denoising and smoothing processes are performed to eliminate detection jitter, and the smoothed 3D coordinates are then processed. C smooth Mapping back to the two-dimensional pixel coordinates on the screen yields an AR rendered image with acupoint markers superimposed.

[0049] In step 6, gesture recognition and interactive feedback are performed based on the image frame data and the AR rendered image with overlaid acupoint markers, specifically as follows: Based on a parallel-running hand detection model, the three-dimensional coordinates C of the user's index fingertip are identified. finger Specifically, it includes: 1. Parallel extraction of key hand points While running the face mesh model, the terminal locally launches a lightweight hand skeleton tracking network (such as the SSD detector based on MobileNet) to extract 21 3D key points of the hand. M hand ; Real-time locking of the 3D coordinates of the index fingertip ; 2. Multi-source coordinate system mapping Using real-time depth information from the camera or relative depth deduced from facial feature dimensions, the facial coordinate system is... S face With hand coordinate system S hand All data are uniformly mapped to a normalized three-dimensional spatial coordinate system with the camera as the origin. 3. Dynamic distance determination and collision detection Calculate the three-dimensional Euclidean distance between the fingertip and the target acupoint. D diff ,for D diff =|| C smooth -C finger ||;To adapt to different face shapes, trigger thresholds are set. Instead of setting a fixed physical value, it is based on the personalized bone measurement unit calculated in step 3. L unit hook up; This invention provides a three-dimensional Euclidean distance. D diff The detailed formula is as follows: (7) In the formula, The smoothed three-dimensional coordinates of the acupoints are output from step 5; This invention also provides an adaptive triggering discriminant as follows: (8) in, The tolerance coefficient (usually 0.2~0.5) ensures that the accuracy of the pressure detection matches the individual differences of the user (such as finger thickness, face size); This invention also provides temporal denoising logic. To avoid vibration feedback jitter caused by single-frame detection errors, a sliding window integral determination is adopted, as follows: (9) Only when Only when the value is >0.8 (i.e., multiple consecutive frames are successfully detected) will the feedback execution module be driven to output vibration or UI prompts.

[0050] This invention also provides a three-dimensional acupoint recognition interactive terminal based on end-side calculation and bone measurement method, applied to the above-mentioned three-dimensional acupoint recognition interactive method based on end-side calculation and bone measurement method, including: The image acquisition and preprocessing module is used to acquire video stream data and preprocess it to obtain image frame data. The 3D face mesh reconstruction module is connected to the image acquisition and preprocessing module. It is used to reconstruct the 3D face mesh based on the image frame data using an integrated lightweight deep neural network, so as to obtain 3D face mesh data containing depth information. The bone datum calculation module is connected to the 3D face mesh reconstruction module and is used to perform bone datum calculation based on anatomical landmarks based on the 3D face mesh data to obtain the user's personalized bone datum unit value and anatomical datum coordinate system. The acupoint coordinate calculation module is connected to the bone reference calculation module. It is used to calculate the three-dimensional coordinates of the target acupoint based on the personalized bone unit value, the anatomical reference coordinate system and the preset acupoint positioning rules, so as to obtain the current frame three-dimensional coordinates of the target acupoint. The coordinate optimization and AR rendering module is connected to the acupoint coordinate calculation module. It is used to perform coordinate temporal smoothing on the three-dimensional coordinates of the current frame and perform AR rendering on the smoothed coordinates to generate an AR rendering image with acupoint markers superimposed. The gesture recognition and interactive feedback module is connected to the image acquisition and preprocessing module and the coordinate optimization and AR rendering module, respectively. It is used to recognize user gestures based on the image frame data and generate interactive feedback instructions according to the spatial relationship between the gestures and acupoint markers.

[0051] The image acquisition and preprocessing module specifically includes: RGB cameras and depth cameras are used to acquire video stream data containing color and depth information in real time; The preprocessing unit performs histogram equalization and resolution adjustment on the acquired video stream data to obtain standardized image frame data.

[0052] Embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0053] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0054] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0055] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0056] Contents not described in detail in this specification are prior art known to those skilled in the art. It is hereby indicated that the above description is intended to help those skilled in the art understand this invention, but does not limit the scope of protection of this invention. Any equivalent substitutions, modifications, improvements, or simplifications of the above descriptions that do not depart from the essential content of this invention fall within the scope of protection of this invention.

Claims

1. A three-dimensional acupoint recognition interactive method based on end-side calculation and bone measurement, characterized in that, include: Step 1: Acquire video stream data and preprocess it to obtain image frame data; Step 2: Reconstruct 3D face meshes based on image frame data using a lightweight deep neural network to obtain 3D face mesh data containing depth information. The lightweight deep neural network includes a feature extraction backbone, a feature fusion layer, a 3D coordinate regression head, and a coordinate mapping module. The feature extraction backbone is a Backbone-improved MobileNetV3 structure. Step 3: Calculate the bone datum based on anatomical landmarks using 3D face mesh data to obtain the user's personalized bone datum unit value and anatomical reference coordinate system; Step 4: Based on the user's personalized bone unit value, anatomical reference coordinate system and preset acupoint positioning rules, calculate the three-dimensional coordinates of the target acupoint to obtain the current frame three-dimensional coordinates of the target acupoint. Step 5: Perform coordinate temporal smoothing and AR rendering based on the current frame's 3D coordinates to obtain an AR rendered image with acupoint markers superimposed. Step 6: Perform gesture recognition and interactive feedback based on the AR rendered image with image frame data and overlaid acupoint markers.

2. The three-dimensional acupoint recognition interactive method based on end-side calculation and bone measurement method according to claim 1, characterized in that, In step 1, video stream data is acquired and preprocessed to obtain image frame data, specifically as follows: Image frame data is acquired in real time by the image acquisition module, and histogram equalization and resolution adjustment are performed on it to obtain image frame data.

3. The three-dimensional acupoint recognition interactive method based on end-side calculation and bone measurement method according to claim 2, characterized in that, The specific improvements to the Backbone-Improved MobileNetV3 architecture are as follows: Based on the traditional MobileNetV3 architecture, some global average pooling layers are replaced with a lightweight attention mechanism.

4. The three-dimensional acupoint recognition interactive method based on end-side calculation and bone measurement method according to claim 3, characterized in that, In step 3, a bone datum calculation based on anatomical landmarks is performed using 3D face mesh data to obtain the user's personalized bone datum unit value and anatomical reference coordinate system, specifically: Extract the coordinates of anatomical landmarks from 3D face mesh data and construct an anatomical reference coordinate system; Based on the theory of bone measurement, the Euclidean distance or geodesic distance between the marker points is calculated, and then divided by the standard measurement unit to obtain the user's personalized bone measurement unit value.

5. The three-dimensional acupoint recognition interactive method based on end-side calculation and bone measurement method according to claim 4, characterized in that, In step 4, based on the user's personalized bone unit value, anatomical reference coordinate system, and preset acupoint positioning rules, the three-dimensional coordinates of the target acupoint are calculated to obtain the current frame's three-dimensional coordinates of the target acupoint, specifically: Based on preset acupoint positioning rules, in the anatomical reference coordinate system, starting from the anatomical landmark point, the distance of multiple personalized bone unit values ​​is moved along the normal direction of the facial surface to calculate the current frame three-dimensional coordinates of the target acupoint.

6. The three-dimensional acupoint recognition interactive method based on end-side calculation and bone measurement method according to claim 5, characterized in that, In step 5, coordinate temporal smoothing and AR rendering are performed based on the current frame's 3D coordinates to obtain an AR rendered image with overlaid acupoint markers, specifically: Based on the Kalman filter algorithm, combined with the coordinate state of the previous frame, the current frame's three-dimensional coordinates of the target acupoint are denoised and smoothed to eliminate detection jitter. The smoothed three-dimensional coordinates are then mapped back to the screen's two-dimensional pixel coordinates to obtain an AR rendered image with acupoint labels superimposed.

7. The three-dimensional acupoint recognition interactive method based on end-side calculation and bone measurement method according to claim 6, characterized in that, In step 6, gesture recognition and interactive feedback are performed based on the image frame data and the AR rendered image with overlaid acupoint markers, specifically as follows: Based on a parallel-running hand detection model, the system identifies the three-dimensional coordinates of the user's index fingertip, calculates the spatial Euclidean distance between the fingertip coordinates and the acupoint coordinates, and determines whether the Euclidean distance is less than a preset threshold. If it is less, a feedback command is generated; if it is greater than the threshold, no trigger is given.

8. A three-dimensional acupoint recognition interactive terminal based on end-to-end calculation and bone measurement method, applied to the three-dimensional acupoint recognition interactive method based on end-to-end calculation and bone measurement method as described in any one of claims 1-7, characterized in that, include: The image acquisition and preprocessing module is used to acquire video stream data and preprocess it to obtain image frame data. The 3D face mesh reconstruction module is connected to the image acquisition and preprocessing module. It is used to reconstruct the 3D face mesh based on the image frame data using an integrated lightweight deep neural network, so as to obtain 3D face mesh data containing depth information. The bone datum calculation module is connected to the 3D face mesh reconstruction module and is used to perform bone datum calculation based on anatomical landmarks based on the 3D face mesh data to obtain the user's personalized bone datum unit value and anatomical datum coordinate system. The acupoint coordinate calculation module is connected to the bone reference calculation module. It is used to calculate the three-dimensional coordinates of the target acupoint based on the personalized bone unit value, the anatomical reference coordinate system and the preset acupoint positioning rules, so as to obtain the current frame three-dimensional coordinates of the target acupoint. The coordinate optimization and AR rendering module is connected to the acupoint coordinate calculation module. It is used to perform coordinate temporal smoothing on the three-dimensional coordinates of the current frame and perform AR rendering on the smoothed coordinates to generate an AR rendering image with acupoint markers superimposed. The gesture recognition and interactive feedback module is connected to the image acquisition and preprocessing module and the coordinate optimization and AR rendering module, respectively. It is used to recognize user gestures based on the image frame data and generate interactive feedback instructions according to the spatial relationship between the gestures and acupoint markers.

9. A three-dimensional acupoint recognition interactive terminal based on end-side calculation and bone measurement method according to claim 8, characterized in that, The image acquisition and preprocessing module specifically includes: RGB cameras and depth cameras are used to acquire video stream data containing color and depth information in real time; The preprocessing unit performs histogram equalization and resolution adjustment on the acquired video stream data to obtain standardized image frame data.

Images