Intelligent acupuncture point positioning method and system based on image recognition

Abstract

The application provides an intelligent acupuncture point positioning method and system based on image recognition, and relates to the technical field of image processing. The method comprises the following steps: collecting image data, depth point cloud data and posture data of a target part and aligning; extracting anatomical feature points and constructing a feature map, and storing the coordinates and feature vectors of each anatomical feature point correspondingly; determining the concave degree of each pixel point in the image data based on the depth point cloud data, screening candidate nodes to construct a candidate graph, and taking the feature vector of the anatomical feature point closest to the candidate node as the candidate feature vector; determining the matching template and matching score of the candidate graph based on the candidate feature vector and topological connection, determining the stability weight of the candidate node based on the contour difference of adjacent image data frames, calculating the node confidence and determining the high-confidence candidate graph accordingly; and determining the acupoint coordinates based on the topological consistency of the high-confidence candidate graph and the standard acupoint topological graph. The application can improve the accuracy of acupoint positioning.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to an intelligent acupuncture point location method and system based on image recognition. Background Technology

[0002] Acupuncture point location is a core technical aspect of the traditional Chinese medicine acupuncture diagnosis and treatment system. Its core objective is to locate acupuncture points used in clinical treatment based on traditional Chinese medicine meridian theory and anatomical landmarks of the human body, combined with classic location principles such as bone measurement and finger measurement. Accurate location not only determines the effectiveness of acupuncture, the targeting of stimulation, and the efficiency of clinical treatment, but also has a crucial impact on avoiding operational risks and preventing puncture complications. It is a fundamental prerequisite for promoting the development of acupuncture treatment from experience-based to standardized, individualized, and intelligent approaches.

[0003] In related technologies, acupuncture point location assistance technology often uses static recognition of a single visible light image as its core implementation path. Specifically, it involves acquiring a two-dimensional image of a local area of ​​the human body, extracting visual features such as skin texture and contour edges from the image, and then matching it with a pre-stored acupuncture point image template to infer the approximate location of the acupuncture point.

[0004] However, the above methods have the following problems: static recognition of a single image is easily affected by the external environment and human physiological characteristics, which leads to a decrease in the accuracy of visual feature extraction and causes positioning deviation; the image recognition range is limited to local areas of the human body, and it cannot achieve collaborative positioning of acupoints in multiple parts of the body, and it lacks the verification step of the physiological and anatomical characteristics of acupoints, making it difficult to meet the clinical needs of high precision and high reliability; it can only achieve positioning in static scenes and does not have dynamic adaptability. When patients make slight limb movements or position adjustments during diagnosis and treatment, the positioning results are prone to failure and cannot meet the needs of real-time diagnosis and treatment. Summary of the Invention

[0005] To address the shortcomings of existing technologies, such as reliance on single-modal data, lack of physiological validation and dynamic adaptability, and insufficient integration of individual differences and dynamic postural factors, which make it difficult to meet the clinical application needs for acupoint location, this application provides an intelligent acupuncture acupoint location method based on image recognition. The specific technical solution adopted is as follows:

[0006] Collect image data, depth point cloud data, and attitude data of the target area and align them to the same coordinate system in time and space.

[0007] Anatomical feature points are extracted from the image data and a feature map is constructed. The feature map stores the coordinates and feature vectors of each anatomical feature point. The feature vectors include the texture contrast, local concavity degree, and detection confidence of the anatomical feature point.

[0008] Based on the depth point cloud data, the degree of concavity of each pixel in the image data is determined to generate a concavity saliency map. Initial candidate regions are selected from the concavity saliency map, and the pixel with the largest degree of concavity in each initial candidate region is selected as a candidate node. A candidate map containing candidate feature vectors and topological connections is constructed. The candidate feature vector is the feature vector of the anatomical feature point that is closest to the candidate node.

[0009] Based on the candidate feature vector and the topological connection, the matching template and matching score of the candidate image are determined in the pre-built acupoint template library. The stability weight of the candidate node is determined based on the contour difference of adjacent image data frames. The node confidence is determined based on the degree of indentation, the stability weight and the matching score. The high-confidence candidate image is determined based on the node confidence.

[0010] The initial acupoint coordinates are determined based on the topological consistency between the high-confidence candidate map and the standard acupoint topology map, and the initial acupoint coordinates are dynamically compensated based on the posture data to obtain the final acupoint coordinates.

[0011] For example, the acquisition of image data, depth point cloud data, and attitude data of the target area and spatiotemporal alignment includes: triggering an RGB camera to acquire image data, a depth sensor to acquire depth point cloud data, and an inertial measurement unit to acquire attitude data via a unified clock; aligning the image data, depth point cloud data, and attitude data to a unified sampling frequency based on a linear interpolation method; calibrating the intrinsic parameters of the RGB camera and the depth sensor; calculating the relative pose between the RGB camera and the depth sensor; and using the intrinsic parameters as a reference, converting the depth point cloud data to the image coordinate system of the image data based on the relative pose; establishing a right-handed coordinate system with the principal point of the image data as the origin, and mapping the image data, depth point cloud data, and attitude data to the right-handed coordinate system.

[0012] For example, the step of extracting anatomical feature points and constructing a feature map from the image data includes: using an edge detection algorithm to initially extract the edge contour of the target region from the image data, and using morphological closing operations to optimize the edge contour; extracting the anatomical feature points from the optimized edge contour, and outputting the two-dimensional coordinates of the anatomical feature points in the image coordinate system where the image data is located; determining the target peripheral window of the anatomical feature points, converting the image region corresponding to the target peripheral window into a local grayscale image, and determining the texture contrast based on the grayscale values ​​of each pixel in the local grayscale image; extracting depth values ​​from the corresponding depth point cloud data based on the two-dimensional coordinates of the anatomical feature points, and determining the degree of local concavity based on the depth values ​​corresponding to the anatomical feature points; obtaining the detection confidence score by performing edge continuity analysis on the optimized edge contour; recording the vector composed of the texture contrast, the degree of local concavity, and the confidence score as the feature vector of the anatomical feature points, and storing the anatomical feature points and the feature vector in a one-to-one correspondence to obtain the feature map.

[0013] For example, determining the degree of concavity of each pixel in the image data based on the depth point cloud data to generate a concavity saliency map includes: for each pixel in the image data, extracting a depth value from the corresponding depth point cloud data based on the two-dimensional coordinates of the pixel in the image coordinate system; obtaining target pixels in the neighborhood of the current pixel, calculating the difference between the depth values ​​of the current pixel and each of the target pixels and averaging them, which is recorded as the degree of concavity of the current pixel; the target pixels are a preset number of pixels in the neighborhood arranged from closest to farthest from the current pixel; and normalizing the degree of concavity of each pixel in the image data to generate the corresponding concavity saliency map.

[0014] For example, the step of screening initial candidate regions from the concavity saliency map, taking the pixel with the largest concavity degree in each initial candidate region as a candidate node, and constructing a candidate graph containing candidate feature vectors and topological connections includes: obtaining the pixel with a concavity degree greater than a first preset threshold to obtain an initial candidate pixel set; performing connected component analysis on the initial candidate pixel set to remove fragmented regions to obtain the initial candidate regions; taking the pixel in the initial candidate regions as candidate nodes, constructing the candidate graph based on the Euclidean distance between each candidate node using k-nearest neighbor connections, and associating the feature vector of the anatomical feature point closest to the candidate node as the candidate feature vector.

[0015] For example, determining the matching template and matching score of the candidate image in the pre-built acupoint template library based on the candidate feature vector and the topological connection includes: for each body posture standard acupoint template in the pre-built acupoint template library, determining a first candidate matching score between the degree of indentation of each candidate node in the candidate image and the degree of indentation of the corresponding acupoint in the body posture standard acupoint template, and a second candidate matching score for the topological connection between the candidate node and the corresponding acupoint; determining the candidate matching score between the candidate image and the body posture standard acupoint template based on the first candidate matching score and the second candidate matching score of each candidate node; determining the body posture standard acupoint template with the candidate matching score greater than a second preset threshold as a candidate template, thereby obtaining a candidate template set; if the highest candidate... If the matching score is less than the second preset threshold, calculate the product of the local concavity degree and the detection confidence of each candidate node in the candidate image, and record it as the node optimization factor; remove the candidate node with the smallest node optimization factor from the candidate image, and redetermine the candidate matching score of the candidate image after removing the candidate node and each candidate template; repeat the process of redetermining the candidate matching score of the candidate image after removing the candidate node until the candidate matching score is not less than the second preset threshold or the number of iterations reaches a preset number; determine the candidate template with the highest candidate matching score as the matching template, and determine the candidate matching score of the matching template as the matching score.

[0016] For example, determining the stability weight of the candidate node based on the contour difference of adjacent image data frames includes: using an edge detection algorithm to extract the edge contours of different pixel regions in the adjacent image data frames, and determining the corresponding points between the edge contours of the pixel regions to form a set of corresponding points; determining the distance between each of the corresponding points in the set of corresponding points, and determining the displacement degree of the pixel region based on the maximum distance therein, and determining the stability weight of each candidate node in the pixel region based on the region type and the displacement degree.

[0017] For example, determining node confidence based on the degree of indentation, the stability weight, and the matching score, and determining a high-confidence candidate graph based on the node confidence, includes: calculating the product of the degree of indentation of the candidate node and the normalized value of the matching score based on a preset weight ratio to obtain a first confidence factor for the candidate node; recording the stability weight of the candidate node as a second confidence factor, calculating the product of the first confidence factor and the second confidence factor to obtain the node confidence; determining the candidate nodes whose node confidence is greater than a third preset threshold as high-confidence nodes, and constructing the high-confidence candidate graph corresponding to the high-confidence nodes.

[0018] For example, determining the initial acupoint coordinates based on the topological consistency between the high-confidence candidate map and the standard acupoint topology map includes: obtaining the standard acupoint topology map of the target location and constructing a candidate anchor point topology map of the high-confidence candidate map; obtaining the actual distance of each connecting edge in the candidate anchor point topology map and determining a first topological factor based on the actual distance and the standard distance of the corresponding edge in the standard acupoint topology map; obtaining the actual angle between each connecting edge in the candidate anchor point topology map and determining a second topological factor based on the actual angle and the corresponding standard angle in the standard acupoint topology map; determining the topological consistency between the candidate anchor point topology map and the standard acupoint topology map based on the first topological factor and the second topological factor; if the topological consistency is not... If the topological consistency is less than the fourth preset threshold, the candidate node corresponding to the highest node confidence in the high-confidence candidate graph is determined as the initial acupoint coordinates; if the topological consistency is less than the fourth preset threshold, the candidate node corresponding to the highest node confidence in the high-confidence candidate graph is used as a benchmark, and the coordinates of other candidate nodes in the high-confidence candidate graph are corrected based on the relative distance and direction between the corresponding acupoint and other acupoints in the standard acupoint topology graph, and the topological consistency is recalculated. This step is repeated until the topological consistency is not less than the fourth preset threshold; based on the topological consistency and the node confidence of each candidate node after coordinate correction, the comprehensive confidence is determined, and the candidate node corresponding to the highest confidence is determined as the initial acupoint coordinates.

[0019] Correspondingly, this application also provides an intelligent acupuncture point positioning system based on image recognition, including:

[0020] The data acquisition module is used to acquire image data, depth point cloud data, and attitude data of the target area and align them to the same coordinate system in time and space.

[0021] An image processing module is used to extract anatomical feature points from the image data and construct a feature map; the feature map stores the coordinates and feature vectors of each anatomical feature point, and the feature vectors include the texture contrast, local concavity degree, and detection confidence of the anatomical feature point;

[0022] The image processing module is further configured to determine the degree of concavity of each pixel in the image data based on the depth point cloud data, so as to generate a concavity saliency map, filter initial candidate regions from the concavity saliency map, take the pixel with the largest degree of concavity in each initial candidate region as a candidate node, and construct a candidate map containing candidate feature vectors and topological connections, wherein the candidate feature vector is the feature vector of the anatomical feature point that is closest to the candidate node;

[0023] The image processing module is further configured to determine the matching template and matching score of the candidate image in the pre-built acupoint template library based on the candidate feature vector and the topological connection, determine the stability weight of the candidate node based on the contour difference of adjacent image data frames, determine the node confidence based on the degree of indentation, the stability weight and the matching score, and determine the high confidence candidate image based on the node confidence.

[0024] The acupoint positioning module is used to determine the initial acupoint coordinates based on the topological consistency between the high-confidence candidate map and the standard acupoint topology map, and to dynamically compensate the initial acupoint coordinates based on the attitude data to obtain the final acupoint coordinates.

[0025] This application may have some or all of the following beneficial effects:

[0026] In the image recognition-based intelligent acupuncture point location method provided in this application, image data, depth point cloud data, and pose data of the target area are collected and spatiotemporally aligned to the same coordinate system. Multimodal data complementarity counteracts interference from single visual data, and spatiotemporal alignment ensures the synergistic effect of multi-source data, avoiding positioning deviations caused by interference from single visual data. Anatomical feature points are extracted from the image data, and a feature map containing texture contrast, local concavity degree, and detection confidence is constructed. Combined with depth point cloud data, the concavity degree of each pixel in the image data is determined, and a concavity saliency map is generated. This allows for the selection of acupuncture points that conform to the physiological and anatomical rules of acupuncture points (acupuncture points are mostly distributed in concavities such as tendon junctions and bone gaps) from the concavity saliency map. The application uses an initial candidate region (anatomical region) to construct a candidate map, thus verifying the physiological and anatomical features of acupoints. Matching templates for the candidate maps are determined from a pre-built acupoint template library, adapting to individual body shape differences. Stability weights are calculated based on contour differences between adjacent image data frames to quantify and suppress motion interference areas, overcoming the limitations of static positioning failure. Node confidence is determined through multi-dimensional weighted fusion of concavity degree, matching score, and stability weights, avoiding misjudgment due to a single factor and improving the credibility of candidate regions. Topological consistency verification between high-confidence candidate maps and standard acupoint topology maps, along with dynamic compensation of posture data, ensures both positioning accuracy and global compliance while dynamically adapting to slight limb movements and postural adjustments by the patient. In summary, this application achieves high-precision, interpretable intelligent positioning of multiple body parts by fusing multimodal information such as depth point clouds for spatial correction and physiological verification of image recognition results, promoting the intelligent and standardized development of acupuncture technology.

[0027] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0028] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 A flowchart of an image recognition-based intelligent acupuncture point location method according to an exemplary embodiment of this application is shown;

[0030] Figure 2 A visual interactive schematic diagram of an image recognition-based intelligent acupuncture point location method according to an exemplary embodiment of this application is shown;

[0031] Figure 3A schematic block diagram of an image recognition-based intelligent acupuncture point positioning system according to an exemplary embodiment of this application is shown. Detailed Implementation

[0032] To further illustrate the technical means and effects adopted by this application to achieve the intended inventive purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of the image recognition-based intelligent acupuncture point positioning method and system proposed in this application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0034] The following description, in conjunction with the accompanying drawings, details the specific scheme of the image recognition-based intelligent acupuncture point location method and system provided in this application.

[0035] Please see Figure 1 It illustrates a flowchart of an image recognition-based intelligent acupuncture point location method provided in one embodiment of this application, as shown below. Figure 1 As shown, this image recognition-based intelligent acupuncture point location method specifically includes the following steps:

[0036] S110: Collect image data, depth point cloud data, and attitude data of the target area and align them to the same coordinate system in time and space;

[0037] S120: Extract anatomical feature points from image data and construct a feature map; the feature map stores the coordinates and feature vectors of each anatomical feature point, and the feature vectors include the texture contrast, local concavity degree and detection confidence of the anatomical feature points;

[0038] S130: Determine the degree of concavity of each pixel in the image data based on the depth point cloud data to generate a concavity saliency map. Select initial candidate regions from the concavity saliency map, take the pixel with the largest degree of concavity in each initial candidate region as the candidate node, and construct a candidate map containing candidate feature vectors and topological connections. The candidate feature vector is the feature vector of the anatomical feature point that is closest to the candidate node.

[0039] S140: Based on candidate feature vectors and topological connections, determine the matching template and matching score of candidate images in the pre-built acupoint template library; determine the stability weight of candidate nodes based on the contour differences of adjacent image data frames; determine the node confidence based on the degree of concavity, stability weight and matching score; and determine high-confidence candidate images based on the node confidence.

[0040] S150: The initial acupoint coordinates are determined based on the topological consistency between the high-confidence candidate map and the standard acupoint topology map, and the initial acupoint coordinates are dynamically compensated based on the attitude data to obtain the final acupoint coordinates.

[0041] The following is a detailed explanation of each step in the above-mentioned intelligent acupuncture point location method based on image recognition:

[0042] In step S110, image data, depth point cloud data, and attitude data of the target area are acquired and spatiotemporally aligned to the same coordinate system.

[0043] In this embodiment, the target area is a specific region on the human body surface that requires precise acupoint location in clinical acupuncture treatment scenarios; for example, the target area may be the patient's upper limbs, abdomen, back, or other parts.

[0044] In the embodiments of this application, the above-mentioned image data is visual image data of the target part of the body surface; for example, the image data can be an RGB image captured by an RGB camera.

[0045] In this embodiment of the application, the aforementioned depth point cloud data is the three-dimensional spatial location data of the target part; for example, the depth point cloud data can be collected by a depth sensor.

[0046] In this embodiment of the application, the above-mentioned attitude data is the dynamic motion state data of the target part; for example, the attitude data can be collected by an inertial measurement unit (IMU), including angular velocity and acceleration.

[0047] For example, the above-mentioned acquisition of image data, depth point cloud data, and attitude data of the target area and spatiotemporal alignment to the same coordinate system can be achieved as follows: An RGB camera is triggered to acquire image data, a depth sensor to acquire depth point cloud data, and an inertial measurement unit to acquire attitude data via a unified clock. The image data, depth point cloud data, and attitude data are then aligned to a unified sampling frequency using a linear interpolation method. The intrinsic parameters of the RGB camera and depth sensor are calibrated, the relative pose between the RGB camera and depth sensor is calculated, and the depth point cloud data is converted to the image coordinate system of the image data based on the relative pose using the intrinsic parameters as a reference. A right-handed coordinate system is established with the principal point of the image data as the origin, and the image data, depth point cloud data, and attitude data are all mapped to the right-handed coordinate system.

[0048] The process of data acquisition and spatiotemporal alignment described above will be explained in detail below in a specific implementation of an embodiment of this application:

[0049] S1: Based on the actual needs of acupuncture positioning, set the parameters of the equipment used to collect the above image data, depth point cloud data and posture data.

[0050] Specifically, an RGB camera is used to capture the surface contour and anatomical reference boundary of the target area. Its acquisition parameters are set as follows: resolution 1920×1080, frame rate 30fps, and focal length f=5mm. A depth sensor is used to output the spatial coordinate set {(X,Y,Z)} of the three-dimensional point cloud of the target area. This compensates for the susceptibility of RGB images to interference from hair and lighting, providing three-dimensional data support for verifying the physiological and anatomical features of acupoints. Its acquisition parameters are set as follows: accuracy ±2mm and sampling rate 15fps. The IMU outputs the angular velocity of the target area in real time. and acceleration It is used to monitor slight changes in patient posture (such as raising an arm or abdominal movement caused by breathing). Its acquisition parameters are set as follows: 6-axis, posture error ≤0.5°, and sampling rate 100Hz.

[0051] S2: Perform time alignment on image data, depth point cloud data, and pose data.

[0052] Specifically, the sampling actions of the RGB camera, depth sensor, and IMU are synchronously triggered by the unified clock (1μs accuracy) of the main control board of the device to ensure that the timestamp error of the three types of data frames is ≤5ms, and to avoid data misalignment caused by trigger time difference. The linear interpolation method is used to uniformly align the depth point cloud data and IMU attitude data to the 30fps frame rate of the RGB image, and to determine that each frame of RGB image corresponds to one set of depth point cloud data and IMU attitude data, so as to ensure that the three types of data correspond one-to-one in the time dimension.

[0053] S3: Spatial registration of image data, depth point cloud data, and pose data.

[0054] For example, the above spatial registration can be achieved through hardware calibration, extrinsic parameter alignment, and point cloud interpolation preprocessing.

[0055] Specifically, the above hardware calibration process can be implemented as follows: using a 9×12 checkerboard calibration board (square size 10mm), the RGB camera intrinsic parameters (focal length) are obtained through the Zhang Zhengyou calibration method. Main point Radial distortion coefficient (and depth sensor intrinsics.)

[0056] The above extrinsic alignment process can be achieved as follows: Using an RGB camera and a depth sensor, with a 9×12 checkerboard calibration board as the common shooting object, RGB images and depth images of the calibration board are acquired simultaneously. The relative pose of the RGB camera and the depth sensor is solved. This relative pose includes a rotation matrix (describing the rotation relationship of the depth sensor relative to the RGB camera) and a translation vector (describing the translation relationship of the depth sensor relative to the RGB camera). Using the above extrinsic parameters, the original 3D point cloud acquired by the depth sensor can be transformed into the RGB image coordinate system, realizing the spatial association between depth data and visual data.

[0057] The above point cloud interpolation preprocessing can be implemented as follows: To address the issue of pixel mismatch caused by the sampling rate of the depth point cloud being lower than that of the RGB image, bilinear interpolation is used to complete the depth point cloud to a resolution of 1920×1080 (consistent with the RGB image), ensuring that each pixel of the RGB image corresponds to a unique depth value Z, thus achieving point-by-point matching between pixels and depth.

[0058] S4: Map the above image data, depth point cloud data and pose data to the same coordinate system.

[0059] Specifically, based on the principal point of the RGB image Establish a right-handed coordinate system with origin O: the u-axis extends horizontally to the right along the RGB image plane, aligning with the image width direction; the v-axis extends vertically downwards along the RGB image plane, aligning with the image height direction; and the z-axis is perpendicular to the RGB image plane and points towards the target body surface. The principal points mentioned above... The projection point of the optical center of the RGB camera onto the image plane is obtained by hardware calibration. The pixel coordinates of the corrected RGB image, the coordinates of the completed depth point cloud, and the attitude data acquired by the IMU are all mapped to this right-handed coordinate system to achieve a unified association of visual, spatial, and dynamic data.

[0060] It should be noted that the above sensor model and parameters are a preferred embodiment of the present invention. The sampling frequency, resolution or number of channels can be adjusted according to the actual application scenario. As long as it achieves the corresponding data acquisition and registration functions, it falls within the scope of the technical concept of the present invention.

[0061] In step S120, anatomical feature points are extracted from the image data and a feature map is constructed. The feature map stores the coordinates and feature vectors of each anatomical feature point. The feature vectors include the texture contrast, local concavity degree, and detection confidence of the anatomical feature points.

[0062] In this embodiment of the application, the above-mentioned anatomical feature points are rigid anatomical feature points on the body surface of the target site that have clear physiological significance and can be used as a reference for locating acupoints in traditional Chinese medicine. For example, the anatomical feature points may include joint spaces (such as the end point of the elbow crease), bone protrusions (such as the spinous process of the seventh cervical vertebra) and meridian reference lines (such as the line connecting the spinous processes of the spine).

[0063] In the embodiments of this application, the above-mentioned feature map is a structured data set that stores the two-dimensional coordinates of anatomical feature points and their corresponding feature vectors one by one, and serves as a carrier for the association between visual features and spatial location.

[0064] In this embodiment, the aforementioned feature vector is a three-dimensional vector used to describe the attribute features of a single anatomical feature point. It can quantify the anatomical and visual attributes of the anatomical feature point, evaluate the reliability of the anatomical feature point and its correlation with acupoints from different dimensions, and avoid misjudgments caused by relying solely on location information. Specifically, the feature vector includes feature components in three dimensions: texture contrast, local depression degree, and detection confidence. Among them, the aforementioned texture contrast is used to describe the degree of brightness difference of a specific range of pixel windows around the anatomical feature point, which can reflect the complexity of the body surface structure; the aforementioned local depression degree refers to the quantified value of the body surface depression depth at the location corresponding to the anatomical feature point, which is used to associate the physiological and anatomical features of acupoints (acupoints are mostly distributed in depression areas); the aforementioned detection confidence refers to the credibility of the detection result of the anatomical feature point, which is determined based on the continuity detection of the edges of the anatomical feature point in the RGB image (such as whether the edges of the spinous processes of the spine are continuous, and whether the edges of the joint spaces are clear).

[0065] For example, the above-mentioned extraction of anatomical feature points and construction of feature maps from image data can be achieved as follows: An edge detection algorithm is used to initially extract the edge contour of the target region from the image data, and morphological closing operations are used to optimize the edge contour; anatomical feature points are extracted from the optimized edge contour, and the two-dimensional coordinates of the anatomical feature points in the image coordinate system are output; the target surrounding window of the anatomical feature points is determined, the image region corresponding to the target surrounding window is converted into a local grayscale image, and the texture contrast is determined based on the grayscale values ​​of each pixel in the local grayscale image; depth values ​​are extracted from the corresponding depth point cloud data based on the two-dimensional coordinates of the anatomical feature points, and the degree of local concavity is determined based on the depth values ​​corresponding to the anatomical feature points; edge continuity analysis is performed on the optimized edge contour to obtain the detection confidence; the vector composed of texture contrast, degree of local concavity, and confidence is recorded as the feature vector of the anatomical feature points, and the anatomical feature points and feature vectors are stored in a one-to-one correspondence to obtain the feature map.

[0066] The process of extracting anatomical feature points and constructing feature maps is described in detail below in a specific implementation of an embodiment of this application: The RGB image is processed using the Canny edge detection algorithm, with a low threshold set to T1=50 and a high threshold set to T2=150. This dual-threshold filtering preserves areas with clear edges in the image data (such as joint spaces and the edges of bone protrusions) and filters out false edges caused by noise. Morphological closing operations are performed on the edge-processed image data. Specifically, a 3×3 rectangular kernel is used to process the edge detection results, filling in small gaps in the edges (such as edge breaks caused by skin texture) to ensure the edge continuity of the anatomical feature points. Three types of anatomical feature points are manually or automatically identified from the preprocessed image data: joint spaces (such as the endpoint of the elbow crease in the upper limb and the midpoint of the knee crease in the lower limb), bone protrusions (such as the spinous process of the seventh cervical vertebra in the back and the sternal angle in the chest), and meridian reference lines (such as the line connecting the spinous processes of the spine). A two-dimensional coordinate set of the anatomical feature points is output. Where K is the number of feature points, typically ranging from 4 to 6. Let k be the two-dimensional coordinates of the k-th anatomical feature point in the RGB image coordinate system; for each anatomical feature point output... Calculate its eigenvectors and construct feature maps , where the feature vector Specifically, it includes the following feature components: (Texture contrast of the 32×32 pixel window surrounding the k-th anatomical feature point) can be calculated using the following method: taking the k-th anatomical feature point as an example. Centered on the image, a 32×32 pixel window is extracted, and the difference between the maximum and minimum pixel grayscale values ​​within the corresponding grayscale image is calculated. This difference is denoted as the texture contrast. (The degree of local concavity corresponding to the depth value of the k-th anatomical feature point can be calculated by the following method: obtain the k-th anatomical feature point from the right-handed coordinate system mentioned above.) The corresponding depth value Z is used to obtain the depth values ​​of the kth anatomical feature point within its 8-neighborhood and its 6 nearest neighboring points, and to calculate the average difference between the depth values ​​of the kth anatomical feature point and its neighboring points, which is denoted as the aforementioned local depression degree. (The detection confidence of the k-th anatomical feature point, obtained based on edge continuity detection, ranges from [0,1] and can be used to filter out falsely detected fuzzy feature points.) It can be obtained by: analyzing the continuity of the edge of the k-th anatomical feature point and outputting the detection confidence based on preset logic rules. For example, if the edge of the k-th anatomical feature point is unbroken within a 3×3 neighborhood and has a strong correlation with surrounding anatomical structures (such as the line connecting the spinous processes of the spine), the detection confidence is assigned a value of 0.8-1.0; if the edge is fuzzy or isolated, the detection confidence is assigned a value below 0.5, which can be filtered out later. The two-dimensional coordinates of each anatomical feature point are associated one-to-one with the calculated feature vector to obtain the aforementioned feature map. .

[0067] In step S130, the degree of concavity of each pixel in the image data is determined based on the depth point cloud data to generate a concavity saliency map. Initial candidate regions are selected from the concavity saliency map, and the pixel with the largest degree of concavity in each initial candidate region is taken as a candidate node. A candidate map containing candidate feature vectors and topological connections is constructed. The candidate feature vector is the feature vector of the anatomical feature point that is closest to the candidate node.

[0068] In this embodiment of the application, the above-mentioned degree of depression is a quantitative index describing the depth of depression of the corresponding body surface position of the pixel in the image data; since acupoints are mostly distributed in the depression areas such as tendon junctions and bone gaps, the degree of depression can be used to quantify the matching degree between the corresponding pixel and the physiological and anatomical features of the acupoint.

[0069] In this embodiment of the application, the above-mentioned depression saliency map is a visual image obtained by converting the degree of depression of all pixels in the entire image domain, which can intuitively reflect the possibility that each pixel is a potential acupoint area.

[0070] For example, the above-mentioned determination of the concavity degree of each pixel in the image data based on depth point cloud data to generate a concavity saliency map can be achieved as follows: For each pixel in the image data, the depth value is extracted from the corresponding depth point cloud data based on the two-dimensional coordinates of the pixel in the image coordinate system; target pixels in the neighborhood of the current pixel are obtained, the difference between the depth values ​​of each target pixel and the current pixel is calculated and averaged, and recorded as the concavity degree of the current pixel; the target pixels are a preset number of pixels in the neighborhood arranged from near to far from the current pixel; the concavity degree of each pixel in the image data is normalized to generate the corresponding concavity saliency map.

[0071] The process of generating the concavity saliency map is described in detail below in a specific implementation of an embodiment of this application: Taking any pixel in an RGB image as an example, the depth point cloud coordinates of the current pixel and its six nearest neighboring pixels within its eight-neighborhood are obtained based on the three-dimensional point cloud reconstructed by the depth sensor (interpolated to 1920×1080 resolution), and the concavity degree of the current pixel is calculated using the following formula:

[0072]

[0073] in, For the current pixel The degree of indentation; For the current pixel The corresponding depth value in the depth point cloud coordinates; For the current pixel The depth value in the depth point cloud coordinates corresponding to the i-th neighboring point among the 6 nearest neighbors; This proves that the current pixel point The location is a depression area. The larger the value, the deeper the depression, and the more it conforms to the anatomical characteristics of the acupoint.

[0074] For the entire image domain After normalization, the values ​​are converted into probabilities of acupoints:

[0075]

[0076] in, For the current pixel The probability of acupoints obtained after normalizing the degree of indentation is within the range of [value missing]. The closer the value is to 1, the more it reflects the degree of depression at the current pixel position, which is more in line with the acupoint pattern. For the current pixel The degree of indentation; This represents the maximum value of the degree of concavity at each pixel in the entire image domain; This represents the minimum degree of concavity for all pixels across the entire image domain. It should be noted that in extreme cases, if... If the value is 0, then the probability of the acupoint corresponding to this situation is directly set to 0.

[0077] After calculating the probability of acupoint location for each pixel through the above process, the probability of acupoint location for each pixel is used as the pixel grayscale value (or heatmap color value) to generate a 920×1080 resolution concavity saliency map. It intuitively reflects the potential acupoint areas across the entire image domain.

[0078] In this embodiment of the application, the initial candidate region is a set of continuous pixels with a high probability of acupoints selected from the concavity saliency map, which is the potential candidate range of acupoints.

[0079] In this embodiment, the candidate node is the pixel with the highest probability of being an acupoint within each initial candidate region; the candidate feature vector is the feature data associated with each candidate node to describe its attributes, and the candidate feature vector is the feature vector of the anatomical feature point closest to the candidate node.

[0080] In this embodiment of the application, the candidate graph is a structured graph constructed with candidate nodes as nodes and topological relationships between nodes as edges.

[0081] For example, the above-mentioned screening of initial candidate regions from the concavity saliency map, taking the pixel with the largest concavity degree in each initial candidate region as the candidate node, and constructing a candidate graph containing candidate feature vectors and topological connections can be achieved as follows: obtaining pixels with a concavity degree greater than a first preset threshold to obtain an initial candidate pixel set; performing connected component analysis on the initial candidate pixel set to remove fragmented regions to obtain initial candidate regions; taking the pixels in the initial candidate regions as candidate nodes, constructing a candidate graph based on the Euclidean distance between each candidate node using k-nearest neighbor connections, and associating the feature vector of the anatomical feature point closest to the candidate node as a candidate feature vector.

[0082] The process of constructing the candidate graph described above will be explained in detail below in a specific implementation of an embodiment of this application: setting a first preset threshold. For example, the first preset threshold value ranges from 0.5 to 0.7, with a preferred threshold value of 0.6; the salience of the depression is preserved in the graph. The initial candidate pixel set is obtained by analyzing the pixels. An 8-neighbor connected component analysis is performed on this initial candidate pixel set, setting the minimum connected component area to 30 pixels (corresponding to an effective stimulation area diameter of approximately 3mm in clinical acupoints). Fragmented small regions with an area less than 30 pixels are removed, retaining 8-10 consecutive pixel sets that meet the criteria, i.e., the initial candidate regions. For each initial candidate region, the pixel area of ​​all pixels within the initial candidate region is traversed. ,turn up The pixel with the largest value is selected as the candidate node. Where M is the number of candidate nodes. Let be the two-dimensional coordinates of the i-th candidate node in the RGB image coordinates; for each candidate node, select the anatomical feature point closest to it, and associate the feature vector of the anatomical feature point with the candidate feature vector of the candidate node. ,in, Candidate nodes The probability of acupoints. Candidate nodes Texture contrast; Candidate nodes The detection confidence is calculated; k-nearest neighbor connections are established based on the Euclidean distance between nodes to construct a candidate graph. Specifically, for each candidate node, the Euclidean distance between the candidate node and all other candidate nodes is calculated, and the two closest nodes are selected to establish a connection, forming an edge between the candidate nodes, thus constructing the candidate graph. ,in, This includes the coordinates of the corresponding candidate nodes and the candidate feature vectors. This involves connecting nodes. Furthermore, the edge weight between the i-th and j-th candidate nodes can be taken as the cosine similarity of the feature vectors of the two nodes. This value reflects the consistency of features between two nodes; the closer it is to 1, the more consistent the features are.

[0083] In step S140, the matching template and matching score of the candidate image are determined in the pre-built acupoint template library based on the candidate feature vector and topological connection. The stability weight of the candidate node is determined based on the contour difference of adjacent image data frames. The node confidence is determined based on the degree of concavity, stability weight and matching score. The high confidence candidate image is determined based on the node confidence.

[0084] In this embodiment, the aforementioned pre-built acupoint template library is a pre-constructed set of templates containing standard acupoint features for different body shapes (e.g., fat, thin) and different target locations (e.g., upper limbs, abdomen, back). It serves as a reference benchmark for candidate image matching, covering anatomical differences in different ages, genders, and body shapes, ensuring the templates' adaptability to individual differences. Each body shape standard acupoint template in the pre-built acupoint template library includes a standard range for the degree of acupoint depression, acupoint topological connection parameters (e.g., distance and angle between standard acupoints), and information on the association of anatomical landmarks on the body surface (e.g., the relative position of acupoints with elbow creases and spinous processes of the spine).

[0085] In this embodiment of the application, the matching template is a template selected from the pre-built acupoint template library that has a high similarity to the features of the candidate image; the matching score is an indicator used to quantify the degree of matching between the candidate image and the body posture standard acupoint template, and the value range is [0,1]. The higher the matching score, the stronger the feature consistency between the candidate image and the corresponding body posture standard acupoint template.

[0086] For example, the above-mentioned determination of the matching template and matching score of the candidate image in the pre-built acupoint template library based on candidate feature vectors and topological connections can be achieved as follows: For each body posture standard acupoint template in the pre-built acupoint template library, determine the first candidate matching score of the indentation degree of each candidate node in the candidate image and the indentation degree of the corresponding acupoint in the body posture standard acupoint template, and the second candidate matching score of the topological connection between the candidate node and the corresponding acupoint; determine the candidate matching score between the candidate image and the body posture standard acupoint template based on the first candidate matching score and the second candidate matching score of each candidate node; determine the body posture standard acupoint template with a candidate matching score greater than a second preset threshold as the candidate template, and obtain the candidate template. Set up a set of templates; if the highest candidate matching score is less than the second preset threshold, calculate the product of the local concavity degree and the detection confidence of each candidate node in the candidate graph, and record it as the node optimization factor; remove the candidate node with the smallest node optimization factor from the candidate graph, and redetermine the candidate matching score between the candidate graph after removing the candidate node and each candidate template; repeat the process of if the highest candidate matching score is less than the second preset threshold until the candidate matching score between the candidate graph after removing the candidate node and each candidate template is redetermined, until the candidate matching score is not less than the second preset threshold or the number of iterations reaches the preset number; determine the candidate template with the highest candidate matching score as the matching template, and determine the candidate matching score of the matching template as the matching score.

[0087] The following is a detailed description of the process for determining the matching template and matching score in a specific implementation of an embodiment of this application: Traverse each body posture standard acupoint template in the pre-built acupoint template library, and calculate the concavity degree matching score (i.e., the first candidate matching score) for each candidate node in the candidate image. Specifically, the calculation rule can be as follows: If the candidate node falls within the standard range of the corresponding acupoint of the body posture standard acupoint template, it is assigned a higher base score; otherwise, the score is linearly reduced according to the degree of deviation. (For example, when it falls within the standard range, the base score is 1; when the deviation is 10%, the base score is 0.9; when the deviation is 50%, the base score is 0.5); Calculate the topological connection matching score (i.e., the second candidate matching score) as follows: For node pairs with connections in the candidate image, calculate the deviation of their actual distance and angle from the standard parameters of the body posture standard acupoint template. The smaller the deviation, the higher the corresponding second candidate matching score (e.g., if the deviation is less than 5%, the second candidate matching score is 1.0; if the deviation is greater than 20%, the second candidate matching score is 0.4); Combine the candidate matching scores to obtain the candidate matching score between the candidate image and each body posture standard acupoint template. For example, the scores of the first and second candidate matching points are weighted and summed in a 6:4 ratio to obtain the corresponding candidate matching scores; the body shape standard acupoint templates with matching scores higher than the second preset threshold are retained to form a candidate template set to narrow down the subsequent matching range; if the score of the best matching template in the candidate template set is <0.8 (ideal threshold, which can be taken as the second preset threshold mentioned above), it is determined that the candidate image may have interference or redundant nodes; the product of the indentation intensity and detection confidence of each candidate node (i.e., the node optimization factor mentioned above) is calculated and sorted, and the candidate node with the smallest node optimization factor is removed from the candidate image, and the candidate matching scores of the candidate image after node removal and each candidate template are recalculated; the above optimization steps are repeated (set to a maximum of 3 iterations) until the score of the best matching template is ≥0.8 or the maximum number of iterations is reached; the candidate template with the highest final candidate matching score is selected as the matching template, and its matching score is recorded.

[0088] In this embodiment, the stability weight is a dynamic confidence coefficient assigned to each candidate node based on the contour differences of adjacent image data frames. The larger the value, the less motion interference the candidate node is affected by.

[0089] For example, the above-mentioned determination of the stability weight of candidate nodes based on the contour difference of adjacent image data frames can be implemented as follows: an edge detection algorithm is used to extract the edge contours of different pixel regions in adjacent image data frames, and the corresponding points between the edge contours of the pixel regions are determined to form a set of corresponding points; the distance between each corresponding point in the set of corresponding points is determined, the displacement degree of the pixel region is determined based on the maximum distance, and the stability weight of each candidate node in the pixel region is determined based on the region type and the displacement degree.

[0090] The determination process of the stability weight is described in detail below in a specific implementation of an embodiment of this application: RGB images of the current frame and the previous frame are acquired, and the surface contours of the target area are extracted using Canny edge detection to obtain the pixel coordinate set of the two frame contours; the maximum offset between the two frame contours is calculated using the Hausdorff distance algorithm: if the maximum offset ≤ 2 pixels, it is determined to be no significant motion; if 2 pixels < maximum offset ≤ 8 pixels, it is determined to be slight motion; if the maximum offset > 8 pixels, it is determined to be significant displacement; for each candidate node, the region type of its pixel area (including rigid regions and soft tissue regions) is first determined. If it is a rigid region (such as the spinous process of the spine, joint space), then when there is no significant motion, the stability weight is... =1; stability weight during slight motion. Decrease linearly based on the maximum offset; stability weights are applied during significant motion. =0; if it is a soft tissue area (such as the abdomen or upper arm), then during significant movement, the stability weight is 0. =1; stability weight during slight motion. Decrease exponentially by the maximum offset; stability weights are applied when there is no significant motion. =0.

[0091] In this embodiment, the node confidence level is the final confidence index calculated for each candidate node by comprehensively considering the degree of indentation, stability weight, and matching score. The larger the value, the more likely the corresponding candidate node is to be the target acupoint.

[0092] In this embodiment of the application, the high-confidence candidate graph is a structured graph selected from the initial candidate graph that meets the node confidence standard and has no duplicate candidate nodes.

[0093] For example, the above method of determining node confidence based on indentation degree, stability weight, and matching score, and determining high-confidence candidate graphs based on node confidence, can be implemented as follows: Calculate the product of the indentation degree of the candidate node and the normalized value of the matching score based on a preset weight ratio to obtain the first confidence factor of the candidate node; denote the stability weight of the candidate node as the second confidence factor, calculate the product of the first confidence factor and the second confidence factor to obtain the node confidence; determine candidate nodes with node confidence greater than a third preset threshold as high-confidence nodes, and construct the high-confidence candidate graph corresponding to the high-confidence nodes.

[0094] The following is a detailed description of the process for determining the high-confidence nodes and high-confidence candidate graphs in a specific implementation of an embodiment of this application: The matching scores calculated above are normalized, and the probability of acupoints for candidate nodes is calculated in a 6:4 ratio. The static confidence level (i.e., the first confidence factor mentioned above) is obtained by weighting the normalized matching scores; the stability weights of the first confidence factor and the candidate nodes are then calculated. The node confidence level is obtained by multiplying the product of (that is, the second confidence factor mentioned above). Set the node confidence threshold (i.e., the third preset threshold mentioned above) to 0.7, and retain the node confidence. Candidate nodes with a confidence level of not less than 0.7 (i.e., the high-confidence nodes mentioned above) are selected. The Euclidean distance between the nodes is calculated and retained. If the center-to-center distance between two nodes is less than 10 pixels (approximately 3 mm, the minimum resolvable distance for acupoints in clinical practice), the node with the higher confidence level is retained to avoid duplicate candidates in the same acupoint area. The selected high-confidence nodes (including coordinates, candidate feature vectors, and node confidence) are integrated, and the k-nearest neighbor connections between nodes (k=2) are reconstructed to form a high-confidence candidate map.

[0095] In step S150, the initial acupoint coordinates are determined based on the topological consistency between the high-confidence candidate map and the standard acupoint topology map, and the initial acupoint coordinates are dynamically compensated based on the attitude data to obtain the final acupoint coordinates.

[0096] In this embodiment of the application, the aforementioned standard acupoint topology map refers to a structured map of the spatial distribution of acupoints in the target area established in accordance with national standards. It includes the coordinates of all standard acupoints in the target area and the topological connection relationship between acupoints, and is used to verify the compliance of the global meridian pathway of the high-confidence candidate map.

[0097] In the embodiments of this application, the above-mentioned topological consistency refers to the degree of matching between the node topological relationship of the high-confidence candidate graph and the standard acupoint topological graph, which is the core indicator for judging whether the candidate region conforms to the meridian rules of traditional Chinese medicine.

[0098] For example, the above-mentioned determination of initial acupoint coordinates based on the topological consistency between the high-confidence candidate map and the standard acupoint topology map can be achieved as follows: Obtain the standard acupoint topology map of the target location and construct a candidate anchor point topology map of the high-confidence candidate map; obtain the actual distance of each connecting edge in the candidate anchor point topology map, and determine a first topological factor based on the actual distance and the standard distance of the corresponding edge in the standard acupoint topology map; obtain the actual included angle between each connecting edge in the candidate anchor point topology map, and determine a second topological factor based on the actual included angle and the corresponding standard included angle in the standard acupoint topology map; determine the topological consistency between the candidate anchor point topology map and the standard acupoint topology map based on the first topological factor and the second topological factor; if If the topological consistency is not less than the fourth preset threshold, the candidate node corresponding to the highest confidence in the high-confidence candidate graph is determined as the initial acupoint coordinates. If the topological consistency is less than the fourth preset threshold, the coordinates of other candidate nodes in the high-confidence candidate graph are corrected based on the relative distance and direction between the corresponding acupoint and other acupoints in the standard acupoint topology graph, and the topological consistency is recalculated. This step is repeated until the topological consistency is not less than the fourth preset threshold. Based on the topological consistency and the node confidence of each candidate node after coordinate correction, the comprehensive confidence is determined, and the candidate node corresponding to the highest confidence is determined as the initial acupoint coordinates.

[0099] In one specific implementation of this application embodiment, the above-mentioned determination of initial acupoint coordinates can be achieved as follows: calling the corresponding standard acupoint topology map based on the current target location. ,in, For all standard acupoints in the target area, use the anatomical feature points on the body surface as the origin. The relative coordinates in the image coordinate system For topological connections between acupoints, the edge attributes include standard distance. Standard included angle (The angle between the acupoint connection line and the body surface reference line); Based on the candidate nodes in the high-confidence candidate map, the 3D point cloud coordinates and node confidence of each candidate node are integrated to form a node set. If two candidate nodes are in the standard acupuncture point topology graph If adjacent acupoints exist, an edge is constructed between these two candidate nodes, resulting in the aforementioned candidate anchor point topology graph. The edge attributes include the actual distance. (Euclidean distance calculated using three-dimensional coordinates), actual included angle (The angle between the candidate node connection and the surface reference line); For each connection edge in the candidate anchor point topology graph, the first topology factor is calculated using the following formula:

[0100]

[0101] in, This refers to the first topological factor mentioned above; The actual distance to any connecting edge in the candidate anchor point topology graph; The standard length of the connecting edge in the candidate anchor point topology graph corresponds to any connecting edge in the standard acupoint topology graph.

[0102] Furthermore, for any included angle corresponding to a connecting edge in the above candidate anchor point topology graph, the second topology factor is calculated using the following formula:

[0103]

[0104] in, This refers to the second topological factor mentioned above; The actual included angle corresponding to any connecting edge in the above candidate anchor point topology graph; This is the standard angle corresponding to the actual angle in the standard acupoint topology diagram.

[0105] Combining the formulas for calculating the first and second topological factors, the closer the actual distance of each connecting edge in the candidate anchor point topology map is to the standard distance of its corresponding edge in the standard acupoint topology map, the more consistent the actual included angle of each connecting edge in the candidate anchor point topology map with its corresponding standard included angle in the standard acupoint topology map, the higher the topological consistency between the candidate anchor point topology map and the standard acupoint topology map, and the more reliable the positioning result.

[0106] For example, a nonlinear decay rule can be used to determine the topological scores corresponding to the first and second topological factors. Specifically, an exponential decay model and a power function decay model can be used to implement the nonlinear decay rule. First, the distance score and angle score of a single edge are calculated and fused. Then, the average score of each edge is taken as the topological consistency score, achieving the effect of maintaining high scores for small deviations and rapidly reducing scores for large deviations. All connected edges are weighted and averaged according to the weight of edge length (70%) and angle (30%) to obtain the topological consistency score. If the topological consistency score is ≥0.7 (i.e., the fourth preset threshold mentioned above), it indicates that the topological relationship of the candidate anchor point is highly consistent with the standard map, and the positioning result is reliable. The candidate node corresponding to the highest node confidence is directly determined as the initial acupoint coordinates. Otherwise, the most reliable candidate node is used as the benchmark, and the high-confidence candidate map is corrected as a whole according to the relative positional relationship in the standard acupoint topological map: the point with the highest node confidence is selected from all candidate nodes. This serves as the positioning benchmark; based on the relative distance and direction of this benchmark point to other acupoints in the standard acupoint topology map, the corrected coordinates of other candidate points are recalculated. : Standard distance × cos (The cosine of the standard included angle); Standard distance × sin (The sine of the standard included angle); The coordinates remain unchanged (the body surface depth is basically stable); the topological consistency score is recalculated using the corrected coordinates. If it reaches the fourth preset threshold, the result is accepted; otherwise, the above process can be repeated to ensure that the localization result conforms to the anatomical rules of traditional Chinese medicine; the confidence scores of each candidate node are then combined. With topology consistency Calculate the final overall confidence level. ,choose The highest-ranking candidate node is the final acupoint, and its initial acupoint coordinates are output. .

[0107] After determining the initial acupoint coordinates through the above process, this embodiment of the application can also dynamically compensate for the initial acupoint coordinates based on attitude data to correct the offset of the initial acupoint coordinates caused by motion in real time. Specifically, this can be achieved by using the angular velocity output in real time by the IMU. With acceleration The device's attitude change relative to the body surface (rotation matrix R, translation vector T) is calculated, and the initial acupoint coordinates are corrected in real time.

[0108]

[0109] in, The final acupoint coordinates are determined by a compensation frequency and a sampling rate of 100Hz, ensuring that displacement ≤2mm can be corrected in real time and that the positioning error is controlled within the clinically permissible range.

[0110] After determining the final acupoint coordinates, these coordinates are mapped back to an RGB image and overlaid on the real-time screen using AR technology. The acupoint names, confidence levels, and anatomical reference markers are then labeled to aid in confirmation. Figure 2 As shown. Furthermore, if the doctor confirms a deviation in the localization results, the current multimodal features and corrected coordinates must be recorded, and the global acupoint image template library updated. Specifically, this involves fine-tuning the indentation degree of standard acupoints for templates of the same body type, age, and gender. Range and topology parameters; after accumulating 100 feedbacks of the same type, retrain the weight coefficients of template matching to improve the adaptability of subsequent localization.

[0111] The foregoing mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0112] Correspondingly, embodiments of this application also provide an intelligent acupuncture point positioning system based on image recognition, see reference. Figure 3 As shown, the image recognition-based intelligent acupuncture point positioning system 300 may include a data acquisition module 310, an image processing module 320, and an acupuncture point positioning module 330, wherein:

[0113] The data acquisition module is used to acquire image data, depth point cloud data, and attitude data of the target area and align them to the same coordinate system in time and space.

[0114] The image processing module is used to extract anatomical feature points from image data and construct feature maps. The feature maps store the coordinates and feature vectors of each anatomical feature point. The feature vectors include the texture contrast, local concavity degree, and detection confidence of the anatomical feature points.

[0115] The image processing module is also used to determine the degree of concavity of each pixel in the image data based on the depth point cloud data, so as to generate a concavity saliency map, filter the initial candidate regions from the concavity saliency map, take the pixel with the largest degree of concavity in each initial candidate region as the candidate node, and construct a candidate map containing candidate feature vectors and topological connections. The candidate feature vector is the feature vector of the anatomical feature point that is closest to the candidate node.

[0116] The image processing module is also used to determine the matching template and matching score of the candidate image in the pre-built acupoint template library based on the candidate feature vector and topological connection, determine the stability weight of the candidate node based on the contour difference of the adjacent image data frames, determine the node confidence based on the degree of concavity, stability weight and matching score, and determine the high confidence candidate image based on the node confidence.

[0117] The acupoint localization module is used to determine the initial acupoint coordinates based on the topological consistency between the high-confidence candidate map and the standard acupoint topology map, and to dynamically compensate the initial acupoint coordinates based on the attitude data to obtain the final acupoint coordinates.

[0118] The specific implementation details of the above-mentioned image recognition-based intelligent acupuncture point positioning system have been explained in detail in the corresponding section of the image recognition-based intelligent acupuncture point positioning method, so they will not be repeated here.

[0119] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0120] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A method for locating intelligent acupuncture points based on image recognition, characterized in that, The method includes: Collect image data, depth point cloud data, and attitude data of the target area and align them to the same coordinate system in time and space. Anatomical feature points are extracted from the image data and a feature map is constructed. The feature map stores the coordinates and feature vectors of each anatomical feature point. The feature vectors include the texture contrast, local concavity degree, and detection confidence of the anatomical feature point. Based on the depth point cloud data, the degree of concavity of each pixel in the image data is determined to generate a concavity saliency map. Initial candidate regions are selected from the concavity saliency map, and the pixel with the largest degree of concavity in each initial candidate region is selected as a candidate node. A candidate map containing candidate feature vectors and topological connections is constructed. The candidate feature vector is the feature vector of the anatomical feature point that is closest to the candidate node. Based on the candidate feature vector and the topological connection, the matching template and matching score of the candidate image are determined in the pre-built acupoint template library. The stability weight of the candidate node is determined based on the contour difference of adjacent image data frames. The node confidence is determined based on the degree of indentation, the stability weight and the matching score. The high-confidence candidate image is determined based on the node confidence. The initial acupoint coordinates are determined based on the topological consistency between the high-confidence candidate map and the standard acupoint topology map, and the initial acupoint coordinates are dynamically compensated based on the posture data to obtain the final acupoint coordinates. The determination of the stability weights of the candidate nodes based on the contour differences of adjacent image data frames includes: An edge detection algorithm is used to extract the edge contours of different pixel regions in adjacent image data frames, and the corresponding points between the edge contours of the pixel regions are determined to form a set of corresponding points. The distance between each corresponding point in the set of corresponding points is determined, the displacement degree of the pixel region is determined based on the maximum distance, and the stability weight of each candidate node in the pixel region is determined based on the region type and the displacement degree.

2. The intelligent acupuncture point location method based on image recognition according to claim 1, characterized in that, The acquisition of image data, depth point cloud data, and pose data of the target area, followed by spatiotemporal alignment, includes: The image data is acquired by an RGB camera triggered by a unified clock, the depth point cloud data is acquired by a depth sensor, and the attitude data is acquired by an inertial measurement unit. The image data, the depth point cloud data, and the attitude data are then aligned to a unified sampling frequency based on a linear interpolation method. The intrinsic parameters of the RGB camera and the depth sensor are calibrated, the relative pose between the RGB camera and the depth sensor is calculated, and the depth point cloud data is transformed into the image coordinate system of the image data based on the relative pose using the intrinsic parameters as a reference. A right-handed coordinate system is established with the principal point of the image data as the origin, and the image data, the depth point cloud data, and the pose data are all mapped to the right-handed coordinate system.

3. The intelligent acupuncture point location method based on image recognition according to claim 1, characterized in that, The step of extracting anatomical feature points and constructing a feature map from the image data includes: An edge detection algorithm is used to initially extract the edge contour of the target region in the image data, and morphological closing operation is used to optimize the edge contour; Extract the anatomical feature points from the optimized edge contour and output the two-dimensional coordinates of the anatomical feature points in the image coordinate system where the image data is located; The target surrounding window of the anatomical feature point is determined, the image region corresponding to the target surrounding window is converted into a local grayscale image, and the texture contrast is determined based on the grayscale value of each pixel in the local grayscale image. Depth values ​​are extracted from the corresponding depth point cloud data based on the two-dimensional coordinates of the anatomical feature points, and the degree of local depression is determined based on the depth values ​​corresponding to the anatomical feature points. The detection confidence level is obtained by performing edge continuity analysis on the optimized edge contour; The vector formed by the texture contrast, the degree of local depression, and the confidence level is denoted as the feature vector of the anatomical feature point. The anatomical feature point and the feature vector are stored in a one-to-one correspondence to obtain the feature map.

4. The intelligent acupuncture point location method based on image recognition according to claim 1, characterized in that, The step of determining the degree of concavity of each pixel in the image data based on the depth point cloud data to generate a concavity saliency map includes: For each pixel in the image data, a depth value is extracted from the corresponding depth point cloud data based on the two-dimensional coordinates of the pixel in the image coordinate system. Obtain target pixels within the neighborhood of the current pixel, calculate the difference between the depth values ​​of the current pixel and each of the target pixels and average them, and record the degree of concavity of the current pixel; the target pixels are a preset number of pixels in the neighborhood arranged from the nearest to the farthest from the current pixel. The degree of concavity of each pixel in the image data is normalized to generate a corresponding concavity saliency map.

5. The intelligent acupuncture point location method based on image recognition according to claim 4, characterized in that, The step of selecting initial candidate regions from the concavity saliency map, and using the pixel with the largest concavity degree in each initial candidate region as a candidate node, to construct a candidate map containing candidate feature vectors and topological connections includes: The pixels with a concavity greater than a first preset threshold are obtained to obtain an initial candidate pixel set; Connectivity analysis is performed on the initial candidate pixel set to remove fragmented regions and obtain the initial candidate region; The pixels in the initial candidate region are used as candidate nodes. The candidate graph is constructed based on the Euclidean distance between the candidate nodes using k-nearest neighbor connection. The feature vector of the anatomical feature point that is closest to the candidate node is associated with the candidate feature vector.

6. The intelligent acupuncture point location method based on image recognition according to claim 1, characterized in that, The step of determining the matching template and matching score of the candidate image in the pre-built acupoint template library based on the candidate feature vector and the topological connection includes: For each body posture standard acupoint template in the pre-built acupoint template library, determine the first candidate matching score of the degree of indentation of each candidate node in the candidate image and the degree of indentation of the corresponding acupoint in the body posture standard acupoint template, and the second candidate matching score of the topological connection between the candidate node and the corresponding acupoint. Based on the first candidate matching score and the second candidate matching score of each candidate node, the candidate matching score between the candidate image and the body posture standard acupoint template is determined, and the body posture standard acupoint template with the candidate matching score greater than the second preset threshold is determined as a candidate template, thus obtaining a candidate template set; If the highest candidate matching score is less than the second preset threshold, calculate the product of the local concavity degree and the detection confidence of each candidate node in the candidate graph, and record it as the node optimization factor; Remove the candidate node with the smallest node optimization factor from the candidate graph, and redetermine the candidate matching score between the candidate graph after removing the candidate node and each candidate template; Repeat the process of determining the candidate matching score of the candidate graph and each candidate template after removing the candidate node, if the highest candidate matching score is less than the second preset threshold, until the candidate matching score is not less than the second preset threshold or the number of iterations reaches the preset number; The candidate template with the highest candidate matching score is determined as the matching template, and the candidate matching score of the matching template is determined as the matching score.

7. The intelligent acupuncture point location method based on image recognition according to claim 1, characterized in that, The step of determining node confidence based on the degree of indentation, the stability weight, and the matching score, and determining high-confidence candidate graphs based on the node confidence, includes: The product of the concavity degree of the candidate node and the normalized value of the matching score is calculated based on a preset weight ratio to obtain the first confidence factor of the candidate node. The stability weight of the candidate node is denoted as the second confidence factor. The product of the first confidence factor and the second confidence factor is calculated to obtain the confidence level of the node. Candidate nodes whose node confidence is greater than a third preset threshold are identified as high-confidence nodes, and a high-confidence candidate graph corresponding to the high-confidence nodes is constructed.

8. The intelligent acupuncture point location method based on image recognition according to claim 1, characterized in that, The determination of initial acupoint coordinates based on the topological consistency between the high-confidence candidate map and the standard acupoint topology map includes: Obtain the standard acupoint topology map of the target area, and construct the candidate anchor point topology map of the high-confidence candidate map; Obtain the actual distance of each connecting edge in the candidate anchor point topology graph, and determine the first topology factor based on the actual distance and the standard distance of the corresponding edge in the standard acupoint topology graph; Obtain the actual included angle between each connecting edge in the candidate anchor point topology map, and determine the second topology factor based on the actual included angle and the corresponding standard included angle in the standard acupoint topology map; The topological consistency between the candidate anchor point topology map and the standard acupoint topology map is determined based on the first topology factor and the second topology factor. If the topological consistency is not less than the fourth preset threshold, the candidate node corresponding to the highest confidence of the node in the high-confidence candidate graph is determined as the initial acupoint coordinates. If the topological consistency is less than the fourth preset threshold, the candidate node corresponding to the highest confidence node in the high confidence candidate graph is used as a benchmark. Based on the relative distance and direction between the corresponding acupoint and other acupoints in the standard acupoint topology graph, the coordinates of other candidate nodes in the high confidence candidate graph are corrected, and the topological consistency is recalculated. This step is repeated until the topological consistency is not less than the fourth preset threshold. Based on the topological consistency and coordinate correction of each candidate node, a comprehensive confidence score is determined, and the candidate node corresponding to the highest confidence score is determined as the initial acupoint coordinates.

9. An intelligent acupuncture point positioning system based on image recognition, characterized in that, The system includes: The data acquisition module is used to acquire image data, depth point cloud data, and attitude data of the target area and align them to the same coordinate system in time and space. An image processing module is used to extract anatomical feature points from the image data and construct a feature map; the feature map stores the coordinates and feature vectors of each anatomical feature point, and the feature vectors include the texture contrast, local concavity degree, and detection confidence of the anatomical feature point; The image processing module is further configured to determine the degree of concavity of each pixel in the image data based on the depth point cloud data, so as to generate a concavity saliency map, filter initial candidate regions from the concavity saliency map, take the pixel with the largest degree of concavity in each initial candidate region as a candidate node, and construct a candidate map containing candidate feature vectors and topological connections, wherein the candidate feature vector is the feature vector of the anatomical feature point that is closest to the candidate node; The image processing module is further configured to determine the matching template and matching score of the candidate image in the pre-built acupoint template library based on the candidate feature vector and the topological connection, determine the stability weight of the candidate node based on the contour difference of adjacent image data frames, determine the node confidence based on the degree of indentation, the stability weight and the matching score, and determine the high confidence candidate image based on the node confidence. The acupoint positioning module is used to determine the initial acupoint coordinates based on the topological consistency between the high-confidence candidate map and the standard acupoint topology map, and to dynamically compensate the initial acupoint coordinates based on the attitude data to obtain the final acupoint coordinates. The determination of the stability weights of the candidate nodes based on the contour differences of adjacent image data frames includes: An edge detection algorithm is used to extract the edge contours of different pixel regions in adjacent image data frames, and the corresponding points between the edge contours of the pixel regions are determined to form a set of corresponding points. The distance between each corresponding point in the set of corresponding points is determined, the displacement degree of the pixel region is determined based on the maximum distance, and the stability weight of each candidate node in the pixel region is determined based on the region type and the displacement degree.

Images