Ear acupoint and body surface acupoint individual positioning method based on image recognition

Abstract

The application discloses an auricular point and body surface point individual positioning method based on image recognition and belongs to the field of medical image processing and point positioning. In order to solve the problem that automatic positioning of points in multiple parts is influenced by body proportion difference, posture change and shooting angle, the application realizes the technical effect of stable positioning of points in multiple parts through image quality detection, part classification, region segmentation, reference point detection, rule atlas calling, individualized anatomical coordinate system construction and constraint mapping, and outputs point coordinates, region masks and positioning reliability.

Description

Technical Field

[0001] This invention relates to the field of medical image processing and acupoint location, and particularly to a method for individualized location of auricular and body surface acupoints based on image recognition. Background Technology

[0002] The location of acupoints on the body surface and auricle plays an important role in traditional Chinese medicine diagnosis and treatment, rehabilitation therapy, acupoint stimulation device guidance, and health management. Existing location methods mostly rely on physician experience, body surface proportion measurement, and manual reference point judgment. However, the selection of reference points, proportion rules, and local anatomical structures vary greatly among different target areas such as the neck and shoulders, abdomen, limbs, and ears, making it difficult to achieve consistency in manual location and automated reproduction.

[0003] With the development of image recognition and human key point detection technologies, using images of target areas for acupoint-assisted localization has become a feasible approach. However, different heights, body types, skin textures, pose variations, and shooting angles can alter the contours of the target area and the positional relationship of anatomical reference points. Single templates or fixed-ratio mappings are prone to localization shifts. Furthermore, when image quality, segmentation boundary stability, and multi-frame consistency are not uniformly incorporated into confidence calibration, the system struggles to distinguish between directly output localization results and candidate results requiring verification.

[0004] Therefore, there is a need for an individualized method for locating auricular and body surface acupoints that can overcome the shortcomings of the existing technologies. Summary of the Invention

[0005] One objective of this invention is to propose an image recognition-based individualized localization method for auricular and body surface acupoints. Addressing the problems in existing technologies such as the difficulty in unifying reference points for acupoints across multiple locations and the susceptibility of automated recognition to variations in body proportions and shooting posture, this invention proposes a technical solution involving target location image quality detection, location classification, body surface region segmentation, anatomical reference point detection, standard acupoint rule atlas retrieval, individualized anatomical coordinate system construction, and location-adaptive anatomical constraint diagrams and uncertainty calibration for rule mapping. This invention offers the technical advantages of outputting point coordinates, region masks, and location reliability, while also supporting verification prompts.

[0006] This invention provides an image recognition-based method for individualized localization of auricular and body surface acupoints, comprising: S1, acquiring images of target body parts and performing image quality detection to obtain qualified target images and acquisition posture parameters; S2, classifying the qualified target images into target body parts to obtain body part categories and category confidence scores; S3, segmenting the qualified target images into body surface regions based on body part categories to obtain body surface region masks and segmentation boundaries; S4, detecting anatomical reference points based on body surface region masks and segmentation boundaries to obtain a set of reference points and key point confidence scores; S5, calling up standard acupoint rule maps that match the body part categories and constructing an individualized anatomical coordinate system based on the set of reference points; S6, mapping the standard acupoint rule maps to the individualized anatomical coordinate system through a body part adaptive anatomical constraint map and an uncertainty calibration module, and outputting acupoint localization results, which include the point coordinates, region masks, and localization confidence scores of auricular or body surface acupoints.

[0007] Optionally, S1 includes:

[0008] Calculate the sharpness index, exposure index, occlusion pixel ratio, target area occupancy ratio, and the angle between the main axis of the human contour and the acquisition coordinate axis for the target human body image.

[0009] A quality score is generated based on the aforementioned sharpness index, exposure index, occlusion pixel ratio, target area occupancy ratio, and included angle.

[0010] When the quality score meets the preset quality threshold, the image of the target human body part is determined as a qualified target image, and the included angle and the proportion of the target area in the image are used as the acquisition posture parameters.

[0011] Optionally, S2 includes:

[0012] The qualified target image is input into the trained target part classification model to extract contour proportion features, texture distribution features and local key morphological features;

[0013] Based on the contour proportion features, texture distribution features, and local key morphological features, output the category of one of the following parts: ear, shoulder and neck, abdomen, or limbs, and the category confidence level;

[0014] When the category confidence does not meet the preset category threshold, the candidate body part categories and their corresponding category confidences that are ranked within the preset range by the category confidence output by the target body part classification model are passed to the subsequent body surface region segmentation stage. This allows the subsequent body surface region segmentation stage to generate candidate body surface region masks and candidate segmentation boundaries according to the candidate body part categories, and then pass the candidate body surface region masks and candidate segmentation boundaries to the subsequent anatomical reference point detection stage.

[0015] Optionally, S3 includes:

[0016] Based on the location category, a segmentation template and boundary constraint parameters are selected to segment the body surface region of the qualified target image, thereby obtaining a body surface region mask.

[0017] Morphological closing operations and connected component filtering are performed on the surface region mask to generate segmentation boundaries;

[0018] Based on the registration of consecutive image frames of the same target area acquired during the image acquisition stage, the displacement variance of the corresponding boundary points in the same acquisition coordinate space is used to determine the segmentation boundary stability index.

[0019] Optionally, S4 includes:

[0020] The reference point type is determined according to the body part category, and the anatomical reference point corresponding to the reference point type is detected within the area defined by the body surface mask and segmentation boundary.

[0021] The types of reference points include the auricle contour point, acromion point, suprasternal notch point, umbilicus point, cubital fossa point, wrist crease point, knee eye point, or ankle tip point;

[0022] Based on the local response peak value, the neighborhood response difference, and the constraint results of the reference point falling into the mask of the body surface region, the key point confidence of each anatomical reference point is calculated.

[0023] Optionally, S5 includes:

[0024] Using the anatomical reference points in the set of reference points as coordinate reference points, the local coordinate axes, scale parameters, and rotation parameters are determined according to the category of the body part to form an individualized anatomical coordinate system;

[0025] The acupoint names, acupoint ratio rules, and reference point topology relationships in the standard acupoint rule atlas are called up, and the acupoint ratio rules are converted into the initial acupoint coordinates in the individualized anatomical coordinate system.

[0026] Optionally, S6 includes:

[0027] Construct a site-adaptive anatomical constraint map, wherein the nodes of the site-adaptive anatomical constraint map include the anatomical reference points and candidate acupoints, and the edges include anatomical adjacency relationships, proportional distance constraints, and directional constraints;

[0028] The uncertainty vector is calculated based on the confidence level of the key points, the stability index of the segmentation boundary determined by the displacement variance of the corresponding boundary points in the same acquisition coordinate space after registration of continuous image frames of the same target area acquired in the image acquisition stage, and the displacement consistency index of the corresponding reference points determined by the coordinate difference of the corresponding reference points in the individualized anatomical coordinate system after registration of the continuous image frames.

[0029] Input the uncertainty vector into the calibration mapping table to obtain the coordinate confidence interval and region mask expansion coefficient of each candidate acupoint;

[0030] The initial acupoint coordinates are corrected according to the coordinate confidence interval to obtain the acupoint coordinates, and the neighborhood of the candidate acupoint is expanded according to the region mask expansion coefficient to obtain the acupoint region mask.

[0031] The acupoint coordinates and the acupoint region mask are used to form an individualized deformation mapping and output the acupoint location result;

[0032] Furthermore, the uncertainty vector is also corrected by a time-series correction index, which is determined by the following method: registering consecutive image frames of the same target area acquired during the image acquisition stage in chronological order, and calculating the coordinate difference of the corresponding reference points in the individualized anatomical coordinate system.

[0033] The temporal correction index is generated by normalizing and fusing the variance of the coordinate difference, the stability index of the segmentation boundary determined by the displacement variance of the corresponding boundary points in the continuous frames, and the amount of acquisition posture change determined by the acquisition posture parameter difference of the continuous image frames.

[0034] Furthermore, when the quality score generated by the sharpness index, exposure index, occlusion pixel ratio, target area occupancy ratio, and the angle between the main axis of the human body contour and the acquisition coordinate axis does not meet the preset quality threshold, a re-acquisition prompt is output. The re-acquisition prompt includes the target part and the type of quality failure.

[0035] When the category confidence does not meet the preset category threshold, the candidate part category and the corresponding category confidence are passed to the subsequent body surface region segmentation stage and the subsequent anatomical reference point detection stage to generate candidate body surface region masks, candidate segmentation boundaries and candidate reference point sets. The candidate part category is the part category whose category confidence ranking is within the preset range output by the target part classification model. The candidate acupoint location results are generated by the part adaptive anatomical constraint map and uncertainty calibration module and marked as pending verification.

[0036] When the width of the coordinate confidence interval exceeds the preset width threshold, the corresponding acupoint location result is marked as pending review, and the reference point missing type and corresponding candidate acupoint are output.

[0037] If none of the above conditions are triggered, the acupoint location result will be output for display or subsequent device control.

[0038] Furthermore, the acupoint location results are output in a structured manner according to the target location identifier, acupoint identifier, point coordinates, region mask, location confidence, coordinate confidence interval, and status to be verified.

[0039] The location confidence is calculated based on the key point confidence, the segmentation boundary stability index, the temporal correction index determined by the variance or normalized value of the coordinate difference of the corresponding reference points in the individualized anatomical coordinate system after registration of continuous image frames of the same target part acquired in the image acquisition stage, and the width of the coordinate confidence interval, and establishes an association record with the point coordinates and the region mask under the same acupoint identifier.

[0040] The beneficial effects of this invention are:

[0041] 1. By first performing image quality detection and target part classification, and then completing body surface region segmentation and anatomical reference point detection based on the part category, the localization process of different parts can obtain their own suitable reference points and region boundaries under a unified image processing chain, reducing localization differences caused by human experience and fixed measurement methods.

[0042] 2. By calling the standard acupoint rule map and constructing an individualized anatomical coordinate system, the acupoint ratio rules are mapped to the coordinate framework of the individual target part, so that the acupoint coordinates and region mask can be adjusted according to the scale, direction and reference point position of the target part, improving the consistency of positioning in cross-body shape and cross-pose scenarios.

[0043] 3. By using the adaptive anatomical constraint map of the part and the uncertainty calibration module, the confidence of key points, the stability of segmentation boundaries, the consistency of displacement of corresponding reference points, the temporal correction index and the coordinate confidence interval are introduced into the positioning result output. This enables the system to generate positioning confidence and output a re-acquisition prompt or a pending review status when the quality is unqualified, the category has low confidence or the coordinate interval is too wide. Attached Figure Description

[0044] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0045] Figure 1 This is a flowchart of an image recognition-based method for individualized localization of auricular and body surface acupoints.

[0046] Figure 2 This is a flowchart of step S6 of the present invention. Detailed Implementation

[0047] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0048] refer to Figures 1-2A method for individualized localization of auricular and body surface acupoints based on image recognition includes: S1, acquiring images of target body parts and performing image quality detection to obtain qualified target images and acquisition posture parameters; S2, classifying the qualified target images into target body parts to obtain body part categories and category confidence scores; S3, segmenting the qualified target images into body surface regions based on body part categories to obtain body surface region masks and segmentation boundaries; S4, detecting anatomical reference points based on body surface region masks and segmentation boundaries to obtain a set of reference points and key point confidence scores; S5, calling up standard acupoint rule maps that match the body part categories and constructing an individualized anatomical coordinate system based on the set of reference points; S6, mapping the standard acupoint rule maps to the individualized anatomical coordinate system through a body part adaptive anatomical constraint map and uncertainty calibration module, and outputting acupoint localization results, which include the point coordinates, region masks, and localization confidence scores of auricular or body surface acupoints.

[0049] In this specific embodiment, S1 includes:

[0050] The acquisition terminal acquires color images of target human body parts in fixed focal length and fixed exposure modes. The image The resolution is ,in Indicates the image width in pixels. Indicates the image height in pixels;

[0051] Image Perform size normalization to And convert to grayscale. ,in Indicates the image The grayscale image obtained by calculating the brightness channel;

[0052] For images Input human foreground segmentation model Obtain the human foreground probability map and threshold it to generate a human foreground mask. ,in For a semantic segmentation network trained offline and with parameters fixed, the network structure adopts an encoder-decoder configuration of MobileNetV3-Large encoder and DeepLabv3+ decoder, with an output stride of 16 and a dilated convolution rate set of [missing information]. The model input is a three-channel... The image is processed by a model, which outputs a single-channel foreground probability map. A threshold of 0.5 is applied, and the image is then bilinearly upsampled back to its original resolution in the output space to obtain the mask. In the mask, pixels with a value of 1 are defined as foreground pixels of the human body, and pixels with a value of 0 are defined as background pixels.

[0053] According to the mask Calculate the map coverage ratio of the target area ,in Represents the number of pixels in the foreground of the human body and the total number of pixels in the image. The ratio, and As one of the acquired attitude parameters, it is output.

[0054] Based on mask Extract the human foreground contour point set and perform principal axis analysis on the contour point set to obtain the human contour principal axis direction vector. The principal axis analysis is achieved by centering the contour point coordinates, calculating the covariance matrix, and taking the eigenvector corresponding to the largest eigenvalue.

[0055] The angle is obtained by comparing the angle between the principal axis direction vector of the human body contour and the acquisition coordinate axis. ,in The angle of deflection of the principal axis of the human body contour relative to the acquisition coordinate axis is expressed in degrees. The acquisition coordinate axis is taken as the vertical axis of the image coordinate system. As one of the acquired attitude parameters, it is output.

[0056] In the mask grayscale image within a defined foreground region Calculate the Laplacian operator response and use its variance as a sharpness index. ,in Indicates the intensity of high-frequency information related to the edges and textures of the foreground region;

[0057] In the mask Exposure index is obtained by statistically analyzing the average brightness and the proportion of overexposed pixels within a defined foreground area and then fusing them. ,in This indicates the degree to which the image brightness falls within the effective dynamic range and the degree of overexposure and underexposure is relatively low.

[0058] For images Input occlusion segmentation model Obtain the occlusion probability map and threshold it to generate an occlusion mask. ,in This is a pixel-level binary classification segmentation network that is trained offline and has its parameters fixed. The network structure uses a skip connection structure between a ResNet18 encoder and a U-Net decoder, and the model input is a three-channel network. The image is output as a single-channel occlusion probability map by the model. The threshold is set to 0.5, and the image is upsampled back to the original resolution to obtain the occlusion mask. In the occlusion mask, a pixel with a value of 1 is defined as an occlusion pixel;

[0059] Human foreground mask Internal calculation of occlusion pixel ratio ,in Indicates satisfaction and The number of pixels and satisfying The ratio of the number of pixels;

[0060] Clarity index Exposure indicators Occlusion pixel ratio Target area occupies a certain percentage of the map and included angle Truncate and linearly normalize according to their respective lower and upper limits to obtain and The quantities marked with wavy lines represent those normalized to... The dimensionless value, and Adopt relative to the maximum permissible deflection angle Normalize the degree and truncate to Obtained in this way;

[0061] Based on the normalized indicators, a quality score and the proportion of overexposed pixels are generated and fused to obtain the exposure index. ,in This indicates the degree to which the image brightness falls within the effective dynamic range and the degree of overexposure and underexposure is relatively low.

[0062] For images Input occlusion segmentation model Obtain the occlusion probability map and threshold it to generate an occlusion mask. ,in This is a pixel-level binary classification segmentation network that is trained offline and has its parameters fixed. The network structure uses a skip connection structure between a ResNet18 encoder and a U-Net decoder, and the model input is a three-channel network. The image is output as a single-channel occlusion probability map by the model. The threshold is set to 0.5, and the image is upsampled back to the original resolution to obtain the occlusion mask. In the occlusion mask, a pixel with a value of 1 is defined as an occlusion pixel;

[0063] Human foreground mask Internal calculation of occlusion pixel ratio ,in Indicates satisfaction and The number of pixels and satisfying The ratio of the number of pixels;

[0064] Clarity index Exposure indicators Occlusion pixel ratio Target area occupies a certain percentage of the map and included angle Truncate and linearly normalize according to their respective lower and upper limits to obtain and The quantities marked with wavy lines represent those normalized to... The dimensionless value, and Adopt relative to the maximum permissible deflection angle Normalize the degree and truncate to Obtained in this way;

[0065] A quality score is generated based on the normalized indicators. The quality score Calculate using the following formula:

[0066] ;

[0067] in Indicates quality score, Represents the sharpness weight and assigns a value. This represents the normalized sharpness index. Indicates the exposure weight and its value. This represents the normalized exposure index. Indicates the occlusion weight and its value. Indicates the normalized proportion of occluded pixels. Indicates the weight of the proportion of the map and its value. This indicates the proportion of the normalized target region in the map. Indicates the attitude deflection weights and their values. This represents the normalized angle deflection.

[0068] Set quality threshold ,in This represents the threshold for determining whether an image quality is acceptable. Image Mark the qualified target image and output the acquisition posture parameters. For use in subsequent steps, when Image Marked as unqualified and will not proceed to the subsequent target part classification and processing process.

[0069] In this specific embodiment, S2 includes:

[0070] qualified target image Processed according to the same size normalization method as S1 Three-channel input tensor ,in Indicates the image Linear scaling of pixel values ​​to Subtract the mean vector of the training set from each channel. And divided by the standard deviation vector of the training set The resulting standardized tensor, the mean vector This represents the mean of the red, green, and blue channels, and the values ​​are respectively... The standard deviation vector This represents the standard deviation of the red, green, and blue channels, with values ​​of [values ​​to be filled in]. ;

[0071] The input tensor Input the target part classification model obtained through training and with fixed parameters Obtain the probability vector of part category ,in This describes a convolutional neural network classification model used to categorize target body parts into four types: ear, neck and shoulder, abdomen, and limbs. The backbone network of the model is ResNet34, and the input layer is... The convolution is performed with a stride of 2, followed by a max-pooling layer, and the number of backbone residual blocks is... After global average pooling, the output dimension of the fully connected layer is 4, and the probability vector is output through the Softmax function. ,in Indicates the probability of ear category. Indicates the probability of the shoulder and neck category. Indicates the probability of the abdominal category. This represents the probability of each limb category, and ;

[0072] The target part classification model During the training phase, the cross-entropy loss function is used, the optimizer is Adam, and the learning rate is [value missing]. The batch size is 32, the number of training rounds is 60, and the parameter with the highest accuracy on the validation set is used as the fixed parameter.

[0073] During the reasoning phase, from the probability vector The category with the highest probability is taken as the part category. ,in These represent the ear, neck and shoulder, abdomen, and limbs, respectively, and the highest probability value is used as the category confidence score. ,in Indicates the reliability of the target part classification;

[0074] Meanwhile, in order to meet the feature extraction requirements, in the model During the inference process, three types of feature vectors for interpretability recording are derived from a specified intermediate layer and bound to the classification results for storage, among which the contour proportion feature... From input tensor Corresponding foreground mask The aspect ratio, area ratio, and principal axis direction of the circumscribed rectangle Composition, texture distribution characteristics The key local morphological features are obtained by global average pooling from the output feature map of the third-stage residual block of the backbone network. The feature map output from the fourth stage residual block of the backbone network is processed by... The convolutional dimension is reduced to 128 dimensions, and then global average pooling is used to obtain the result.

[0075] Set category threshold ,in The threshold representing the reliability of determining the body part category, when Output unique part category at time and corresponding category confidence levels And enter S3;

[0076] when When, for probability vector Sort by probability value from largest to smallest and take the top results. Each category is used as a set of candidate part categories. ,in Represents the set of candidate part categories. This represents the candidate part category with the highest probability. This represents the candidate body part category with the second highest probability, and also outputs the corresponding set of confidence scores for each candidate category. ,in express The class probability and equal to express The category probability is equal to the second largest probability value, and... Along with qualified target images The results are then passed to step S3, whereby step S3 generates candidate surface region masks and candidate segmentation boundaries according to the candidate site category, and continues to pass them to the subsequent anatomical reference point detection process to support the output of multi-branch localization calculation results in the case of low confidence classification.

[0077] In this specific embodiment, S3 includes:

[0078] Receiving part category and qualified target images ,in ear, neck, abd, limb The qualified target images represent the ear, neck and shoulder, abdomen, and limbs, respectively. Consistent with step S2, it is preprocessed into an input tensor. Simultaneously read the human foreground mask Acquire attitude parameters ,in This represents the normalized three-channel image tensor. Indicates the foreground mask for the human body. This represents the angle between the principal axis of the human body contour and the acquisition coordinate axes. Indicates the percentage of the map area occupied by the target region;

[0079] According to body part category Select segmentation template With boundary constraint parameter set ,in Indicates in The normalized coordinate system represents the prior mask of the standard body surface region obtained offline statistically for this part. The prior mask is obtained by affine alignment of manually labeled masks of the same part in the training set, calculating the mean value pixel by pixel, and then binarizing it with a threshold of 0.5. This represents the set of parameters used to restrict the form of the segmentation result, and is set to . ,in This represents the side length (in pixels) of the structuring element used in morphological closing operations and takes the value... , This represents the minimum area threshold number of pixels to retain for connected components, and its value is... , and Represent the lower and upper limits of the aspect ratio of the bounding rectangle of the body surface region, respectively, and take values ​​of 1000 and 2000. , ;

[0080] Input tensor Input and Part Categories Matching body surface region segmentation model Obtaining a probability map of the body surface region ,in Indicates the target area The semantic segmentation network is trained and its parameters are fixed. The network structure adopts a skip connection structure of ResNet18 encoder and U-Net decoder. The number of encoder output channels is as follows: The decoder upsamples at each level and concatenates them with the corresponding coding layer features, finally... Convolution Output Single Channel Probability Map The model is trained using a loss function that is the sum of the Dice loss and the binary cross-entropy loss. The optimizer is Adam, and the learning rate is... The batch size is 16, the number of training rounds is 80, and the parameter with the highest Dice coefficient is used as the fixed parameter on the validation set.

[0081] Probability map of body surface area Human foreground mask Perform template constraint clipping within the template to retain only those falling into the template. The probability response within the expansion region, wherein the expansion radius of the expansion region is fixed at 15 pixels, thereby suppressing false detections of non-target areas;

[0082] The cropped probability map is thresholded. Binarization yields the initial mask for the body surface region. ,in A pixel representing a mask of the body surface region with a value of 1 is defined as a pixel of the target body surface region;

[0083] Masking of the initial body surface area Morphological closing operations are performed to fill small holes and connect narrow fractures. These closing operations are executed in the order of expansion followed by erosion, and the structuring element has a side length of [missing information]. Square structural elements;

[0084] Perform connected component marking on the mask after the closing operation and delete those with areas smaller than 100%. The connected components are calculated, and the aspect ratio of the bounding rectangle of the connected components is preserved, while only the aspect ratio falls within the bounding rectangle. The connected components are used to obtain the final body surface region mask. ;

[0085] Based on the final body surface region mask The outer contour is extracted using a boundary tracking algorithm, and the outer contour is arranged into segmentation boundaries according to pixel order. ,in The boundary represents a segmentation boundary and is composed of an ordered sequence of boundary points. The sequence of boundary points is fixed by resampling the outer contour with equal arc lengths. A set of boundary points are used to establish a correspondence between boundary points with the same name across frames;

[0086] To calculate the segmentation boundary stability index, during the image acquisition phase, the target image is compared with the boundary boundary. Simultaneously cache a set of consecutive image frames of the same target area. and ,in Indicates the first Frame image and With the same imaging parameters, the corresponding segmentation boundary is generated for each frame using the steps described above. And obtain the coordinates of the resampled boundary points of the same name. ,in Indicates the first The first frame segmentation boundary The two-dimensional coordinates of the corresponding boundary points in the image coordinate system of this frame;

[0087] A collection of consecutive image frames Registration is performed using frame 1 as the reference frame. The registration employs a two-dimensional similarity transformation model and maximizes the enhancement correlation coefficient in the human foreground mask. Estimate the transformation from each frame to the reference frame within the coverage area. ,in Indicates the first A similar transformation that maps frame coordinates to reference frame coordinates and includes rotation, scaling, and translation components;

[0088] Transform the coordinates of the corresponding boundary points in each frame. Mapped to the same acquisition coordinate space ,in Indicates the first Frame number The one-dimensional coordinates of the corresponding boundary points in the reference frame coordinate system;

[0089] The stability index of the segmentation boundary is determined based on the displacement variance of the corresponding boundary points in the same acquisition coordinate space. The stability index of the segmentation boundary Calculate using the following formula:

[0090] ;

[0091] in This represents the stability index of the segmentation boundary, and the smaller the value, the more stable the segmentation boundary. Indicates the number of boundary points with the same name and takes the value. Indicates the index of the boundary point with the same name. Indicates the frame index variance calculation, Indicates the number of consecutive image frames and takes the value. Indicates the first Frame number The two-dimensional coordinates of the corresponding boundary points in the reference frame coordinate system Indicates the reference frame number Two-dimensional coordinates of boundary points with the same name, Denotes the one-dimensional Euclidean norm;

[0092] When S2 outputs the candidate part category set At that time, select according to the above process respectively. and as well as and And the candidate body surface region masks were obtained respectively. With candidate segmentation boundary and its corresponding segmentation boundary stability index and will and and Or candidate results The data is passed to step S4 for anatomical reference point detection within the mask and boundary-defined area.

[0093] In this specific embodiment, S4 includes:

[0094] Receiver surface area mask With dividing boundary And receive part category and qualified target images ,in A pixel representing a binary mask of the target surface region with a mask value of 1 is defined as a pixel of the target surface region. Indicates by A sequence of dividing boundary points consisting of ordered boundary points. ear, neck These respectively represent the ears, shoulders and neck, abdomen, and limbs;

[0095] According to body part category Determine the set of reference point types ,when season Points on the auricle contour and on the boundary Sampled using equal arc lengths A contour reference point, when season acromion point, suprasternal notch And the shoulder peak point is constrained at the boundary. The suprasternal notch is constrained by the mask. Inside, when season navel And the umbilicus is constrained by the mask. Inside, when season Elbow crease, wrist crease, knee joint, ankle tip All four types of reference points are output as candidate reference points for subsequent steps to combine with confidence scores.

[0096] During reference point detection, the first step is to use a mask. Calculate its bounding rectangle and extend it by 10 pixels in all directions to obtain the detection cropping box. Then, extract the image... The detection sub-image is cropped within the detection clipping box and then bilinearly scaled. Forming model input tensors ,in This represents the normalized input tensor used for keypoint detection, and its normalization method is the same as that of the input tensor in step S2. Channel mean with standard deviation Consistent;

[0097] Will Input reference point detection model Obtain the set of reference point heatmaps ,in Indicates the category of body part The multi-keypoint heatmap regression network was trained and its parameters were solidified. The network backbone adopted HRNet-W32 and maintained a high-resolution branch parallel structure. The final output resolution was [resolution value missing]. The heatmap outputs a channel count equal to the number of reference points. ,when hour ,when hour ,when hour ,when hour ;

[0098] The model training uses mean squared error loss and generates supervised heatmaps with a two-dimensional Gaussian kernel, where the Gaussian standard deviation is taken as 2 heatmap pixels. The optimizer is Adam, and the learning rate is [value missing]. The batch size is 32, the number of training rounds is 100, and the parameter with the smallest average keypoint localization error is used as the fixed parameter on the validation set.

[0099] For each channel heatmap Perform a peak search to obtain heatmap coordinates ,in Indicates the reference point index. Indicates the first A reference point at The peak coordinates in the heatmap coordinate system are then inversely transformed back to the original image coordinate system according to the scaling ratio and the cropping box position to obtain the reference point coordinates. ,in Indicates the first The two-dimensional coordinates of each anatomical reference point in the original coordinate system. Represents the pixel value at the horizontal coordinate. Represents the pixel value on the ordinate;

[0100] When the reference point type is constrained to the boundary When going up, Projected onto the boundary Corrected coordinates obtained above The projection method is in Searching for and at each boundary point The boundary point with the smallest Euclidean distance is used as the boundary point. When the reference point type is constrained to the mask When inside, As And check Whether it falls into The region where the value is 1;

[0101] Calculate the keypoint confidence level for each reference point. The confidence score of key points is determined by combining the peak value of local response, the difference in neighborhood response, and the constraint of the reference point falling within the mask region of the body surface, and is calculated using the following formula:

[0102] ;

[0103] in Indicates the first The keypoint confidence scores for each anatomical reference point are given, and the range of values ​​is: This represents the weight of the peak term and its value is... This represents the normalized value of the peak response and is derived from the peak value of the heatmap. Linear truncation and normalization yielded the following: This represents the weight of the neighborhood difference term and takes the value of . Represents the normalized value of the neighborhood response difference and is given by Linear truncation and normalization yielded the following: This represents the set of eight neighboring offsets centered at the peak coordinates. This represents the weight of the mask constraint term and its value is... This represents the mask constraint result and when the coordinates are corrected. Falling into the body surface area mask The value is 1 when it is in the foreground region, and 0 otherwise.

[0104] All corrected reference point coordinates Forming a set of reference points , and the confidence level of the corresponding key points As an adjunct property of the set of reference points and together with Common output to steps Used to construct an individualized anatomical coordinate system and subsequent uncertainty calibration processing, when step S3 is in the candidate site category branch mode, the above detection process is repeated for each group of candidate body surface region masks and candidate segmentation boundaries, and candidate reference point set and candidate key point confidence set are output respectively.

[0105] In this specific embodiment, S5 includes:

[0106] Receive reference point set With key point confidence set And receive part category ,in Indicates by A set consisting of the corrected coordinates of anatomical reference points. Indicates the first The two-dimensional coordinates of each anatomical reference point in the original coordinate system. Indicates the first Key point confidence of each anatomical reference point These respectively represent the ears, shoulders and neck, abdomen, and limbs;

[0107] According to body part category Constructing an individualized anatomical coordinate system The individualized anatomical coordinate system From the origin Unit coordinate axis vector and Scale parameters and and by and The rotation parameters of the implicit representation are jointly determined, where This represents the coordinates of the origin of the individualized anatomical coordinate system within the original image coordinate system. Representing an individualized anatomical coordinate system Unit direction vector of axis, Representing an individualized anatomical coordinate system The unit direction vector of the axis and satisfying and Orthogonal Indicates along Pixel scale factor in direction, Indicates along Pixel scale factor in direction;

[0108] when At that time, from The middle screening meets the requirements The auricle contour points and their centroids are calculated as... The principal axis analysis of the selected auricle contour points yields the first principal direction as... And will The orientation is unified so that the angle between the orientation and the vertical axis of the original image is less than 10°. The direction is to eliminate symbolic ambiguity, so that To be Rotate clockwise The obtained unit vectors, using a right-handed direction convention, are used to project the selected auricle contour points onto... and The above is determined by the difference between the maximum and minimum projected values. and Make them equal to the pixel span of the auricle in both axial directions;

[0109] when At that time, from Take the coordinates of the shoulder peak. Coordinates of the suprasternal notch Both are required to have a key point confidence level of not less than 0.5. Set as ,Will Set as from point to The normalized direction vector and Set as counterclockwise Rotate the unit vector while using The dot product with the original graph's vertical axis is positive, serving as a directional constraint to eliminate ambiguity of sign. Let... Furthermore, the body surface region mask is based on the output of S3. exist Determining the projected span in the direction ;

[0110] when At that time, from Take the coordinates of the umbilicus Furthermore, it requires that the confidence level of its key points be no less than 0.5. Set as Based on body surface region mask The first principal direction is obtained by performing principal axis analysis on the foreground pixel coordinates. And constrain the sign according to the direction where the dot product with the vertical axis of the original graph is positive, let for clockwise Rotate the unit vector, let and These are respectively equal to the foreground pixels of the mask in and Directional projection span;

[0111] when At that time, from When reading the nest wrist creases , knee eye point With ankle tip And read the confidence scores of their key points respectively. and If the confidence level of each key point is not less than 0.5, then... Set as And will Set as from point to The normalized direction vector and The positive dot product with the original graph's vertical axis serves as a directional constraint. If the above conditions are not met, and and If the confidence level of each key point is not less than 0.5, then... Set as And will Set as from point to The normalized direction vector and The positive dot product with the original graph's vertical axis serves as a directional constraint; if neither of these conditions is met, then... Set as a mask for the body surface area The centroid of the mask is used as the first principal direction obtained by analyzing the principal axis of the mask foreground pixel. And similarly, a directional constraint is applied that has a positive dot product with the vertical axis of the original image, letting... for clockwise Rotate the unit vector, let Take the Euclidean distance between two selected anatomical reference points, or, if no two points are available, take the distance between the mask foreground pixels. The projected span of the direction, let Take the foreground pixel of the mask The projected span of the direction is used to characterize the limb width;

[0112] In obtaining the individualized anatomical coordinate system Then, call the part category. Matching standard acupoint rule map ,in The data is stored in a structured rule-based table in a local database, categorized by body part. Each acupoint record must contain at least an acupoint identifier. Acupoint names Reference point topological relationships and the rules of acupoint ratio ,in Indicates an index of acupoints. This is used to specify the set of anatomical reference point types associated with the acupoint and their role in the coordinate axis construction. Used to give the acupoint in an individualized anatomical coordinate system Normalized coordinates under and The numerical values ​​are obtained and solidified by parsing the acupoint ratio rules and the topological relationship of the reference points during the offline construction of the rule map.

[0113] For each acupoint record, based on Normalize its coordinates Convert to the initial acupoint coordinates in the original image coordinate system And calculate according to the following formula:

[0114] ;

[0115] in Indicates the first The initial two-dimensional coordinate vectors of each acupoint in the original coordinate system. This represents the two-dimensional coordinate vector of the origin of the individualized anatomical coordinate system in the original image coordinate system. Indicates along Pixel scale factor in direction, Indicates the first Acupoints in the individual anatomical coordinate system lower edge Normalized scale coordinates of the axes, Representing an individualized anatomical coordinate system Unit direction vector of axis, Indicates along Pixel scale factor in direction, Indicates the first Acupoints in the individual anatomical coordinate system lower edge Normalized scale coordinates of the axes, Representing an individualized anatomical coordinate system Unit direction vector of axis;

[0116] All acupoints Forming the initial set of acupoints And together with individualized anatomical coordinate system The output is sent to step S6 for subsequent site adaptive anatomical constraint map construction and uncertainty calibration mapping.

[0117] In this specific embodiment, S6 includes:

[0118] Receive individualized anatomical coordinate system With the initial set of acupoints ,in Indicates origin Unit coordinate axis vector and Scale parameters and A defined coordinate system Indicates the presence of acupoint markers. Acupoint names with initial acupoint coordinates The set, Indicates acupoint index and Indicates the first The two-dimensional coordinate vectors of each acupoint in the original coordinate system;

[0119] Receiver surface area mask Dividing Boundaries With the stability index of the segmentation boundary ,in This represents a binary mask representing the target surface region. Represents the sequence of dividing boundary points. This represents the stability index obtained from the displacement variance of the corresponding boundary points.

[0120] Receive reference point set With key point confidence set ,in Indicates the first Two-dimensional coordinates of an anatomical reference point Indicates the first Key point confidence of each anatomical reference point Indicates the number of reference points;

[0121] When quality rating Quality threshold not met If this happens, this step will directly output a prompt to re-collect data. ,in It includes target area identification and quality failure type identification. The quality failure type identification is determined by a single threshold of sharpness index, exposure index, occlusion pixel ratio, target area occupancy ratio and included angle, and is used to drive the acquisition end to adjust focus, exposure, occlusion and framing range.

[0122] when When calling the part category Matching standard acupoint rule map And construct a site-adaptive anatomical constraint map ,in It is a set of nodes that includes reference point nodes and candidate acupoint nodes. The reference point nodes are composed of... Given candidate acupoint nodes by Given, The edge is a set of edges, and each edge carries an edge type and edge parameters. The edge types include three categories: anatomical adjacency, proportional distance constraint, and direction constraint. The edge parameters are derived from the regular graph. The reference point topological relationship and acupoint ratio rules were analyzed to obtain the results.

[0123] based on Index of each acupoint Calculate the uncertainty vector ),in Indicates acupoints The inverse vector of the keypoint confidence scores of directly connected reference point nodes, and taken as... Indicates acupoints The mean confidence level of the key points of the reference points on which they are relied upon and the corresponding The average is obtained. Indicates the instability of the partition boundary and is determined by Through the upper limit Linear normalization and truncation to get, The inverse vector of the displacement consistency index of the same reference point is formed by registering consecutive image frames of the same target location during the image acquisition stage. Calculate the coordinate difference between corresponding reference points, take their root mean square, and then apply the upper limit. Linear normalization and truncation to get, This represents the time-series correction index and the normalized variance of the difference in coordinates of corresponding reference points in consecutive frames, the normalized value of the segmentation boundary stability index, and the value derived from the acquired attitude parameters. The normalized values ​​of the acquired pose changes obtained from the inter-frame difference are weighted. Merge and cut off to get;

[0124] Uncertainty vector Input calibration mapping table Obtain the half-width of the coordinate confidence interval With the area mask expansion coefficient ,in For data tables that are built and persisted offline, the offline build process will... Each component within the interval The bins are divided into four equal sections to form a four-dimensional bin index, and then labeled according to acupoints. Store them separately, and calculate the acupoint localization error for each validation set sample that falls into each bin. The quantile is used as the half-width of the confidence interval for the coordinates. At the same time, select the option that satisfies "based on the reference radius". The expansion can cover the above. The minimum expansion coefficient of the quantile error is used as the region mask expansion coefficient. The reference radius The ears are defined as 6 pixels, the shoulders and neck as 10 pixels, the abdomen as 12 pixels, and the limbs as 10 pixels.

[0125] In obtaining Later Perform constraint mapping solution to obtain the initial acupoint coordinates Corrected to acupoint coordinates The constraint mapping solution employs bounded weighted least squares optimization and uses... As the initial value, the optimized residual consists of the proportional distance constraint residual, the orientation constraint residual, and the dissecting adjacency smoothing residual, with their respective weights set to... Compared to 1, the number of optimization iterations is set to 30, and after each iteration, [the following is omitted as the original text is incomplete and cannot be translated]. Projection back Centered on and with radius The feasible region is used to achieve the coordinate confidence interval constraint;

[0126] After obtaining the coordinates of the acupoints With expansion coefficient Subsequently, a mask is generated for the acupoint area. The acupoint area mask In the original coordinate system Draw a circle with a radius of centered on the center. The circular neighborhood and the mask of the body surface region The intersection is used to avoid crossing boundaries into non-surface areas;

[0127] Will and Together they form individualized deformation mapping The individualized deformation mapping Using a set of reference points Acupoints To achieve the piecewise affine mapping of control points, the piecewise method involves performing Delaunay triangulation on the control points in the original coordinate system and calculating the affine transformation matrix from the normalized coordinates of the standard regular map to the original coordinates within each triangle, thereby providing a continuous mapping for subsequent display overlay and device control.

[0128] Generate a pending review status based on the pending review strategy. ,in and This indicates that the process is pending review, and the category confidence level in step S2 is... Category threshold not met And enter the candidate part category set In multi-branch processing, this step is performed separately for each candidate site category branch, and all acupoints within that branch are output and marked as... When the width of the coordinate confidence interval Exceeding the preset width threshold Mark the corresponding acupoints as pixels. ,in Take as And based on acupoints Among the associated reference point types, those that satisfy The reference point generates a reference point missing type identifier and is matched with the acupoint identifier. Related output;

[0129] Calculate location reliability and acupoint coordinates Masking of acupoint areas Establish the same acupoint identifier The associated records below, the location reliability Calculate using the following formula:

[0130] ;

[0131] in Indicates the first Location reliability of acupoints This indicates that the input value will be truncated to... The truncation function, Indicates the confidence weight of key points and takes Indicates acupoints Mean confidence level of key points on reference points Represents the stability weights of the partition boundary and takes This indicates the stability index of the segmentation boundary. By upper limit Normalize and truncate to The normalized result shows that the larger the value, the more unstable it is. Indicates the timing correction weight and takes This indicates that the time series correction index is normalized and truncated to... The larger the value, the more pronounced the timing inconsistency. Represents the coordinate confidence interval weights and takes This indicates the width of the coordinate confidence interval. By width threshold Normalize and truncate to The larger the value, the wider the interval;

[0132] The acupoint location results are output and recorded in a structured format. Output, the structured output record At least include the target area identification. Acupoint markings Acupoint coordinates Acupoint area mask Location reliability Coordinate confidence interval Status pending review ,in Defined as Centered on and half-width is Align the axis with the rectangular area and Record in the form of This represents the component representation of the acupoint coordinate vector.

[0133] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

[0134] This invention organizes the quality detection, classification, segmentation, anatomical reference point detection, and acupoint rule mapping of target area images into a continuous data processing chain, so that the output of each stage can serve as the input or calibration condition for the next stage, thereby providing a clear data source for the point coordinates, region masks, and location reliability of auricular acupoints and body surface acupoints.

[0135] This invention introduces a site-adaptive anatomical constraint map and an uncertainty calibration module in the acupoint mapping stage. By jointly constraining individualized deformation mapping through reference point topology of different sites, acupoint ratio rules, key point confidence, segmentation boundary stability, and continuous frame temporal correction, it is beneficial to maintain stable output of acupoint localization results under the conditions of multiple body types, multiple poses, and multiple site images.

Claims

1. A method for individualized localization of auricular and body surface acupoints based on image recognition, characterized in that, include: S1. Acquire images of the target human body parts and perform image quality detection to obtain qualified target images and acquisition posture parameters; S2. Classify the target parts of the qualified target image to obtain the part category and category confidence; S3. Segment the body surface region of the qualified target image according to the part category to obtain the body surface region mask and segmentation boundary; S4. Detect anatomical reference points based on the surface region mask and segmentation boundary to obtain the reference point set and key point confidence. S5. Call the standard acupoint rule map that matches the location category and construct an individualized anatomical coordinate system based on the reference point set. S6. Map the standard acupoint rule map to the individualized anatomical coordinate system through the location adaptive anatomical constraint map and uncertainty calibration module, and output the acupoint location results. The acupoint location results include the point coordinates, region mask and location confidence of the ear acupoint or body surface acupoint.

2. The method according to claim 1, characterized in that, S1 includes: The system calculates the sharpness index, exposure index, occlusion pixel ratio, target area occupancy ratio, and the angle between the main axis of the human contour and the acquisition coordinate axis for the image of the target human body part. A quality score is generated based on the sharpness index, exposure index, occlusion pixel ratio, target area occupancy ratio, and angle. When the quality score meets a preset quality threshold, the image of the target human body part is determined as a qualified target image, and the angle and target area occupancy ratio are used as acquisition posture parameters.

3. The method according to claim 1, characterized in that, S2 include: The qualified target image is input into the trained target part classification model to extract contour proportion features, texture distribution features, and local key morphological features. Based on the contour proportion features, texture distribution features, and local key morphological features, the model outputs a part category (ear, shoulder and neck, abdomen, or limbs) and a category confidence score. When the category confidence score does not meet a preset category threshold, the candidate part categories and their corresponding category confidence scores, ranked within a preset range by the category confidence scores output by the target part classification model, are passed to the subsequent body surface region segmentation stage. This allows the subsequent body surface region segmentation stage to generate candidate body surface region masks and candidate segmentation boundaries according to the candidate part categories, and then pass these candidate body surface region masks and candidate segmentation boundaries to the subsequent anatomical reference point detection stage.

4. The method according to claim 1, characterized in that, S3 includes: selecting a segmentation template and boundary constraint parameters according to the category of the body part, segmenting the qualified target image into a body surface region to obtain a body surface region mask; performing morphological closing operation and connected component filtering on the body surface region mask to generate a segmentation boundary; and determining the segmentation boundary stability index based on the displacement variance of the same boundary points in the same acquisition coordinate space after registration of consecutive image frames of the same target body part acquired during the image acquisition stage.

5. The method according to claim 1, characterized in that, S4 includes: The reference point type is determined according to the location category, and anatomical reference points corresponding to the reference point type are detected within the area defined by the body surface region mask and segmentation boundary. The reference point types include auricular contour point, acromion point, suprasternal notch point, umbilicus point, cubital fossa point, wrist crease point, knee eye point, or ankle tip point. The key point confidence of each anatomical reference point is calculated based on the local response peak value, the neighborhood response difference, and the constraint result of the reference point falling into the body surface region mask.

6. The method according to claim 1, characterized in that, S5 include: Using the anatomical reference points in the set of reference points as coordinate reference points, the local coordinate axes, scale parameters, and rotation parameters are determined according to the category of the body part to form an individualized anatomical coordinate system; The acupoint names, acupoint ratio rules, and reference point topology relationships in the standard acupoint rule atlas are called up, and the acupoint ratio rules are converted into the initial acupoint coordinates in the individualized anatomical coordinate system.

7. The method according to claim 6, characterized in that, S6 include: An adaptive anatomical constraint map is constructed, wherein the nodes of the adaptive anatomical constraint map include the anatomical reference points and candidate acupoints, and the edges include anatomical adjacency relationships, proportional distance constraints, and directional constraints. An uncertainty vector is calculated based on the key point confidence level, the segmentation boundary stability index determined by the displacement variance of corresponding boundary points in the same acquisition coordinate space after registration of continuous image frames of the same target site acquired during the image acquisition stage, and the corresponding reference point displacement consistency index determined by the coordinate difference of corresponding reference points in the individualized anatomical coordinate system after registration of continuous image frames. The uncertainty vector is input into a calibration mapping table to obtain the coordinate confidence interval and region mask expansion coefficient for each candidate acupoint. The initial acupoint coordinates are corrected according to the coordinate confidence interval to obtain the acupoint location coordinates, and the neighborhood of the candidate acupoint is expanded according to the region mask expansion coefficient to obtain the acupoint region mask. The acupoint coordinates and the acupoint region mask are used to form an individualized deformation mapping and output the acupoint location result.

8. The method according to claim 7, characterized in that, The uncertainty vector is further corrected using a temporal correction index, which is determined as follows: consecutive image frames of the same target area acquired during the image acquisition phase are registered in chronological order, and the coordinate difference of the corresponding reference points in the individualized anatomical coordinate system is calculated; the temporal correction index is generated by normalizing and fusing the variance of the coordinate difference, the segmentation boundary stability index determined by the displacement variance of the corresponding boundary points in the consecutive frames, and the acquisition posture change determined by the acquisition posture parameter difference of the consecutive image frames.

9. The method according to claim 7, characterized in that, When the quality score generated by the sharpness index, exposure index, occlusion pixel ratio, target area occupancy ratio, and the angle between the human body contour principal axis and the acquisition coordinate axis does not meet the preset quality threshold, a re-acquisition prompt is output. The re-acquisition prompt includes the target part and the type of quality failure. When the category confidence does not meet the preset category threshold, the candidate site category and corresponding category confidence are passed to the subsequent body surface region segmentation stage and the subsequent anatomical reference point detection stage to generate candidate body surface region masks, candidate segmentation boundaries, and candidate reference point sets. The candidate site category is the site category whose category confidence ranking is within the preset range output by the target site classification model. The candidate acupoint location results are generated through the site adaptive anatomical constraint map and uncertainty calibration module and marked as pending verification. When the width of the coordinate confidence interval exceeds the preset width threshold, the corresponding acupoint location results are marked as pending verification, and the reference point missing type and corresponding candidate acupoint are output. When none of the above conditions are triggered, the acupoint location results for display or subsequent device control are output.

10. The method according to claim 9, characterized in that, The acupoint location results are output in a structured manner according to the target location identifier, acupoint identifier, point coordinates, region mask, location confidence, coordinate confidence interval, and status to be verified. The location confidence is calculated based on the key point confidence, the segmentation boundary stability index, the temporal correction index determined by the variance or normalized value of the coordinate difference of the corresponding reference points in the individualized anatomical coordinate system after registration of continuous image frames of the same target part acquired in the image acquisition stage, and the width of the coordinate confidence interval, and establishes an association record with the point coordinates and the region mask under the same acupoint identifier.

Images