A method for predicting key points of the internal skeleton based on plantar surface data

Abstract

This invention relates to a method for predicting key points of the internal skeleton based on plantar surface data, belonging to the field of computer vision. The method includes: acquiring an X-ray image of a patient's foot and labeling key points of the foot bones in the image; acquiring a depth map of the same patient's foot based on the camera viewpoint at the time of acquiring the X-ray image, and registering the edge contours of the X-ray image and the depth map to associate the labeled key points in the X-ray image with the features of the depth map; inputting the registered depth map into an improved YOLOv8-pose model to train the model, thereby enabling the model to predict key points of the foot bones based on the foot depth map. This invention can extract key points of the plantar bones in a faster, lower-cost, and non-invasive manner. These key points can be used to diagnose skeletal developmental disorders such as flat feet and high arches, and have significant clinical diagnostic value.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and relates to the extraction of key points, and in particular to a method for predicting internal skeletal key points based on plantar surface data. Background Technology

[0002] With societal development and increased public awareness of foot health, research on foot structure and function has become particularly important. The skeletal structure of the foot is complex and intricate, comprising the metatarsals, tarsal bones, phalanges, and corresponding joints. This complex structure enables the foot to support body weight, provide balance, absorb impact, and perform various motor functions. The formation of the arch and joint flexibility play crucial roles in maintaining the biomechanical balance and function of the foot. Therefore, acquiring and analyzing the health of the foot's skeletal structure is of great significance for overall physical activity and quality of life.

[0003] Traditional medical imaging techniques such as X-ray and CT (Computed Tomography) scans provide reliable means of acquiring internal skeletal structures, but these methods are limited by radiation exposure, high costs, and operational complexity. Against this backdrop, research utilizing external data to acquire images of the internal skeleton has attracted significant attention. Non-invasive methods for acquiring plantar surface data aim to provide a more comfortable experience for patients while avoiding the potential risks of radiation, offering a faster, safer, and lower-cost new approach for collecting datasets of relevant human physiological information. Currently, the main methods for pathological diagnosis of plantar bones, such as flat feet and high arches, and other skeletal developmental disorders, include:

[0004] 1. X-ray images of the skeleton are used to determine the angular or distance relationships between key areas, and to classify them according to certain criteria to determine whether they belong to a certain type of skeletal disease. Taking flat feet as an example, the most traditional method is to judge by the arch of the foot. By marking points and connecting them to form the angle of the arch, an angle of 160° or higher is considered flat feet. Currently, there are also studies that combine computer vision methods to automatically judge flat feet from X-ray images to reduce the workload of manually annotating key points. These studies are often for end-to-end detection tasks for a certain type of pathology and lack interpretability compared to using key points. X-rays themselves are invasive to the individual being examined, and are complex and costly to operate, making them unsuitable for scenarios such as rapid screening of large numbers of patients.

[0005] 2. Using plantar pressure analysis provides limited information, primarily focusing on pressure distribution and weight shift. This method cannot directly observe details of skeletal structure or specific issues in skeletal development, and therefore may not provide sufficient diagnostic evidence in some cases.

[0006] 3. By using 3D scan data of the foot under weight-bearing conditions, a statistical model is established to calculate the key points of the internal skeleton. This type of model is usually a universal method derived from the doctor's clinical experience, medical research, or statistical analysis, but it is not absolutely applicable to all situations, and therefore may have limitations or inaccuracies in certain specific cases. Summary of the Invention

[0007] In view of this, the purpose of this invention is to provide a method for predicting key points of the internal skeleton based on plantar surface data. Based on three-dimensional scanning data under plantar weight-bearing conditions, the method combines computer vision models to achieve cross-modal prediction from plantar surface data to key point information of the internal skeleton.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] A method for predicting internal skeletal key points based on plantar surface data, the method comprising:

[0010] S1. Obtain X-ray images of the patient's feet and mark the key points of the foot bones in the images;

[0011] S2. Based on the camera perspective when acquiring X-ray images, acquire depth maps of the same patient's feet, and register the edge contours of the X-ray images and depth maps to associate the key points marked in the X-ray images with the features of the depth maps.

[0012] S3. Input the depth map after registration into the improved YOLOv8-pose model to train the model, so that the model can predict the key points of the foot skeleton based on the foot depth map.

[0013] Furthermore, in step S2, the registration of the edge contours of the X-ray image and the depth map includes:

[0014] S21. Convert the X-ray image and depth map into grayscale images and binarize them respectively. Extract the contours from the binarized images.

[0015] S22. Sample the contour point set of the extracted X-ray image and the contour point set of the depth map respectively to obtain the first point set corresponding to the depth map and the second point set corresponding to the X-ray image.

[0016] S23. Obtain an affine transformation matrix such that the first point set multiplied by the affine transformation matrix is ​​transformed into the second point set.

[0017] Furthermore, in step S22, sampling the extracted contour point set includes:

[0018] Randomly select a point from the set of contour points, denoted as point a, add point a to the set of sampling points, and take point a as the starting point;

[0019] Find the point in the contour point set that has the farthest Euclidean distance from point a, denoted as point b, and add point b to the sampling point set;

[0020] Using point b as the starting point, obtain the point with the farthest Euclidean distance from point b among the remaining points in the contour point set and add it to the sampling point set;

[0021] The above operations are performed sequentially to sample the contour points until the preset number of samples is reached.

[0022] Further, step S23 specifically involves: initializing the affine transformation matrix and multiplying it with the first point set to transform the first point set into the third point set; using a Gaussian mixture model to describe the matching relationship between the second and third point sets to obtain the probability density function between the second and third point sets; and maximizing this probability density function to obtain the optimal affine transformation matrix.

[0023] Furthermore, in step S3, the improved YOLOv8-pose model comprises four parts: Input, Backbone, Slim-neck, and Head. The Input part is used to scale the input image and perform data preprocessing and enhancement operations; the Backbone part is used to extract image features; the Slim-neck part is used to integrate the top-down and bottom-up information flows in the network to enhance detection performance; and the Head part utilizes feature maps of different sizes to obtain the category and location information of objects of different sizes.

[0024] The Backbone section includes the Conv module, C2f module, SPPF module, and CBAM module. The Conv module converts the output of the convolutional layers into feature maps suitable for object detection tasks. The C2f module is sequentially placed in the Backbone section along with the Conv module. The CBAM module enhances feature extraction by combining channel attention and spatial attention mechanisms, and is placed in the last layer of the Backbone section. The SPPF module reduces computational complexity by merging pooling operations originally performed at multiple scales into a single scale through pointwise convolution and grouped convolution. Furthermore, the SPPF module employs pointwise max pooling to further reduce computation. The SPPF module is placed in the layer preceding the CBAM module; it's important to note that the layer preceding the SPPF module should be the C2f module.

[0025] Furthermore, in the improved YOLOv8-pose model, Focal Loss is used as the loss function. By adjusting the weights of this loss function, the model can pay more attention to key points that are difficult to detect, thereby improving the detection accuracy of key points.

[0026] The beneficial effects of this invention are as follows: This invention proposes a cross-modal prediction method that, based on three-dimensional scanning data under foot weight-bearing conditions, combines computer vision models to predict key points of internal bones from foot surface data. This invention can extract key points of the foot bones in a faster, lower-cost, and non-invasive manner. These key points can be used to diagnose skeletal developmental disorders such as flat feet and high arches, and have significant clinical diagnostic value.

[0027] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0028] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0029] Figure 1 This is a flowchart illustrating the method described in this invention;

[0030] Figure 2 A diagram illustrating the key points of the foot bones;

[0031] Figure 3 The sampling results are for the contour point set;

[0032] Figure 4 This is a schematic diagram of the data preprocessing workflow;

[0033] Figure 5 This is a schematic diagram of the key point detection model structure;

[0034] Figure 6 This is a schematic diagram of the C2f module.

[0035] Figure 7 This is a schematic diagram of the SPPF module structure;

[0036] Figure 8 This is a schematic diagram of the CBAM module structure;

[0037] Figure 9 This is a schematic diagram of the GSConv and VoV-GSCSP module structures. Detailed Implementation

[0038] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0039] This invention proposes a cross-modal prediction method for extracting key skeletal information from foot surface data using 3D foot scans under weight-bearing conditions, combined with a computer vision model. This method uses X-ray images of the patient to identify some key points and correlates these points with the foot depth map features from the same perspective in 3D scans of the same patient's foot. These correlations are then used to train a deep learning model, ultimately enabling the direct prediction of key skeletal features from the patient's 3D scan data, eliminating the need for X-ray image-based key point identification. This method allows for faster, lower-cost, and non-invasive extraction of skeletal key points, which can be used to diagnose skeletal developmental disorders such as flat feet and high arches, demonstrating potential clinical diagnostic value.

[0040] like Figure 1 As shown, the method includes the following:

[0041] I. Dataset Processing

[0042] This method trains the model by applying keypoint labels from X-ray images of the plantar skeleton (modal B) to corresponding 3D plantar scan data (modal A) for cross-modal learning, thereby enabling the prediction of target data in modal B from modal A. This avoids the inaccuracies caused by direct manual annotation in modal A and also makes the model more interpretable. The initial data preprocessing stage requires... Figure 4 The operation shown:

[0043] 1. Extract depth map

[0044] Unified intrinsic and extrinsic parameters for the camera were determined to ensure consistency between the endoscopic and extrinsic parameters during modality A data acquisition and those during modality B data acquisition. Under the determined camera parameters (including attitude and endoscopic angle), a depth map of the foot surface was extracted using Halcon. In acquiring foot X-ray images, in addition to acquisition from different endoscopic angles, there are also weight-bearing and non-weight-bearing categories. For example, for the detection of conditions such as flat feet or high arches, weight-bearing X-ray images of the lateral aspect of the foot are used for diagnosis, requiring acquisition while maintaining a near-weight-bearing state even if the patient cannot stand. In this embodiment, all data for modality B are foot X-ray images under weight-bearing conditions.

[0045] 2. Data labeling

[0046] Based on the advice of medical experts and in conjunction with current research on foot bone pathology diagnosis, eight key points that are relatively important in X-ray images of the foot bones have been identified, such as... Figure 2 As shown, the distances or angles between these points often serve as a direct basis for doctors to diagnose some foot diseases. The key points are located at: the lower edge of the hindfoot heel, the lowest point of the calcaneus, the lowest point of the fifth toe hook, the lowest point of the lower edge of the fifth metatarsal bone, the junction of the outer periphery of the forefoot and hindfoot, the lower edge of the connection between the navicular and cuneiform bones, the outermost periphery of the forefoot, and the lowest point of the calcaneocuboid joint.

[0047] 3. Data registration

[0048] Since the training data and labeled data come from different modalities, the alignment degree between the two modalities directly affects the final accuracy of the model's prediction. In this embodiment, the registration operation is performed using a common feature of the two modalities—edge contours—as described below:

[0049] First, the image is converted to grayscale. A suitable threshold is then used for binarization, and Canny edge detection is employed to extract contours from the binarized image. Since the background of the foot is relatively clean in each sample image, the outermost contour of the foot can be directly extracted from the longest set of contours. In actual results, although the two types of extracted contours share common features, direct use for image registration still results in some differences, outliers, and noise. To reduce the impact of these outliers and noise while shortening the registration time, Jitendra's sampling method is used to sample the contour point set. For the contour coordinate point set N = {I1, I2, ..., I...} N The Euclidean distance matrix of} can be obtained using the following formula:

[0050]

[0051] Where D ijLet ||i|| represent the Euclidean distance from point i to point j, and ||·|| represent the L2 norm, which is the Euclidean distance. Each element in the matrix obtained by this formula represents the Euclidean distance between two points. Before actually calculating the Euclidean distance matrix, the matrix is ​​shuffled to ensure the randomness of point sampling, and then a point I is randomly selected from the point set. a Starting from the current point and according to equation (1), the point I with the greatest Euclidean distance from the current point is obtained. b Point I b The points included in the sampling set are then used as point I. b This serves as the starting point for the next sampling, and the process is repeated until the set number of samples is reached. This method results in a more uniform distribution of the contour point set while effectively suppressing outliers. The contour point set sampling results are as follows: Figure 2 As shown.

[0052] For images of mode A and mode B, their contour point sets are obtained using the method described above, and the sampling results are as follows: Figure 3 As shown.

[0053] Obtain the contour point set L of mode A and its corresponding mode B. A With L B Then, using the point set L B Find an affine transformation T for the reference, such that L is as close as possible to L. B =L A T. A Gaussian Mixture Model (GMM) can be used to describe the point set L. A With L B Matching relationships between them:

[0054]

[0055] in, σ represents the position of point i in the current transformation within the set of points in mode A. 2 This is the variance parameter of the GMM, used to control the range of the probability distribution. From this, we can obtain the transformed... With L B The probability density function P between ij The optimal transformation T is found by maximizing this probability density matrix, as shown below:

[0056]

[0057] Where R is a rotation matrix and a scaling transformation matrix, and t is a translation vector. This transformation matrix is ​​applied to the original image of mode A to complete the registration of the two images.

[0058] II. Model Building

[0059] YOLOv8 is used as the backbone of the model and improved to suit the task scenario. YOLOv8 is a relatively new detection algorithm in the YOLO series, suitable for various visual tasks. In this embodiment, YOLOv8-pose, which performs well and has a suitable size in the COCO dataset, is selected as the base model. The detection network of this model is divided into four parts: Input, Backbone, Neck, and Head.

[0060] Input: This part is responsible for scaling the input image to the size required for training and performing data preprocessing and augmentation operations. Preprocessing includes image normalization and scaling to ensure consistency in input size and pixel value range. For data augmentation, techniques such as scaling, tone adjustment, mosaic enhancement, and random transformations such as cropping, rotation, and flipping are employed. Furthermore, an anchorless mechanism is used to directly predict the center of objects, reducing complexity and dependence on predefined anchor point size and shape.

[0061] Backbone: The backbone network is used for feature extraction and includes Conv, C2f, and SPPF modules. The novel C2f structure applies residual features for learning, enriching gradient flow information. SPPF, or Spatial Pyramid Pooling, is used to transform feature maps of arbitrary size into feature vectors of fixed size.

[0062] Neck: The neck structure follows the Feature Pyramid Network (FPN) and Path Aggregation Network (PAN) to effectively integrate the top-down and bottom-up information flow in the network, enhancing detection performance.

[0063] Head: The head portion utilizes feature maps of different sizes to obtain category and location information for objects of different sizes. It applies the concept of Distributed Focal Loss (DFL), reducing parameter size and computational complexity.

[0064] The YOLOv8 series models perform well in terms of detection accuracy and speed. This embodiment uses YOLOv8-pose as an improved foundation to further enhance detection performance. The initial YOLOv8-pose model is a keypoint detection model for human pose recognition. Since foot keypoint detection differs somewhat in features from human pose keypoint detection, and foot keypoint detection requires higher accuracy, the original YOLOv8-pose model has been improved to enhance its clinical application potential. Figure 5 The image shows the improved YOLOv8-pose model.

[0065] The improvements are as follows: 1) Given the varying density of key points in the foot, a Convolutional Block Attention Module (CBAM) is introduced into the backbone feature extraction network to enhance feature representation capabilities and focus more attention on each key point. CBAM fully considers global information, thereby improving model performance. 2) A Slim-neck is used instead of the original neck structure. By fusing features from different layers, the network's ability to detect objects at different scales is effectively improved, reducing detection errors caused by changes in object scale. The Slim-neck improves model accuracy while reducing computational complexity. 3) Furthermore, the loss function is modified to use Focal Loss.

[0066] The specific improvements are explained below:

[0067] 1. Backbone section

[0068] (1) C2f module structure: The C2f module is designed with reference to the C3 module and the ELAN concept. It can obtain richer gradient flow information while ensuring the model is lightweight. Its detailed structure is as follows: Figure 6 As shown, C2f represents the transformation process from the last convolutional layer to the output layer, responsible for converting the output of the convolutional layer into a feature map suitable for the object detection task. Figure 6 It can be seen that after the input (h*w*c_out) features are split, the number of channels becomes half of the original (h*w*0.5c_out). One half of the feature map is left unprocessed, while the other half is fed into the BottleNeck module for feature fusion. The C2f operation can be simply described as follows:

[0069] C2f=Concat(Upfeatures,Lowerfeatures) (4)

[0070] Here, Upfeatures is a high-resolution feature map obtained by upsampling the feature maps of the previous layer or several layers, and Lowerfeatures is a low-resolution feature map obtained from the current layer. Concat(·) means concatenating these two feature maps according to the channel dimension.

[0071] (2) SPPF Module Structure: SPPF (Spatial Pyramid Pooling Fast) is an innovative pooling technique proposed in YOLOv8. SPP (Spatial Pyramid Pooling) is a pooling strategy introduced in YOLOv3, which can capture spatial information at different scales and enhance the robustness of the model. However, SPP has a large computational cost, affecting the inference speed of the model. To solve this problem, YOLOv8 proposed SPPF, which achieves fast computation of SPP through algorithm optimization. Its pooling formula is as follows:

[0072]

[0073] The SPPF structure involves three consecutive max pooling operations with residual structures. The convolutional kernels are all 5x5. Finally, the results before and after each pooling operation are concatenated. Since the feature map shrinks after feature extraction during max pooling, padding is increased before feature extraction to ensure the feature map size remains constant after each pooling operation. The core idea of ​​SPPF is to improve the model's inference speed by reducing computational redundancy while maintaining the functionality of SPP. Its network structure is as follows: Figure 7 As shown.

[0074] Specifically, SPPF employs pointwise convolution and grouped convolution strategies, merging pooling operations that were originally performed at multiple scales into a single scale, thereby reducing computational complexity. Furthermore, SPPF uses pointwise max pooling instead of traditional spatial pyramid pooling, further reducing computational load. By introducing SPPF, YOLOv8 significantly improves inference speed while maintaining high accuracy. This makes YOLOv8 more competitive in practical applications, especially in scenarios requiring real-time processing.

[0075] (3) CBAM Module Structure: This module is an attention mechanism module that combines spatial and channel-based approaches. The CBAM module can bring better feature representation, better localization ability, and enhanced generalization ability to the foot keypoint extraction task, thereby improving the performance of the keypoint extraction model. The specific structure is as follows: Figure 8 As shown.

[0076] Suppose an input feature map is So channel attention The processing procedure is defined as follows:

[0077]

[0078] The process of compressing the spatial dimension of input features by using a multi-channel attention module involves three steps: First, based on width and height, two different one-dimensional vector features are obtained through global max pooling and global average pooling, respectively. and Then, these vector features are passed through a shared multilayer perceptron (MLP), and their outputs are added element-wise; finally, the sigmoid function is used to generate the final channel attention feature map. The function of the channel attention module in this part can be represented as:

[0079]

[0080] Where σ is the activation function of CBAM.

[0081] Spatial attention The process is defined as:

[0082]

[0083] Where F′ represents the input feature.

[0084] The spatial attention module's processing involves three steps: First, it uses channel-level global max pooling and global average pooling to obtain two 2D features, namely... and Secondly, the two obtained features are concatenated according to the channel dimension, and then merged into one channel through a convolution operation (with a kernel size of 7×7); finally, the sigmoid function is used to generate the final spatial attention feature. The role of the spatial attention module in this part can be represented as follows:

[0085]

[0086] Among them, f 7×7 This represents a convolution operation with a kernel size of 7×7. For each input gait contour G... i Channel attention and spatial attention focus on the feature content and feature location respectively. To a certain extent, they complement each other and improve the feature extraction effect of the network.

[0087] 2. Slim-neck section

[0088] In Slim-neck, the traditional convolutional Conv is replaced by the lightweight GSConv. GSConv is a convolutional operation based on global pooling and group sparse connections. By dividing the input channels into groups and performing independent convolution operations on each group, it improves the model's expressiveness while reducing computational cost. The VoV-GSCSP module is introduced based on GSConv. The structure of the VoV-GSCSP module is as follows: Figure 9 As shown.

[0089] 3. Loss Function

[0090] In foot keypoint detection, some keypoints may be more difficult to detect than others, leading to keypoint imbalance in the dataset. Additionally, most background pixels of foot keypoints are easily classified negative samples, which may interfere with model training.

[0091] Focal Loss can help alleviate this imbalance by adjusting the weights of the loss function, making the model pay more attention to key points that are difficult to detect, thereby improving the detection accuracy of these key points.

[0092] Focal Loss, based on binary cross-classification loss, modifies the cross-entropy loss by adding an adjustment factor. This factor reduces the loss value of correctly classified samples, allowing the model to focus its training on harder-to-classify samples. This approach is particularly effective in improving the detection performance of small targets or targets easily overlooked in complex backgrounds. It can be represented as follows:

[0093] FL(p t )=-α t (1-p t ) γ log(p t (10)

[0094] in:

[0095] p t It is a function of p:

[0096]

[0097] The value of p ranges from 0 to 1, and it represents the probability that the model predicts the subject to be the foreground.

[0098] γ is a parameter ranging from [0, 5], which can be used to control the imbalance between the number of simple and difficult-to-distinguish samples; (1-p t ) γ This can reduce the loss contribution of easily separable samples, thereby increasing the loss proportion of difficult-to-separate samples. For example, when p tIf the modulation factor approaches 1, it indicates that the sample is easily distinguishable. In this case, the modulation factor (1-p) t ) γ The value tending towards 0 indicates a smaller contribution to the loss, meaning it reduces the proportion of easily distinguishable samples in the loss calculation. However, when a sample with a particularly low probability of being a foreground is misclassified as a positive sample, the modulation factor (1-p) increases. t ) γ The value tends towards 1, meaning the impact on the loss is almost negligible.

[0099] α t To adjust the ratio between positive and negative sample losses, the foreground category uses α. t When the corresponding background category is 1-α, t .

[0100] In short, Focal Loss allows the model to "focus" (or "concentrate") on learning those samples that are more critical to improving overall performance.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for predicting key points of the internal skeleton based on plantar surface data, characterized in that: Obtain X-ray images of the patient's foot and identify key points of the foot bones in the images; Based on the camera perspective when acquiring X-ray images, a depth map of the same patient's foot is acquired, and the edge contours of the X-ray image and the depth map are registered so that the key points marked in the X-ray image are associated with the features of the depth map. The depth map after registration is input into the improved YOLOv8-pose model to train the model, so that the model can predict the key points of the foot skeleton based on the foot depth map. The registration of edge contours between the X-ray image and the depth map includes: first, converting the X-ray image and the depth map into grayscale images and binarizing them; then, extracting contours from the binarized images; next, sampling the extracted contour point sets of the X-ray image and the depth map to obtain a first point set corresponding to the depth map and a second point set corresponding to the X-ray image; and finally, obtaining an affine transformation matrix, multiplying the first point set by the affine transformation matrix to convert it into the second point set. The improved YOLOv8-pose model comprises four parts: Input, Backbone, Slim-neck, and Head. The Input part scales the input image and performs data preprocessing and enhancement operations. The Backbone part extracts image features. The Slim-neck part integrates top-down and bottom-up information flows in the network to enhance detection performance. The Head part utilizes feature maps of different sizes to obtain the category and location information of objects of different sizes. The Backbone section includes a Conv module, a C2f module, an SPPF module, and a CBAM module. The Conv module converts the output of the convolutional layer into a feature map suitable for object detection tasks. The C2f module and the Conv module are sequentially placed in the Backbone section. The CBAM module enhances the feature extraction effect by combining channel attention and spatial attention mechanisms. The CBAM module is placed in the last layer of the Backbone section. The SPPF module combines pooling operations that were originally performed at multiple scales into one scale by using pointwise convolution and grouped convolution to reduce computational complexity. At the same time, the SPPF module also uses pointwise max pooling to further reduce the amount of computation. The SPPF module is placed in the layer before the CBAM module.

2. The method according to claim 1, characterized in that: Sampling the extracted contour point set includes: randomly selecting a point from the contour point set, denoted as point A. a , will point a Add to the sampling point set, and put the points a Starting from this point; obtain the contour point set and points. a The point with the greatest Euclidean distance is denoted as point A. b , will point b Add to the sampling point set; put the points b Starting from this point, obtain the remaining points in the contour point set and the points... b The point with the farthest Euclidean distance is added to the sampling point set; the above operations are performed sequentially to sample the contour point set until the preset number of samples is reached.

3. The method according to claim 1, characterized in that: Obtaining the affine transformation matrix includes: initializing the affine transformation matrix and multiplying it with the first point set to transform the first point set into a third point set; using a Gaussian mixture model to describe the matching relationship between the second point set and the third point set to obtain the probability density function between the second point set and the third point set; and maximizing the probability density function to obtain the optimal affine transformation matrix.

4. The method according to claim 1, characterized in that: In the improved YOLOv8-pose model, Focal Loss is used as the loss function. By adjusting the weights of this loss function, the model can pay more attention to key points that are difficult to detect, thereby improving the detection accuracy of key points.

Images