Internal skeleton key point prediction method based on plantar surface data

Abstract

The invention relates to an internal skeleton key point prediction method based on plantar surface data, and belongs to the field of computer vision. The method comprises the following steps: acquiring an X-ray image of a foot of a patient, and calibrating key points of a foot skeleton in the image; acquiring a depth map of the foot of the same patient on the basis of a camera view angle when the X-ray image is acquired, and registering the edge contours of the X-ray image and the depth map, so that the calibrated key points in the X-ray image are associated with the features of the depth map; and inputting the depth map subjected to the registration operation into an improved YOLOv8-pose model to train the model, so that the model can predict foot skeleton key points according to the foot depth map. According to the method, the key points of the pelma skeleton can be extracted in a faster, lower-cost and harmless mode, the key points can be used for judging skeletal development diseases such as flatfoot and high arch foot, and the important clinical diagnosis value is achieved.

Description

Technical Field

[0001] The invention belongs to the field of computer vision, relates to the extraction of key points, and in particular to an internal skeleton key point prediction method based on plantar surface data. Background Art

[0002] With the development of society and the increasing attention paid to foot health, the study of foot structure and function has become particularly important. The bone structure of the foot is complex and sophisticated. The main components of the foot bone structure include metatarsals, tarsal bones, phalanges and corresponding joints. The complex structure of the foot bones enables it to support the body weight, provide balance, absorb impact, and complete various sports functions. The formation of the arch and the flexibility of the joints play an important role in maintaining the biomechanical balance and function of the foot. Therefore, obtaining and analyzing the health of the foot bone structure is of great significance to the overall body movement and quality of life.

[0003] Traditional medical imaging technologies such as X-ray and CT (Computed Tomography) scans provide reliable means of obtaining internal bone structures, but these methods are limited by radiation exposure, high costs and operational complexity. In this context, research on using external data to obtain internal bone images has attracted much attention. The non-invasive plantar surface data acquisition method is intended to provide patients with a more comfortable experience while avoiding the potential risks of radiation, and to provide a faster, safer and lower-cost new way to collect human-related physiological information data sets. At present, the main detection methods for plantar bone pathological judgments such as flat feet, high arches and other skeletal developmental diseases are:

[0004] 1. Use X-ray images of bones to determine the angle or distance relationship between key parts, and classify whether it is a certain type of bone disease according to certain standards. Taking flat feet as an example, the most traditional method is to judge by the arch of the foot. By calibrating points and connecting lines to form the angle of the arch of the foot, angle values ​​of 160° and above are considered flat feet. There are also related studies that combine computer vision methods to automatically judge flat feet in X-ray images to reduce the workload of manual key point annotation. These studies are often end-to-end detection tasks for a certain type of pathology, which lacks certain interpretability compared to the use of key points. X-ray itself carries damage to the individual being tested, and the operation is complex and costly. It is not suitable for scenarios such as rapid screening of a large number of patients.

[0005] 2. Using the pressure on the sole of the foot, plantar pressure analysis can only provide limited information, mainly focusing on the pressure distribution on the sole of the foot and the shift of the center of gravity. This method cannot directly observe the details of the bone structure or specific problems of bone development, so it may not provide sufficient diagnostic basis in some cases.

[0006] 3. Use the 3D scanning data of the plantar weight to establish a statistical model to calculate the key points of the internal skeleton. This type of model is usually a universal method based on the doctor's clinical experience, medical research or statistical analysis, but it is not absolutely applicable to all situations, so it may have limitations or inaccuracies in certain specific situations. Summary of the invention

[0007] In view of this, the purpose of the present invention is to provide a method for predicting internal bone key points based on plantar surface data, which, based on the three-dimensional scanning data under plantar weight-bearing conditions, combines a computer vision model to realize cross-modal prediction from plantar surface data to internal bone key point information.

[0008] In order to achieve the above object, the present invention provides the following technical solutions:

[0009] A method for predicting internal skeleton key points based on plantar surface data, the method comprising:

[0010] S1, obtain an X-ray image of the patient's foot and calibrate the key points of the foot bones in the image;

[0011] S2, based on the camera angle when acquiring the X-ray image, obtaining a depth map of the foot of the same patient, and registering the edge contours of the X-ray image and the depth map, so that the key points calibrated in the X-ray image are associated with the depth map features;

[0012] S3. Input the depth map after the registration operation into the improved YOLOv8-pose model to train the model, so that the model can predict the key points of the foot bones according to the foot depth map.

[0013] Furthermore, in step S2, registering the edge contours of the X-ray image and the depth map includes:

[0014] S21, converting the X-ray image and the depth map into grayscale images respectively and performing binarization, and performing contour extraction on the binarized images;

[0015] S22, sampling the extracted contour point set of the X-ray image and the contour point set of the depth map respectively to obtain a first point set corresponding to the depth map and a second point set corresponding to the X-ray image;

[0016] S23, obtaining an affine transformation matrix, and converting the first point set into a second point set after multiplying the first point set by the affine transformation matrix.

[0017] Furthermore, in step S22, sampling the extracted contour point set includes:

[0018] Randomly select a point in the contour point set, record it as point a, add point a to the sampling point set, and use point a as the starting point;

[0019] Get the point in the contour point set that has the farthest Euclidean distance from point a, record it as point b, and add point b to the sampling point set;

[0020] Take point b as the starting point, obtain the point with the farthest Euclidean distance from point b among the remaining points in the contour point set and add it to the sampling point set;

[0021] Through the above operations, samples are collected at the contour points in sequence until the preset sampling number is reached.

[0022] Furthermore, step S23 is specifically as follows: initialize the affine transformation matrix and multiply it with the first point set to transform the first point set into a third point set; use a Gaussian mixture model to describe the matching relationship between the second point set and the third point set to obtain a probability density function between the second point set and the third point set; maximize the probability density function to obtain the optimal affine transformation matrix.

[0023] Furthermore, in step S3, the improved YOLOv8-pose model includes four parts: Input, Backbone, Slim-neck and Head. The Input part is used to scale the input image and perform data preprocessing and enhancement operations; the Backbone part is used to extract image features; the Slim-neck part is used to integrate the top-down and bottom-up information flows in the network to enhance the detection performance; the Head part uses feature maps of different sizes to obtain the category and location information of objects of different sizes.

[0024] The Backbone part includes Conv module, C2f module, SPPF module and CBAM module. The Conv module is used to convert the output of the convolution layer into a feature map suitable for the target detection task. The C2f module and the Conv module are sequentially arranged in the Backbone part. The CBAM module enhances the effect of feature extraction by combining the channel attention and spatial attention mechanisms. The CBAM module is arranged in the last layer of the Backbone part. The SPPF module combines the pooling operations originally performed on multiple scales into one scale through point-by-point convolution and grouped convolution to reduce the computational complexity. At the same time, the SPPF module also uses point-by-point maximum pooling to further reduce the amount of calculation. The SPPF module is arranged in the previous layer of the CBAM module. It should be noted that the previous layer of the SPPF module should be the C2f module.

[0025] Furthermore, in the improved YOLOv8-pose model, Focal Loss is used as the loss function. By adjusting the weight of the loss function, the model can pay more attention to key points that are difficult to detect, so as to improve the detection accuracy of key points.

[0026] The beneficial effect of the present invention is that the present invention proposes a cross-modal prediction method that can predict the internal bone key point information from the plantar surface data based on the three-dimensional scanning data under the load-bearing condition of the plantar, combined with a computer vision model. The present invention can extract the key points of the plantar bones in a faster, lower-cost and non-destructive manner. These key points can be used to judge bone development diseases such as flat feet and high arches, and have important clinical diagnostic value.

[0027] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0029] Figure 1 It is a schematic diagram of the process of the method of the present invention;

[0030] Figure 2 A schematic diagram for marking key points of the foot skeleton;

[0031] Figure 3 is the sampling result of the contour point set;

[0032] Figure 4 This is a schematic diagram of the data preprocessing process;

[0033] Figure 5 This is a schematic diagram of the key point detection model structure;

[0034] Figure 6 It is a schematic diagram of the structure of the C2f module;

[0035] Figure 7 It is the schematic diagram of the SPPF module structure;

[0036] Figure 8 It is a schematic diagram of the CBAM module structure;

[0037] Fig. 9 Schematic diagram of the GSConv and VoV-GSCSP module structure. DETAILED DESCRIPTION

[0038] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0039] The present invention proposes a cross-modal prediction method from plantar surface data to internal skeletal key point information based on three-dimensional scanning data under plantar weight-bearing conditions, combined with a computer vision model. This method uses the patient's X-ray image to calibrate some key points, and associates these points with the foot depth map features at the same perspective of the three-dimensional scanning data obtained from the same patient's foot, and uses them to train a deep learning model, ultimately achieving direct prediction of the key point features of the patient's internal skeleton through the three-dimensional scanning data collected from the patient, without relying on X-ray image calibration of key points. This method can extract skeletal key points in a faster, lower-cost and non-destructive manner, and these key points can be used to diagnose skeletal developmental diseases such as flat feet and high arches, and has potential clinical diagnostic value.

[0040] like Figure 1 As shown, the method includes the following contents:

[0041] 1. Dataset processing

[0042] The training method of this method is to apply the key point label data of the plantar bone X-ray image (defined as mode B) to the corresponding plantar 3D scan data (defined as mode A) for cross-modal learning tasks, so as to predict the target data of mode B from mode A. This can avoid the inaccuracy caused by direct human annotation in mode A, and also make the model more interpretable. In the early data preprocessing stage, the following needs to be done: Figure 4 The operations shown are:

[0043] 1. Extract depth map

[0044] Determine the unified internal and external parameters of the camera so that the viewing angle of mode A can be consistent with the internal and external parameters of mode B during data collection. Under the determined camera parameters (including posture and viewing angle), halcon is used to extract the depth map of the plantar surface. Among them, in addition to the collection from different perspectives when collecting plantar X-ray images, there are also weight-bearing and non-weight-bearing categories. For example, the detection of diseases such as flat feet or high arches uses weight-bearing X-ray images of the side of the foot for diagnosis. Even if the patient cannot stand, he needs to maintain an approximate weight-bearing state for collection. In this embodiment, the data of mode B class all use X-ray images of the foot under weight-bearing state.

[0045] 2. Data annotation

[0046] Based on the advice of medical experts and combined with current research on the diagnosis of foot bone pathology, eight key points in the plantar bone X-ray images are established, such as Figure 2 As shown in the figure, the distance or angle between these points is often used as a direct basis for doctors to diagnose some foot diseases. Among them, the positions of the key points are: the lower edge of the posterior heel, the lowest point of the posterior heel bone, the lowest point of the fifth toe hook, the lowest point of the lower edge of the fifth metatarsal, the outer junction of the forefoot and the hindfoot, the lower edge of the connection between the navicular bone and the cuneiform bone, the outermost part of the forefoot, and the lowest point of the calcaneocuboid joint.

[0047] 3. Data Registration

[0048] Since the training data and the labeled data come from different modalities, the degree of alignment between the two modalities directly affects the final accuracy of the model prediction. In this embodiment, the common feature of the two types of modalities, the edge contour, is used for the registration operation, as described below:

[0049] First, convert the image into a grayscale image, find a suitable threshold to binarize the image, and use Canny edge detection to extract the contour of the binarized image. Since the overall background of the foot in each sample image is relatively clean, the group with the largest length among many contours can be directly extracted to obtain the outermost contour of the foot. In the actual results, although the two types of extracted contours have common features, there will still be some differences, outliers, noise, etc. when used directly for image registration. In order to reduce the impact of these outliers and noise and shorten the registration time, the idea of ​​Jitendra's sample (contour point sampling) is used to sample the contour point set. For the contour coordinate point set N = {I1, I2,…, I N}, and the Euclidean distance matrix is ​​obtained by the following formula:

[0050]

[0051] Where D ijrepresents the Euclidean distance from the i-th point to the j-th point, and ||·|| represents the L2 norm, which is the Euclidean distance. Each element in the matrix obtained by this formula represents the Euclidean distance between two points. Before actually calculating the Euclidean distance matrix, the matrix is ​​first shuffled to ensure the randomness of point sampling, and then a point I is randomly obtained from the point set. a As the starting point, according to formula (1), we obtain the point I with the longest Euclidean distance from the current point. b , point I b Included in the sampling point set, and then point I b As the starting point for the next sampling, the cycle continues until the set number of samples is reached. The contour point set obtained under this effect is more evenly distributed and the number of abnormal points is effectively suppressed. The contour point set sampling results are as follows: Figure 2 shown.

[0052] For the images of modality A and modality B, their contour point sets are obtained by the above method. The sampling results are as follows: Figure 3 shown.

[0053] Get the contour point set L of mode A and its corresponding mode B A With L B Then, with the point set L B Find an affine transformation T for the reference, so that L B =L A T. Gaussian Mixture Model (GMM) can be used to describe the point set L A With L B The matching relationship between:

[0054]

[0055] in, represents the position of point i in the A modal point set in the current transformation, σ 2 is the variance parameter of GMM, which is used to control the range of probability distribution. With L B The probability density function P between ij The optimal transformation T is found by maximizing the probability density matrix as follows:

[0056]

[0057] Among them, R is a rotation matrix and scaling transformation matrix, and t is a translation vector. This transformation matrix acts on the original image of modality A to complete the registration of the two images.

[0058] 2. Model Building

[0059] YOLOv8 is used as the model backbone and improved to be suitable for task scenarios. YOLOv8 is a relatively new detection algorithm in the YOLO series of algorithms and is suitable for a variety of visual tasks. In this embodiment, YOLOv8-pose, which performs well in the COCO dataset and is of moderate size, is selected as the basic model. The detection network of the model is divided into four parts: Input, Backbone, Neck, and Head.

[0060] Input: This part is responsible for scaling the input image to the size required for training and performing data preprocessing and augmentation operations. Preprocessing includes normalization and scaling of the image to ensure the consistency of the input size and pixel value range. For data augmentation, techniques such as scaling, hue adjustment, mosaic enhancement, and random transformations such as cropping, rotation, and flipping are used. In addition, an anchor-free mechanism is used to directly predict the center of the object, reducing the complexity and dependence on predefined anchor point size and shape.

[0061] Backbone: The backbone network is used for feature extraction and includes Conv, C2f, and SPPF modules. The new C2f structure uses residual features for learning and enriches the gradient flow information. SPPF is called spatial pyramid pooling and is used to convert feature maps of any size into feature vectors of fixed size.

[0062] Neck: The neck structure follows the Feature Pyramid Network (FPN) and Path Aggregation Network (PAN) to effectively integrate the top-down and bottom-up information flows in the network and enhance the detection performance.

[0063] Head: The head part uses feature maps of different sizes to obtain the category and location information of objects of different sizes. It applies the concept of distribution focal loss (DFL) to reduce the parameter size and computational complexity.

[0064] The YOLOv8 series models perform well in terms of detection accuracy and speed. This embodiment uses YOLOv8-pose as an improvement to further improve the detection performance. The initial model of YOLOv8-pose is a key point detection model for human posture recognition. Since there are certain differences in the features between foot key point detection and human posture key point detection, and foot key point detection requires higher accuracy, the original YOLOv8-pose model is improved to make it have clinical application potential, such as Figure 5 Shown is the improved YOLOv8-pose model.

[0065] The improvements are as follows: 1) As the key points of the foot are distributed sparsely and densely, the CBAM (Convolutional Block Attention Module) attention mechanism is introduced into the backbone feature extraction network to enhance the feature representation capability and focus more attention on each key point. The CBAM attention mechanism can fully consider global information, thereby improving the performance of the model. 2) Slim-neck is used to replace the original neck structure. By fusing features from different layers, the network's ability to detect objects of different scales can be effectively improved, thereby reducing the detection error caused by changes in object scale. Slim-neck can improve the accuracy of the model while reducing the computational complexity of the model. 3) In addition, the loss function is further modified, with Focal Loss as the loss function.

[0066] The specific improvements are described below:

[0067] 1. Backbone

[0068] (1) C2f module structure: The C2f module is designed with reference to the ideas of the C3 module and ELAN. It can obtain richer gradient flow information while ensuring the lightweight of the model. Its detailed structure is as follows: Figure 6 As shown in Figure 2. C2f represents the conversion process from the last convolutional layer to the output layer, which is responsible for converting the output of the convolutional layer into a feature map suitable for the target detection task. Figure 6 It can be seen that after the input (h*w*c_out) feature is split, the number of channels becomes half of the original (h*w*0.5c_out), half of the feature map is not processed, and the other half is passed to the BottleNeck module for feature fusion operation. The operation of C2f can be simply described as:

[0069] C2f=Concat(Upfeatures,Lowerfeatures) (4)

[0070] Among them, Upfeatures is a high-resolution feature map obtained by upsampling the feature map of the previous layer or layers, and Lowerfeatures is a low-resolution feature map obtained from the current layer. Concat(·) means concatenating these two feature maps according to the channel dimension.

[0071] (2) SPPF module structure: SPPF (Spatial Pyramid Pooling Fast) is an innovative pooling technology proposed in YOLOv8. SPP (Spatial Pyramid Pooling) is a pooling strategy introduced in YOLOv3, which can capture spatial information at different scales and enhance the robustness of the model. However, the computational complexity of SPP is large, which affects the inference speed of the model. To solve this problem, YOLOv8 proposed SPPF, which realizes the fast calculation of SPP through optimization algorithm. Its pooling formula is as follows:

[0072]

[0073] The SPPF structure is a continuous maximum pooling with a residual structure for three times. The convolution kernel is unified to 5*5. Finally, the results before and after pooling are concatenated. At the same time, in the maximum pooling operation, the feature map will become smaller after feature extraction, so the padding will be increased before feature extraction to ensure that the size of the feature map remains unchanged after each pooling. The core idea of ​​SPPF is to improve the reasoning speed of the model by reducing computational redundancy while maintaining the SPP function. Its network structure is as follows: Figure 7 shown.

[0074] Specifically, SPPF uses pointwise convolution and grouped convolution strategies to merge pooling operations originally performed at multiple scales into one scale, thereby reducing computational complexity. In addition, SPPF also uses pointwise max pooling to replace traditional spatial pyramid pooling, further reducing the amount of calculation. By introducing SPPF, YOLOv8 significantly improves the inference speed of the model while maintaining high accuracy. This makes YOLOv8 more competitive in practical applications, especially in scenarios that require real-time processing.

[0075] (3) CBAM module structure: This module is a module that combines spatial and channel attention mechanisms. The CBAM module can provide better feature representation, better positioning capability, and enhanced generalization capability for the foot key point extraction task, thereby improving the performance of the key point extraction model. The specific structure is as follows: Figure 8 shown.

[0076] Assume that an input feature map is Then channel attention The processing process is defined as:

[0077]

[0078] By using the multi-channel attention module, the process of compressing the spatial dimension of the input feature includes three steps: first, two different one-dimensional vector features are obtained by global maximum pooling and global average pooling based on width and height, namely and These vector features are then passed through a shared multi-layer perceptron MLP and their outputs are added in element order; finally, a sigmoid function is used to generate the final channel attention feature map. The function of the channel attention module in this part can be expressed as:

[0079]

[0080] Where σ is the activation function of CBAM.

[0081] Spatial Attention The process is defined as:

[0082]

[0083] Among them, F′ is the input feature.

[0084] The processing of the spatial attention module includes three steps: first, two 2D features are obtained using global maximum pooling and global average pooling based on the channel level, namely and Secondly, the two features are concatenated according to the channel dimension, and then merged into one channel through convolution operation (the convolution kernel size is 7×7); finally, the sigmoid function is used to generate the final spatial attention feature. The role of the spatial attention module in this part can be expressed as:

[0085]

[0086] Among them, f 7×7 Represents a convolution operation with a convolution kernel size of 7×7. For each input gait profile G i ,Channel attention and spatial attention focus on the feature content and feature position respectively. The two complement each other to a certain extent, improving the feature extraction effect of the network.

[0087] 2. Slim-neck

[0088] In Slim-neck, the traditional convolution Conv is replaced by the lightweight GSConv, which is a convolution operation based on global pooling and group sparse connection. By dividing the input channels into groups and performing independent convolution operations on each group, the expressiveness of the model is improved while reducing the amount of calculation. The VoV-GSCSP module is introduced based on GSConv. The VoV-GSCSP module structure is as follows: Fig. 9 shown.

[0089] 3. Loss Function

[0090] In foot keypoint detection, some keypoints may be more difficult to detect than other keypoints, resulting in keypoint imbalance in the dataset. At the same time, most of the background pixels of the foot keypoints are easy-to-classify negative samples, which may interfere with the training of the model.

[0091] Focal Loss can help alleviate this imbalance. By adjusting the weight of the loss function, the model pays more attention to key points that are difficult to detect, thereby improving the detection accuracy of these key points.

[0092] Focal Loss is based on the binary cross loss. By modifying the cross entropy loss, an adjustment factor is added. This factor reduces the loss value of samples that have been correctly classified, so that the training focus of the model is more on samples that are difficult to classify. This method is particularly helpful in improving the detection performance of small targets or targets that are easily overlooked in complex backgrounds. It can be expressed as follows:

[0093] FL(p t )=-α t (1-p t ) γ log(p t ) (10)

[0094] in:

[0095] p t is a function of p:

[0096]

[0097] The value of p ranges from 0 to 1, which is the probability that the model predicts that it belongs to the foreground.

[0098] γ is a parameter ranging from [0, 5], which can be used to control the imbalance between the number of simple and difficult samples; (1-p t ) γ It can reduce the loss contribution of easy-to-classify samples, thereby increasing the loss proportion of hard-to-classify samples. tTends to 1, which means that the sample is easy to distinguish. At this time, the modulation factor (1-p t ) γ is close to 0, indicating that the contribution to the loss is small, that is, the loss ratio of easily distinguishable samples is reduced. When a sample with a particularly small probability of being a foreground is misclassified as a positive sample, the modulation factor (1-p t ) γ It tends to 1, and the impact on losses is almost zero.

[0099] α t Used to adjust the ratio between positive and negative sample losses, the foreground category uses α t When , the corresponding background category uses 1-α t .

[0100] In short, Focal Loss allows the model to "focus" (or "focus") on learning samples that are more critical to improving overall performance.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A method for predicting internal skeleton key points based on plantar surface data, characterized in that: Obtain an X-ray image of the patient's foot and calibrate the key points of the foot bones in the image; Based on the camera angle when the X-ray image is acquired, a depth map of the foot of the same patient is acquired, and the edge contours of the X-ray image and the depth map are registered to associate the key points calibrated in the X-ray image with the depth map features; The depth map after the registration operation is input into the improved YOLOv8-pose model for training, so that the model can predict the key points of the foot skeleton based on the foot depth map.

2. The method according to claim 1, characterized in that: The registration of edge contours of an X-ray image and a depth map comprises the following steps: first, converting the X-ray image and the depth map into grayscale images and binarizing them, and extracting contours from the binarized images; then, sampling the extracted contour point set of the X-ray image and the contour point set of the depth map, respectively, to obtain a first point set corresponding to the depth map and a second point set corresponding to the X-ray image; and obtaining an affine transformation matrix, and converting the first point set into the second point set after multiplying the first point set by the affine transformation matrix.

3. The method according to claim 2, characterized in that: Sampling the extracted contour point set includes: randomly selecting a point in the contour point set, recorded as point a, adding point a to the sampling point set, and taking point a as the starting point; obtaining the point in the contour point set that has the farthest Euclidean distance from point a, recorded as point b, and adding point b to the sampling point set; taking point b as the starting point, obtaining the point in the remaining points of the contour point set that has the farthest Euclidean distance from point b and adding it to the sampling point set; through the above operations, sampling in the contour point set in sequence until a preset sampling number is reached.

4. The method according to claim 2, characterized in that: Obtaining the affine transformation matrix includes: initializing the affine transformation matrix and multiplying it with the first point set to transform the first point set into a third point set; using a Gaussian mixture model to describe the matching relationship between the second point set and the third point set to obtain a probability density function between the second point set and the third point set; and maximizing the probability density function to obtain the optimal affine transformation matrix.

5. The method according to claim 1, characterized in that: The improved YOLOv8-pose model includes four parts: Input, Backbone, Slim-neck and Head; the Input part is used to scale the input image and perform data preprocessing and enhancement operations; the Backbone part is used to extract image features; the Slim-neck part is used to integrate the top-down and bottom-up information flows in the network to enhance the detection performance; the Head part uses feature maps of different sizes to obtain the category and position information of objects of different sizes.

6. The method according to claim 5, characterized in that: The Backbone part includes a Conv module, a C2f module, an SPPF module and a CBAM module; the Conv module is used to convert the output of the convolution layer into a feature map suitable for the target detection task, and the C2f module and the Conv module are sequentially arranged in the Backbone part; the CBAM module enhances the effect of feature extraction by combining channel attention and spatial attention mechanisms, and the CBAM module is arranged in the last layer of the Backbone part; the SPPF module merges the pooling operations originally performed on multiple scales into one scale through point-by-point convolution and grouped convolution to reduce the computational complexity. At the same time, the SPPF module also uses point-by-point maximum pooling to further reduce the amount of calculation, and the SPPF module is arranged in the previous layer of the CBAM module.

7. The method according to claim 1 or 5, characterized in that: In the improved YOLOv8-pose model, Focal Loss is used as the loss function. By adjusting the weight of the loss function, the model can pay more attention to key points that are difficult to detect, so as to improve the detection accuracy of key points.

Images