U-Net-based skeleton segmentation image generation method and acupoint positioning method
Through the U-Net-based bone segmentation image generation method and Heisen matrix optimization algorithm, the problem of insufficient acupuncture positioning accuracy in existing massage chairs is solved, and high-accurate acupuncture positioning in various environments is achieved, and the cost of system development is reduced.
Patent Information
- Application Number
- CN202411560955.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-04
- Publication Date
- 2025-02-25
AI Technical Summary
The existing massage chairs lack accuracy in acupoint positioning, which is affected by factors such as lighting, shooting angle and human posture, and the cost of data collection and training is high.
U-Net-based bone segmentation image generation method is used to automatically identify human skeletal images through the bone segmentation model, and the final position of acupuncture points is determined using Heisen matrix and optimization algorithm.
Accurate positioning of acupuncture points in various environments is achieved, which reduces interference from human factors, improves positioning accuracy, and reduces the cost of data collection and training.
Smart Images
Figure CN119515888A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of acupoint positioning, and more specifically, to a skeleton segmentation image generation method and an acupoint positioning method based on U-Net. Background Art
[0002] Nowadays, a variety of massage chairs with various functions have emerged on the market, including heated massage chairs, air compression massage chairs, customized massage chairs, microcurrent massage chairs, etc. However, most massage chairs lack universal applicability and cannot automatically adjust according to the needs of different groups of people. In addition, some massage chairs use simple sensors to detect the location and strength of human acupuncture points, or use cameras to capture back images for feature extraction, but the accuracy of these methods is limited.
[0003] Since the location and morphology of acupoints may vary due to a variety of factors, direct recognition methods may face higher accuracy challenges. The location and morphology of acupoints may vary due to a variety of factors such as individual differences, body shape, muscle condition, etc., which may be difficult to accurately capture directly through image recognition. Existing technologies have high requirements for image quality, and accurate identification of acupoints requires high-quality, clear images. Factors such as lighting conditions, shooting angles, and human posture may affect image quality and thus the accuracy of acupoint recognition. Training large-scale data may require a lot of time and computing resources, and usually a large amount of data is required to achieve good performance. This means that it takes a lot of time and effort to collect, annotate, and train data, increasing the cost and complexity of system development. Summary of the invention
[0004] In order to overcome the defect of inaccurate acupoint positioning in the prior art, the present invention provides a skeleton segmentation image generation method and an acupoint positioning method based on U-Net.
[0005] In order to solve the above technical problems, the technical solution of the present invention is as follows:
[0006] The present invention provides a method for generating a skeleton segmentation image based on U-Net, comprising:
[0007] Acquire human skeleton images;
[0008] Preprocessing the human skeleton image to obtain a preprocessed human skeleton image;
[0009] The constructed bone segmentation model is trained based on the preprocessed human bone image to obtain a trained bone segmentation model;
[0010] The human skeleton image to be segmented is input into the trained skeleton segmentation model to obtain the skeleton segmentation image.
[0011] Preferably, the constructed bone segmentation model is trained based on the preprocessed human bone image to obtain a trained bone segmentation model, including:
[0012] Constructing a skeleton segmentation model, the skeleton segmentation model comprising an input layer, an encoder, a deepest layer, a decoder and an output layer connected in sequence;
[0013] The preprocessed human skeleton image is input into the constructed skeleton segmentation model for feature extraction and prediction to obtain a predicted skeleton segmentation image;
[0014] Set the cross entropy loss function, calculate the cross entropy loss function value based on the predicted bone segmentation image and the true label, and adjust the parameters of the constructed bone segmentation model;
[0015] When the cross entropy loss function value reaches a preset value or reaches a preset number of training times, a trained bone segmentation model is obtained.
[0016] Preferably, the encoder comprises a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a fifth convolutional layer, a sixth convolutional layer, a first maximum pooling layer, a second maximum pooling layer and a third maximum pooling layer;
[0017] The output end of the first convolutional layer is connected to the input end of the second convolutional layer; the output end of the second convolutional layer is connected to the input end of the first maximum pooling layer; the output end of the maximum pooling layer is connected to the input end of the third convolutional layer; the output end of the third convolutional layer is connected to the input end of the fourth convolutional layer; the output end of the fourth convolutional layer is connected to the input end of the second maximum pooling layer; the output end of the second maximum pooling layer is connected to the input end of the fifth convolutional layer; the output end of the fifth convolutional layer is connected to the input end of the sixth convolutional layer; the output end of the sixth convolutional layer is connected to the input end of the third maximum pooling layer;
[0018] The output end of the third maximum pooling layer is connected to the input end of the deepest layer;
[0019] Preferably, the deepest layer includes a seventh convolutional layer and an eighth convolutional layer connected in sequence; the output end of the third maximum pooling layer is connected to the input end of the seventh convolutional layer; and the output end of the eighth convolutional layer is connected to the input end of the decoder.
[0020] Preferably, the decoder comprises a first upsampling layer, a second upsampling layer, a first splicing layer, a second splicing layer, a ninth convolutional layer, a tenth convolutional layer, an eleventh convolutional layer, and a twelfth convolutional layer;
[0021] The output end of the eighth convolutional layer is connected to the input end of the first upsampling layer; the output end of the first upsampling layer is connected to the input end of the first splicing layer; the output end of the first splicing layer is connected to the input end of the ninth convolutional layer; the output end of the ninth convolutional layer is connected to the input end of the tenth convolutional layer; the output end of the tenth convolutional layer is connected to the input end of the second upsampling layer; the output end of the second upsampling layer is connected to the input end of the second splicing layer; the output end of the second splicing layer is connected to the input end of the eleventh convolutional layer; the output end of the eleventh convolutional layer is connected to the input end of the twelfth convolutional layer;
[0022] The output of the twelfth convolutional layer is connected to the input of the output layer.
[0023] Preferably, the output layer includes a thirteenth convolutional layer and a first activation layer connected in sequence; the output end of the twelfth convolutional layer is connected to the input end of the thirteenth convolutional layer.
[0024] Preferably, the formula of the cross entropy loss function is:
[0025]
[0026] Among them, y i is the true label; p i is the predicted bone segmentation image; M is the total number of label categories.
[0027] The present invention also provides an acupoint locating method, comprising:
[0028] Obtaining a skeleton segmentation image, wherein the skeleton segmentation image is generated using the above-mentioned skeleton segmentation image generation method based on U-Net;
[0029] The bone segmentation image is binarized and then calculated to obtain the Hessian matrix corresponding to each pixel in the bone segmentation image;
[0030] Calculate the determinant of the Hessian matrix of each pixel at different scales, and select the local extreme points in the determinant as key feature points;
[0031] Each key feature point is optimized using an optimization algorithm to determine the final position of all acupoints.
[0032] Preferably, an optimization algorithm is used to optimize the key feature points to determine the final acupoint position, including:
[0033] Construct the target point set and source point set according to each key feature point and real feature point, and calculate the corresponding nearest neighbor points according to the target point set and source point set;
[0034] The objective function is set according to each target point and its corresponding nearest neighbor point until the objective function value converges or reaches a preset number of iterations to determine the final positions of all acupoints.
[0035] Preferably, the formula of the objective function is:
[0036]
[0037] Among them, R is the rotation matrix; t is the translation variable; is the target point; is the nearest neighbor point.
[0038] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:
[0039] The present invention proposes a skeleton segmentation image generation method and an acupoint positioning method based on U-Net. The skeleton segmentation model can automatically and accurately identify the human skeleton image. It is not affected by factors such as lighting conditions, shooting angles and human posture. Acupoint positioning can be performed in various environments, reducing the interference of human factors and improving the accuracy of positioning. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Figure 1 This is a flowchart of the U-Net-based skeleton segmentation image generation method described in Example 1;
[0041] Figure 2 is a schematic diagram of the structure of the skeleton segmentation model described in Example 2;
[0042] Figure 3 This is a flow chart of the acupoint locating method described in Example 3. DETAILED DESCRIPTION
[0043] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;
[0044] In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;
[0045] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.
[0046] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0047] Example 1
[0048] This embodiment provides a method for generating a skeleton segmentation image based on U-Net. Figure 1 As shown, including:
[0049] Acquire human skeleton images;
[0050] Preprocessing the human skeleton image to obtain a preprocessed human skeleton image;
[0051] The constructed bone segmentation model is trained based on the preprocessed human bone image to obtain a trained bone segmentation model;
[0052] The human skeleton image to be segmented is input into the trained skeleton segmentation model to obtain the skeleton segmentation image.
[0053] Example 2
[0054] This embodiment provides a method for generating a skeleton segmentation image based on U-Net, comprising:
[0055] Acquire human skeleton images;
[0056] Preprocessing the human skeleton image to obtain a preprocessed human skeleton image;
[0057] The constructed bone segmentation model is trained based on the preprocessed human bone image to obtain a trained bone segmentation model;
[0058] The human skeleton image to be segmented is input into the trained skeleton segmentation model to obtain the skeleton segmentation image.
[0059] The constructed bone segmentation model is trained based on the preprocessed human bone image to obtain a trained bone segmentation model, including:
[0060] Constructing a skeleton segmentation model, the skeleton segmentation model comprising an input layer, an encoder, a deepest layer, a decoder and an output layer connected in sequence;
[0061] The preprocessed human skeleton image is input into the constructed skeleton segmentation model for feature extraction and prediction to obtain a predicted skeleton segmentation image;
[0062] Set the cross entropy loss function, calculate the cross entropy loss function value based on the predicted bone segmentation image and the true label, and adjust the parameters of the constructed bone segmentation model;
[0063] When the cross entropy loss function value reaches a preset value or reaches a preset number of training times, a trained bone segmentation model is obtained.
[0064] like Figure 2 As shown, the encoder includes a first convolution layer, a second convolution layer, a third convolution layer, a fourth convolution layer, a fifth convolution layer, a sixth convolution layer, a first maximum pooling layer, a second maximum pooling layer and a third maximum pooling layer;
[0065] The output end of the first convolutional layer is connected to the input end of the second convolutional layer; the output end of the second convolutional layer is connected to the input end of the first maximum pooling layer; the output end of the maximum pooling layer is connected to the input end of the third convolutional layer; the output end of the third convolutional layer is connected to the input end of the fourth convolutional layer; the output end of the fourth convolutional layer is connected to the input end of the second maximum pooling layer; the output end of the second maximum pooling layer is connected to the input end of the fifth convolutional layer; the output end of the fifth convolutional layer is connected to the input end of the sixth convolutional layer; the output end of the sixth convolutional layer is connected to the input end of the third maximum pooling layer;
[0066] The output end of the third maximum pooling layer is connected to the input end of the deepest layer;
[0067] The deepest layer includes a seventh convolutional layer and an eighth convolutional layer connected in sequence; the output end of the third maximum pooling layer is connected to the input end of the seventh convolutional layer; and the output end of the eighth convolutional layer is connected to the input end of the decoder.
[0068] The decoder includes a first upsampling layer, a second upsampling layer, a first splicing layer, a second splicing layer, a ninth convolutional layer, a tenth convolutional layer, an eleventh convolutional layer, and a twelfth convolutional layer;
[0069] The output end of the eighth convolutional layer is connected to the input end of the first upsampling layer; the output end of the first upsampling layer is connected to the input end of the first splicing layer; the output end of the first splicing layer is connected to the input end of the ninth convolutional layer; the output end of the ninth convolutional layer is connected to the input end of the tenth convolutional layer; the output end of the tenth convolutional layer is connected to the input end of the second upsampling layer; the output end of the second upsampling layer is connected to the input end of the second splicing layer; the output end of the second splicing layer is connected to the input end of the eleventh convolutional layer; the output end of the eleventh convolutional layer is connected to the input end of the twelfth convolutional layer;
[0070] The output of the twelfth convolutional layer is connected to the input of the output layer.
[0071] The output layer includes a thirteenth convolutional layer and a first activation layer connected in sequence; the output end of the twelfth convolutional layer is connected to the input end of the thirteenth convolutional layer.
[0072] The formula of the cross entropy loss function is:
[0073]
[0074] Among them, y i is the true label; p i is the predicted bone segmentation image; M is the total number of label categories.
[0075] Example 3
[0076] This embodiment also provides a method for generating a skeleton segmentation image based on U-Net, comprising:
[0077] Acquire human skeleton images;
[0078] Preprocessing the human skeleton image to obtain a preprocessed human skeleton image;
[0079] The constructed bone segmentation model is trained based on the preprocessed human bone image to obtain a trained bone segmentation model;
[0080] The human skeleton image to be segmented is input into the trained skeleton segmentation model to obtain the skeleton segmentation image.
[0081] The constructed bone segmentation model is trained based on the preprocessed human bone image to obtain a trained bone segmentation model, including:
[0082] Constructing a skeleton segmentation model, the skeleton segmentation model comprising an input layer, an encoder, a deepest layer, a decoder and an output layer connected in sequence;
[0083] The preprocessed human skeleton image is input into the constructed skeleton segmentation model for feature extraction and prediction to obtain a predicted skeleton segmentation image;
[0084] Set the cross entropy loss function, calculate the cross entropy loss function value based on the predicted bone segmentation image and the true label, and adjust the parameters of the constructed bone segmentation model;
[0085] When the cross entropy loss function value reaches a preset value or reaches a preset number of training times, a trained bone segmentation model is obtained.
[0086] The encoder includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a fifth convolutional layer, a sixth convolutional layer, a first maximum pooling layer, a second maximum pooling layer and a third maximum pooling layer;
[0087] The output end of the first convolutional layer is connected to the input end of the second convolutional layer; the output end of the second convolutional layer is connected to the input end of the first maximum pooling layer; the output end of the maximum pooling layer is connected to the input end of the third convolutional layer; the output end of the third convolutional layer is connected to the input end of the fourth convolutional layer; the output end of the fourth convolutional layer is connected to the input end of the second maximum pooling layer; the output end of the second maximum pooling layer is connected to the input end of the fifth convolutional layer; the output end of the fifth convolutional layer is connected to the input end of the sixth convolutional layer; the output end of the sixth convolutional layer is connected to the input end of the third maximum pooling layer;
[0088] The output end of the third maximum pooling layer is connected to the input end of the deepest layer;
[0089] The deepest layer includes a seventh convolutional layer and an eighth convolutional layer connected in sequence; the output end of the third maximum pooling layer is connected to the input end of the seventh convolutional layer; and the output end of the eighth convolutional layer is connected to the input end of the decoder.
[0090] The decoder includes a first upsampling layer, a second upsampling layer, a first splicing layer, a second splicing layer, a ninth convolutional layer, a tenth convolutional layer, an eleventh convolutional layer, and a twelfth convolutional layer;
[0091] The output end of the eighth convolutional layer is connected to the input end of the first upsampling layer; the output end of the first upsampling layer is connected to the input end of the first splicing layer; the output end of the first splicing layer is connected to the input end of the ninth convolutional layer; the output end of the ninth convolutional layer is connected to the input end of the tenth convolutional layer; the output end of the tenth convolutional layer is connected to the input end of the second upsampling layer; the output end of the second upsampling layer is connected to the input end of the second splicing layer; the output end of the second splicing layer is connected to the input end of the eleventh convolutional layer; the output end of the eleventh convolutional layer is connected to the input end of the twelfth convolutional layer;
[0092] The output of the twelfth convolutional layer is connected to the input of the output layer.
[0093] The output layer includes a thirteenth convolutional layer and a first activation layer connected in sequence; the output end of the twelfth convolutional layer is connected to the input end of the thirteenth convolutional layer.
[0094] The formula of the cross entropy loss function is:
[0095]
[0096] Among them, y i is the true label; p i is the predicted bone segmentation image; M is the total number of label categories.
[0097] This embodiment also provides an acupoint positioning method, such as Figure 3 As shown, including:
[0098] Acquire a skeleton segmentation image, where the skeleton segmentation image is generated using the skeleton segmentation image generation method based on U-Net described in Example 1 or Example 2;
[0099] The bone segmentation image is binarized and then calculated to obtain the Hessian matrix corresponding to each pixel in the bone segmentation image;
[0100] Calculate the determinant of the Hessian matrix of each pixel at different scales, and select the local extreme points in the determinant as key feature points;
[0101] Each key feature point is optimized using an optimization algorithm to determine the final position of all acupoints.
[0102] Use optimization algorithms to optimize key feature points and determine the final acupoint locations, including:
[0103] Construct the target point set and source point set according to each key feature point and real feature point, and calculate the corresponding nearest neighbor points according to the target point set and source point set;
[0104] The objective function is set according to each target point and its corresponding nearest neighbor point until the objective function value converges or reaches a preset number of iterations to determine the final positions of all acupoints.
[0105] The formula of the objective function is:
[0106]
[0107] Among them, R is the rotation matrix; t is the translation variable; is the target point; is the nearest neighbor point.
[0108] In a specific embodiment, human body image information is collected by taking photos, and then the data is preprocessed to perform image normalization and data enhancement.
[0109] Then, a U-Net model is constructed. The model includes an encoder-decoder architecture and skip connections to ensure efficient feature extraction. The model is trained to output a skeleton segmentation map by inputting the preprocessed image. A combination of cross entropy loss and Dice coefficient loss is used in the training process, and the model parameters are optimized using the Adam optimizer. To evaluate the model performance, indicators such as precision, recall, F1 score, and IoU are used, and the stability and generalization ability of the model are evaluated through K-fold cross validation.
[0110] In the conversion stage from bone position to acupoint, first, a standard acupoint database is obtained, which contains the precise position of acupoints relative to the bone structure. The trained U-Net model is used to segment the bone structure from the input image. During the initial registration, the key point detection algorithm is used to detect the key points in the bone image, extract the key point feature descriptors, and perform a preliminary match between the identified bone image and the standard bone model. Finally, the optimization algorithm is used to further fine-tune the acupoint position to ensure the precise match between the acupoint and the bone structure.
[0111] Finally, in the model application and deployment phase, the model is integrated into the system that controls the bionic hand. Factors such as real-time performance, reliability, and user experience also need to be considered. The application must be able to perform image processing and acupoint positioning under real-time requirements so that the massage chair can respond to user needs in a timely manner. The bionic hand will massage according to the located acupoints, thereby realizing an automated and precise acupoint massage function.
[0112] In a specific embodiment, the encoder-decoder architecture: adopts a U-Net model, which includes two parts: downsampling (encoder) and upsampling (decoder). The encoder is responsible for extracting features, and the decoder is responsible for restoring image resolution.
[0113] Skip connection: A skip connection is added between the encoder and decoder to combine feature information of different scales and improve segmentation accuracy.
[0114] Model training module: includes data enhancement, loss function selection, optimizer configuration, and evaluation indicator calculation to ensure the accuracy and stability of the model.
[0115] Data augmentation is particularly important in the U-net model. By introducing new samples without changing the semantic information of the image through geometric transformations such as rotation, scaling, translation, and flipping, the training set size of the model can be increased. And color transformation: by changing the brightness, contrast, and color space conversion of the image. By introducing the above transformations, the model will "see" more situations during training, thereby improving its prediction accuracy when facing unseen data. At the same time, it can also shorten the calculation time.
[0116] The loss function is used to measure the inconsistency between the model's predicted value and the true value. It is a non-negative real-valued function. The smaller the loss function, the better the model's training effect and the stronger its robustness. The loss function used in U-net is the weighted cross entropy loss function. L is the value of the loss function, which is a scalar. i Represents the true value. For binary classification problems, the value of y is 0 or 1; for multi-class classification problems, y i Represents the value of the i-th category in the true value label, using one-hot encoding, that is, only one element is 1 (this category is the true category) and the rest are 0. i It indicates the probability of the model predicting a positive class, and its value range is between 0 and 1. log represents the natural logarithm. The sum is taken over all categories, M is the total number of categories.
[0117] For binary classification problems, the formula of the cross entropy function can be expressed as:
[0118] L=-[ylog(p i )+(1-y)log(1-p i )i=1
[0119] For multi-classification problems, the formula can be expressed as:
[0120]
[0121] In U-net model training, the configuration of the optimizer is crucial because it directly affects the convergence speed and final performance of the model. SGD (stochastic gradient descent) is used, which estimates the overall gradient by randomly selecting a small number of samples and updates the model parameters accordingly. The advantages of SGD are simple implementation and high computational efficiency, but the disadvantages are that it may easily fall into the local optimal solution and is sensitive to the setting of the learning rate. Key parameters: Learning rate: The learning rate determines the step size of the parameter update. Too large a learning rate may cause the model to fail to converge, while too small a learning rate will make the convergence speed too slow. In SGD, it is usually necessary to manually adjust the learning rate or use a learning rate decay strategy. Momentum: Although standard SGD does not directly contain momentum terms, momentum can be introduced to accelerate the convergence of SGD in related directions and suppress oscillations. The momentum term accelerates the current gradient descent direction by accumulating past gradients. By using a learning rate decay strategy, the learning rate can be adjusted at different stages of training, and the momentum term can be added to accelerate the convergence process.
[0122] Evaluation indicators: Dice coefficient is a set similarity measurement function, usually used to calculate the similarity of two samples. The value range is [0,1]. The larger the value, the higher the overlap between the segmentation result and the true value, which indicates that the model has high segmentation accuracy and stability. |A| and |B| represent the true result and the predicted result respectively.
[0123]
[0124] The Intersection of Union (IoU) is used as a metric to describe the overlap between two boxes. An IoU close to 1 indicates a good segmentation effect, and a high overlap rate means that the segmentation result can accurately capture the target area. A and B represent the predicted bounding box and the real bounding box, respectively.
[0125]
[0126] Accuracy: The ratio of the number of correct predictions in positive and negative examples to the total number. An accuracy close to 1 indicates that the model has good classification ability on the test set and can accurately predict the category of most samples. It can be expressed as:
[0127]
[0128] Table 1
[0129] Actual positive Actual Negative Column Predict positive column TP FP Predict Negative Columns FN TN
[0130] As shown in Table 1,
[0131] Conversion of bone positions to acupuncture points
[0132] The segmentation map output by U-Net is binarized, usually the bone area is set to 1 and the background is set to 0. Then the Hessian matrix is calculated for each pixel in the segmentation map. It is used to represent the local curvature of the image, and the detection of feature points is determined by calculating the determinant of the Hessian matrix. The binarization formula is as follows:
[0133]
[0134] Among them, k represents the coordinate of the pixel in the horizontal direction (i.e., the width direction of the image), g represents the coordinate of the pixel in the vertical direction (i.e., the height direction of the image), and I b (k, g) represents the binarization result at position (k, g), which is a binary indicator function used to identify whether the position is a feature point. I(k, g) represents the gray value of the pixel point, and T is the threshold. Usually, an appropriate threshold is selected when the gray difference between the bone area and the background is obvious.
[0135] Key feature point detection: SURF uses the Hessian matrix of spot detection to detect feature points. Its determinant value represents the amount of change around the pixel point, so the feature point needs to take the maximum and minimum values of the determinant. In order to achieve scale invariance, SURF also uses the determinant value of scale σ for feature point detection. Given a point p = (s, d) in a graph, the Hessian matrix at scale σ is H(f, G):
[0136]
[0137] Where s and d represent the coordinate points (pixel positions) of the two-dimensional image, corresponding to the horizontal and vertical positions respectively. For each point in the image, s represents its horizontal coordinate, d represents its vertical coordinate, and p = (s, d) represents a certain pixel point. xx (f,σ),L xy (f,σ),L yy Functions such as (f,σ) are grayscale images after second-order differentiation. σ is the scale.
[0138] The calculation of the second-order derivative can be achieved through convolution operations, usually approximated by using a Gaussian kernel filter. The standard deviation of the Gaussian filter is σ, which is the scale parameter:
[0139]
[0140] Where N σ is the Gaussian kernel function, * represents the convolution operation, and I(f) is the grayscale value of the pixel. Calculate the determinant of the Hessian matrix:
[0141] det(H(F,σ))=L xx (f,σ)L yy(f,σ)-(L xy (f,σ)) 2
[0142] The determinant value is used to detect the most drastically changing points in the image, i.e., feature points. The SURF (Speeded Up Robust Features) algorithm uses the Hessian matrix to detect feature points, and further uses the determinant of the Hessian matrix to filter out extreme points. The specific steps of SURF are as follows:
[0143] The determinant of the Hessian matrix det(H(f,σ)) calculated above is used to judge each pixel of the image. SURF selects local extreme points as key points:
[0144]
[0145] In order to detect stable feature points at different scales, SURF performs multi-scale detection through different scale parameters G. The determinant of the Hessian matrix at each scale can be calculated separately to obtain feature points at different scales.
[0146] Finally, an optimization algorithm is used to further fine-tune the acupoint positions to ensure that the acupoints match the bone structure accurately. ICP algorithm: Given two point cloud sets:
[0147]
[0148] p t is a given target point set, target point set p t It represents the characteristic points of the acupuncture points that we hope to locate accurately. is the nth source point; p s is the source point set, is the nth target point; is the feature point extracted from the actual image. Initially assume that the rotation matrix R is the unit matrix and the translation vector t is the zero vector. For each target point Find the point closest to the source point in the set The nearest neighbor search can be implemented through efficient data structures such as KD-Tree. The formula for calculating the nearest neighbor point is:
[0149]
[0150] Calculate the rigid body transformation (rotation matrix R and translation vector t) that minimizes the transformation from the source point set to the target point set. The goal of the rigid body transformation is to minimize the objective function:
[0151]
[0152] Among them, R is the rotation matrix; t is the translation variable; is the i-th target point; is the i-th source point, N p is an index label; the index label is the position corresponding to the target point in the target point set and the source point in the source point set; the optimal point is found by continuously fixing R and t using the nearest neighbor algorithm.
[0153] Update the source point set using the new R and t:
[0154] P s′ =RP s +t x
[0155] where t x Represents the new translation vector. Until the value of the objective function converges, that is, the error no longer decreases significantly, or the predetermined number of iterations is reached. Model application and deployment
[0156] Real-time processing and prediction module: pre-process the newly input image, use Gaussian filtering to remove noise in the image and improve the image quality. The processed image will then be input into the trained U-Net model to achieve bone segmentation and acupoint location. The convolution structure of U-Net can effectively extract image features and provide accurate segmentation results.
[0157] Result visualization and verification module: The segmentation and localization results will be displayed through visualization tools so that users can intuitively verify the accuracy of the segmentation. This process can highlight the segmented bones and located acupoints using bounding boxes or color markings, which is convenient for subsequent adjustment and optimization.
[0158] Bionic hand control module: Integrating the model into the system that controls the bionic hand is the key to achieving real-time image processing, acupoint positioning and precise control. Specifically, the system captures images in real time, and after denoising and U-Net segmentation, the extracted key point information will be used to control the movement of the bionic hand. By mapping the segmentation results to control instructions, the bionic hand can be flexibly operated according to the needs of the user. In the real-time image processing link, the camera will continue to capture images of the target area. After the acquired image is denoised and predicted by the model, the control module will convert the predicted acupoint coordinates into motion instructions for the bionic hand to ensure that the hand movement is consistent with the positioning results. The entire system needs to ensure high efficiency and low latency to support real-time operations of users.
[0159] The same or similar reference numerals correspond to the same or similar components;
[0160] The terms used in the drawings to describe positional relationships are only used for illustrative purposes and should not be construed as limiting this patent;
[0161] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.
Claims
1. A method for generating a skeleton segmentation image based on U-Net, characterized in that: include: Acquire human skeleton images; Preprocessing the human skeleton image to obtain a preprocessed human skeleton image; The constructed bone segmentation model is trained based on the preprocessed human bone image to obtain a trained bone segmentation model; The human skeleton image to be segmented is input into the trained skeleton segmentation model to obtain the skeleton segmentation image.
2. The method for generating a skeleton segmentation image based on U-Net according to claim 1, characterized in that: The constructed bone segmentation model is trained based on the preprocessed human bone image to obtain a trained bone segmentation model, including: Constructing a skeleton segmentation model, the skeleton segmentation model comprising an input layer, an encoder, a deepest layer, a decoder and an output layer connected in sequence; The preprocessed human skeleton image is input into the constructed skeleton segmentation model for feature extraction and prediction to obtain a predicted skeleton segmentation image; Set the cross entropy loss function, calculate the cross entropy loss function value based on the predicted bone segmentation image and the true label, and adjust the parameters of the constructed bone segmentation model; When the cross entropy loss function value reaches a preset value or reaches a preset number of training times, a trained bone segmentation model is obtained.
3. The method for generating a skeleton segmentation image based on U-Net according to claim 2, characterized in that: The encoder includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a fifth convolutional layer, a sixth convolutional layer, a first maximum pooling layer, a second maximum pooling layer and a third maximum pooling layer; The output end of the first convolutional layer is connected to the input end of the second convolutional layer; the output end of the second convolutional layer is connected to the input end of the first maximum pooling layer; the output end of the maximum pooling layer is connected to the input end of the third convolutional layer; the output end of the third convolutional layer is connected to the input end of the fourth convolutional layer; the output end of the fourth convolutional layer is connected to the input end of the second maximum pooling layer; the output end of the second maximum pooling layer is connected to the input end of the fifth convolutional layer; the output end of the fifth convolutional layer is connected to the input end of the sixth convolutional layer; the output end of the sixth convolutional layer is connected to the input end of the third maximum pooling layer; The output end of the third maximum pooling layer is connected to the input end of the deepest layer.
4. The method for generating a skeleton segmentation image based on U-Net according to claim 3, characterized in that: The deepest layer includes a seventh convolutional layer and an eighth convolutional layer connected in sequence; the output end of the third maximum pooling layer is connected to the input end of the seventh convolutional layer; and the output end of the eighth convolutional layer is connected to the input end of the decoder.
5. The method for generating a skeleton segmentation image based on U-Net according to claim 4, characterized in that: The decoder includes a first upsampling layer, a second upsampling layer, a first splicing layer, a second splicing layer, a ninth convolutional layer, a tenth convolutional layer, an eleventh convolutional layer, and a twelfth convolutional layer; The output end of the eighth convolutional layer is connected to the input end of the first upsampling layer; the output end of the first upsampling layer is connected to the input end of the first splicing layer; the output end of the first splicing layer is connected to the input end of the ninth convolutional layer; the output end of the ninth convolutional layer is connected to the input end of the tenth convolutional layer; the output end of the tenth convolutional layer is connected to the input end of the second upsampling layer; the output end of the second upsampling layer is connected to the input end of the second splicing layer; The output end of the second concatenation layer is connected to the input end of the eleventh convolution layer; the output end of the eleventh convolution layer is connected to the input end of the twelfth convolution layer; The output of the twelfth convolutional layer is connected to the input of the output layer.
6. The method for generating a skeleton segmentation image based on U-Net according to claim 5, characterized in that: The output layer includes a thirteenth convolutional layer and a first activation layer connected in sequence; the output end of the twelfth convolutional layer is connected to the input end of the thirteenth convolutional layer.
7. The method for generating a skeleton segmentation image based on U-Net according to claim 2, characterized in that: The formula of the cross entropy loss function is: Among them, y i is the true label; p i is the predicted bone segmentation image; M is the total number of label categories.
8. A method for locating acupoints, characterized in that: include: Acquire a bone segmentation image, wherein the bone segmentation image is generated by using the U-Net-based bone segmentation image generation method according to any one of claims 1 to 6; The bone segmentation image is binarized and then calculated to obtain the Hessian matrix corresponding to each pixel in the bone segmentation image; Calculate the determinant of the Hessian matrix of each pixel at different scales, and select the local extreme points in the determinant as key feature points; Each key feature point is optimized using an optimization algorithm to determine the final position of all acupoints.
9. The acupoint locating method according to claim 8, characterized in that: Use optimization algorithms to optimize key feature points and determine the final acupoint locations, including: Construct the target point set and source point set according to each key feature point and real feature point, and calculate the corresponding nearest neighbor points according to the target point set and source point set; The objective function is set according to each target point and its corresponding nearest neighbor point until the objective function value converges or reaches a preset number of iterations to determine the final positions of all acupoints.
10. The acupoint locating method according to claim 9, characterized in that: The formula of the objective function is: Among them, R is the rotation matrix; t is the translation variable; is the i-th target point; is the i-th source point, N p For index labels.