Precise positioning method and system based on posture recognition
Through deep learning methods based on posture recognition, an accurate positioning model is constructed, which solves the problem of organ misalignment caused by incorrect posture of medical scanning equipment, improves positioning accuracy and equipment efficiency, and is suitable for the lightweight deployment of medical scanning equipment.
Patent Information
- Application Number
- CN202411065741.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-05
- Publication Date
- 2024-11-08
AI Technical Summary
When taking patient report films, the medical scanning equipment needs to reshoot due to incorrect postures of the patient, which increases the cost and cumbersomeness.
The precise positioning method based on posture recognition is adopted, and the patient's whole body is scanned by a camera, and the precise positioning model is constructed, trained and verified, and the model is optimized to identify the location of each organ of the patient. This method includes image acquisition, processing, model construction, training and verification, using deep learning technologies such as convolutional layer, pooling layer, jump connection and fully connected layer, designing loss functions and optimization algorithms, combining pruning and quantization techniques, and generating Lightweight model.
It improves the precise positioning accuracy of each organ of the patient, reduces the cost of repeated shooting, is suitable for resource-constrained equipment deployment, and reduces storage and computing needs.
Smart Images

Figure CN118902441A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of precise positioning based on gesture recognition, and in particular to a precise positioning method and system based on gesture recognition. Background Art
[0002] Common medical scanning equipment, such as X-ray machines, gamma-ray machines, and magnetic resonance imaging equipment, etc., when scanning patients, it is difficult to ensure that the patient's posture completely meets the detection requirements. This sometimes results in the report film not meeting the requirements due to the misalignment or unclearness of some organs due to incorrect patient posture and angle. Therefore, it is sometimes necessary to re-shoot the test, which is more cumbersome and leads to increased costs. Therefore, in order to accurately locate the organs of the patient in different postures, a precise positioning method and system based on posture recognition is invented. Summary of the invention
[0003] In order to accurately locate the organs of a patient in different postures, the present invention provides a precise positioning method and system based on posture recognition, which uses a camera to scan a specific area, then constructs a precise positioning model, and trains, verifies and optimizes the precise positioning model. Finally, the optimized model is used to perform posture recognition on the original full-body image of the patient, thereby accurately locating the position of the patient's organs.
[0004] In the first aspect, the present invention provides a precise positioning method based on posture recognition, which adopts the following technical solution:
[0005] A precise positioning method based on posture recognition comprises the following steps:
[0006] S1. Equipment installation:
[0007] Set up equally spaced marking points on both sides of the bed, and set up a camera directly above the bed so that the camera can capture the area directly below and diagonally below the bed;
[0008] S2. Image acquisition and processing:
[0009] Image acquisition, moving the bed at a constant speed, so that the camera acquires a full-body image of the patient, and generates an original full-body image of the patient, wherein the original full-body image of the patient includes an original full-body image of the patient in a supine position and an original full-body image of the patient in a prone position;
[0010] Image Processing:
[0011] Organ labeling: labeling each organ in the original patient's whole body image;
[0012] Format conversion, performing format conversion and data normalization on the original patient whole body image, and normalizing the patient whole body image;
[0013] Image enhancement, performing color transformation, advanced transformation and combined enhancement on the normalized patient whole body image to obtain an enhanced patient whole body image;
[0014] S3, model construction: build an accurate positioning model based on the enhanced patient's whole body image;
[0015] Precision positioning models include:
[0016] 1) Input layer;
[0017] Input image format: The input layer accepts fixed-size images, which are normalized to the same size and resolution through preprocessing;
[0018] Input channel: Usually the input image contains three channels, namely width channel, height channel and color channel;
[0019] 2) Convolutional layer;
[0020] Feature extraction: The convolution layer extracts features through convolution kernels, extracts low-level edge and texture features, and then extracts high-level semantic features layer by layer;
[0021] Convolution kernel size: Use 3x3 or 5x5 convolution kernel;
[0022] Activation function: Use nonlinear activation function to increase the nonlinear representation ability of the network;
[0023] 3) Pooling layer;
[0024] Downsampling: The pooling layer downsamples the feature map through maximum pooling or average pooling operations to reduce the amount of calculation and retain important features;
[0025] Pooling window: Use a 2x2 pooling window to compress the spatial size of the feature map by gradually downsampling;
[0026] 4) Skip connection;
[0027] Residual connection: Add residual connections between specific layers, directly adding the output of the previous layer to the input of the next layer;
[0028] 5) Fully connected layer;
[0029] Feature mapping: Several fully connected layers map the features extracted by the convolutional layer to a specific output space to generate the predicted regions of each organ;
[0030] Output dimension: Determine the output dimension of the fully connected layer based on the number of bounding boxes that need to be predicted and the parameters of each bounding box;
[0031] 6) Output layer;
[0032] Bounding box prediction: The output layer predicts the specific location and range of each organ through multiple bounding box predictors. The bounding box is usually expressed in the form of (x, y, w, h), where (x, y) is the center coordinate and (w, h) is the width and height;
[0033] Confidence score: Each bounding box also outputs a confidence score, indicating the credibility of the prediction;
[0034] Category probability: Output the probability distribution of the category to which each bounding box belongs and determine the specific organ category;
[0035] S4. Design a loss function for the precise positioning model;
[0036] The loss functions include:
[0037] 1) Position loss;
[0038] Regression loss: Use the L1 and L2 loss weights to calculate the error between the predicted bounding box and the true bounding box, where the position loss formula is:
[0039] Position loss = ∑i|pi-gi|\text{Position loss
[0040] loss} = \sum_{i}\left|p_i-g_i\right|position loss = ∑i|pi-gi|; or,
[0041] Position loss = ∑i(pi-gi)2\text{Position loss
[0042] Loss} = \sum_{i}\left(p_i-g_i\right)^2 Position loss = ∑i(pi-gi)2, where pip_ipi is the predicted bounding box parameter and gig_igi is the true bounding box parameter;
[0043] 2) Classification loss;
[0044] Cross entropy loss: The error between the predicted category and the true category is calculated using cross entropy loss, which is used to measure the difference between two probability distributions; the classification loss formula is:
[0045] Classification loss = -∑igilog(pi), where pip_ipi is the predicted category probability and gig_igi is the one-hot encoding of the true category;
[0046] 3) Confidence loss;
[0047] Mean square error: The error between the predicted confidence and the true confidence is calculated using the mean square error, which is used to measure the square difference between the predicted value and the true value; the formula for confidence loss is:
[0048] Confidence loss = 1N∑i(pi-gi)2\text{Confidence loss} = \frac{1}{N}\sum_{i}\left(p_i-g_i\right)^2Confidence loss = N1∑i(pi-gi)2, where pip_ipi is the predicted confidence, gig_igi is the true confidence, and NNN is the number of samples;
[0049] 4) Total loss;
[0050] Weighted total loss: The position loss, classification loss and confidence loss are combined to form a total loss. The position loss, classification loss and confidence loss are weighted to guide the optimization of the model. The total loss can be a weighted combination of the losses of each part according to specific needs; the formula for the total loss is:
[0051] Total loss = α position loss + β classification loss + γ confidence loss\text{Total loss}
[0052] =\alpha\cdot\text{position loss}+\beta\cdot\text{classification loss}+\gamma\cdot\text{confidence loss}Total loss = α position loss + β classification loss + γ confidence loss, where α\alphaα, β\betaβ and γ\gammaγ are weighting coefficients, which adjust the weight of each part of the loss in the total loss;
[0053] S5. Train the precise positioning model to obtain a precise positioning training model;
[0054] Training the precise positioning model consists of the following steps:
[0055] 1) Data preprocessing;
[0056] Image normalization: normalize the pixel values of the input image to the range of 0-1, standardize the image data, and ensure input consistency;
[0057] Data augmentation: During training, images are randomly cropped, rotated, flipped, and scaled to increase data diversity.
[0058] 2) Batch training;
[0059] Small batch training: Divide the training data into several small batches, input each batch into the model for forward propagation and back propagation, and update the precise positioning model parameters;
[0060] Batch size: Choose an appropriate batch size to make efficient use of the GPU / CPU during training;
[0061] 3) Optimization algorithm;
[0062] Stochastic Gradient Descent: Use the SGD optimization algorithm to minimize the loss function and optimize the parameters of the precision positioning model by gradually adjusting the weights and biases of the precision positioning model;
[0063] Momentum method: introduce momentum terms, combine the directions of previous gradients, smooth gradient updates, reduce oscillations, and accelerate convergence;
[0064] Adam optimizer: Use the Adam optimizer combined with momentum and adaptive learning rate adjustment;
[0065] 4) Hyperparameter adjustment;
[0066] Learning rate: Choose an appropriate learning rate to accurately locate the model parameters and update them stably;
[0067] Regularization: Regularization terms are introduced to prevent overfitting of the precise positioning model and enhance the generalization ability of the precise positioning model;
[0068] Batch size: Adjust the batch size to balance training speed and stability so that the precision positioning model can be trained effectively;
[0069] Get the accurate positioning training model;
[0070] S6. Verify the precise positioning training model. Verifying the precise positioning training model includes the following steps:
[0071] 1) Validation dataset;
[0072] Independent validation set: Use a data set independent of the training set to evaluate the performance of the precise positioning training model, so that the precise positioning training model has good generalization ability on unseen data;
[0073] Verification data preparation: Perform the same preprocessing and normalization operations on the verification data as the training data to ensure the consistency of the input data;
[0074] 2) Index evaluation;
[0075] Accuracy: measures the accuracy of the prediction results of the precision positioning training model and calculates the proportion of correct predictions;
[0076] Recall rate: measures the recall ability of the precision positioning training model for real positive samples and calculates the proportion of correctly predicted positive samples;
[0077] F1 score: A comprehensive indicator that combines precision and recall to measure the overall performance of the precision positioning training model;
[0078] IOU intersection-over-union ratio: Calculates the ratio of the overlapping area to the joint area between the predicted bounding box and the true bounding box to measure the accuracy of the bounding box prediction;
[0079] 3) Early stopping strategy;
[0080] Validation loss monitoring: monitor the change in loss on the validation set. If the validation loss no longer decreases after several rounds of training, terminate the training early to prevent overfitting of the precise positioning training model.
[0081] Model saving: Save the model parameters when the validation loss is the lowest to ensure the best performance of the final accurate positioning training model;
[0082] S7, optimizing the precise positioning training model, pruning the precise positioning training model, and obtaining a precise positioning optimization model;
[0083] Optimizing the precise positioning training model includes the following steps:
[0084] 1) Weight pruning: Prune the weights according to their importance and remove the weights with low importance;
[0085] Weight importance evaluation: Calculate the absolute value or square value of each weight to measure the impact of the absolute value or square value on the model output;
[0086] Pruning strategy: set a pruning threshold, reset weights below the threshold to zero, and retain weights of high importance;
[0087] Fine-tuning training: After pruning, fine-tune the model to restore model performance and ensure that pruning has minimal impact on model accuracy;
[0088] 2) Filter pruning: According to the importance of the convolution kernel, the filter of the convolution layer is pruned to reduce the computational complexity of the convolution operation;
[0089] Filter importance evaluation: Calculate the L1 norm or L2 norm of each filter to measure the contribution of L1 norm or L2 norm to feature extraction;
[0090] Pruning strategy: set the pruning ratio, remove filters with smaller contributions, and retain filters with main features;
[0091] Fine-tuning training: After pruning, fine-tune the model to restore model performance and ensure that pruning has minimal impact on feature extraction capabilities;
[0092] S8. Use the precise positioning optimization model to process and analyze the collected original patient whole body images to accurately locate the patient's organs.
[0093] Preferably, the training of the precise positioning model further comprises adopting a quantization technology, and the quantization technology comprises the following steps:
[0094] 1) Static quantization: After the model training is completed, the model parameters are quantized;
[0095] Quantization step: convert floating-point weights and activation values into integer representations. Common methods include linear quantization and nonlinear quantization.
[0096] Quantization calibration: Use the calibration data set to calculate the quantization parameters to ensure that the accuracy of the quantized model is not significantly affected;
[0097] 2) Dynamic quantization: During the model inference process, the model parameters are dynamically quantized;
[0098] Dynamic quantization step: During the inference process, floating-point parameters are dynamically converted to low-precision representation to reduce memory usage and computational complexity;
[0099] Applicable scenarios: Dynamic quantization is applicable to the model parts with large computational workload during the inference process, including fully connected layers and RNN layers;
[0100] 3) Quantization-aware training (QAT): quantization operations are added during model training to adapt the model to the low-precision representation after quantization.
[0101] During training, the quantization operation is simulated to convert floating-point weights and activation values into low-precision representations, and the loss is calculated for backpropagation.
[0102] In a second aspect, the present invention provides a precise positioning system based on posture recognition, which adopts the following technical solution:
[0103] A precise positioning system based on gesture recognition, applied to the precise positioning method based on gesture recognition described above, comprising:
[0104] Equipment with scanning function and movable bed, for patients lying on the bed to collect full-body images of the patients;
[0105] Camera, used to collect the patient's whole body image and generate the original patient's whole body image for uploading;
[0106] An image processing module, used for performing organ and format conversion and image enhancement processing on the original patient's whole body image;
[0107] Model building module, used to build precise positioning module;
[0108] The loss function module is used to design the loss function for the precise positioning module;
[0109] A model training module is used to train the precise positioning model to obtain a precise positioning training model;
[0110] The model verification module is used to verify the precise positioning training model and obtain the precise positioning verification model;
[0111] Model optimization module, used to optimize the precise positioning training module and obtain the precise positioning optimization model;
[0112] The model precise positioning module is used to accurately locate the patient's organs based on the precise positioning optimization model.
[0113] Preferably, the image processing module includes:
[0114] The organ labeling unit is used to label the various organs in the original patient's whole body image.
[0115] A format conversion unit, used for successively performing format conversion and data normalization on an original patient whole body image, and normalizing the patient whole body image;
[0116] The image enhancement unit is used for performing color transformation, advanced transformation and combined enhancement on the normalized patient whole body image to obtain an enhanced patient whole body image.
[0117] Preferably, the loss function module includes:
[0118] A position loss calculation unit, used for calculating the position loss;
[0119] Classification loss calculation unit, used to calculate classification loss;
[0120] A confidence calculation unit, used for calculating confidence;
[0121] The total loss calculation unit is used to calculate the total loss.
[0122] Preferably, the model training module includes:
[0123] A data preprocessing unit, used for preprocessing images, including image normalization and data enhancement;
[0124] Batch training unit, used to perform batch training on images. Batch training includes small batch training and selecting a suitable batch size.
[0125] Optimization algorithm unit, used for optimization algorithms, including stochastic gradient descent and momentum method;
[0126] Hyperparameter tuning unit, used for hyperparameter tuning, which includes selecting an appropriate learning rate, introducing regularization terms, and adjusting the batch size.
[0127] Preferably, the model verification module includes:
[0128] A data set verification unit, used to verify the data set;
[0129] An indicator evaluation unit, used to evaluate the indicator;
[0130] Early stopping strategy unit, used for strategic early stopping.
[0131] Preferably, the model optimization module includes:
[0132] A weight pruning unit, used to prune weights;
[0133] The filter pruning unit prunes the filters of the convolutional layer.
[0134] Preferably, a quantization technology module is also included, which is used to perform static quantization, dynamic quantization and quantization-aware training QAT on the parameters of the precise positioning training model.
[0135] In summary, the present invention includes the following beneficial technical effects:
[0136] 1. The present invention uses a camera to scan a specific area, then constructs a precise positioning model, trains, verifies and optimizes the precise positioning model, and finally uses the optimized model to perform posture recognition on the patient's original full-body image, thereby accurately locating the position of each organ of the patient.
[0137] 2. The present invention improves the accuracy of precise positioning of various organs of patients and improves the precise positioning capability of the precise positioning model by training, verifying and optimizing the constructed precise positioning model.
[0138] 3. The present invention uses a precise positioning model to accurately identify and locate various organs of the patient in different postures, thereby improving the usefulness of the films taken by the detection equipment and reducing the cost of repeated shooting.
[0139] 4. The present invention compresses the model size and reduces storage and computing requirements through pruning and quantization techniques, and generates a lightweight model version by combining pruning and quantization techniques, which is suitable for deployment on resource-constrained devices. BRIEF DESCRIPTION OF THE DRAWINGS
[0140] Figure 1 It is a schematic diagram of the structure of a scanning device in an embodiment of the present invention.
[0141] Figure 2 yes Figure 1Left view in .
[0142] Explanation of the accompanying drawings: 1. Scanning device; 2. Bed; 3. Camera. DETAILED DESCRIPTION
[0143] The present invention is further described in detail below in conjunction with the accompanying drawings.
[0144] The embodiment of the invention discloses a precise positioning method based on posture recognition.
[0145] The precise positioning method based on posture recognition includes the following steps:
[0146] S1. Equipment installation;
[0147] A scanning device 1 is selected, and the scanning device 1 can be an X-ray machine, a gamma-ray machine, a nuclear magnetic resonance device, or other device with a movable bed 2 .
[0148] Marking points are set at equal intervals on both sides of the bed 2 of the scanning device 1, and a camera 3 is set just above the bed so that the camera 3 can shoot the area just below and diagonally below the bed;
[0149] Specifically, when installing the camera 3, the camera 3 is enabled to shoot the bed 2 on the scanning device 1, and the camera 3 is installed at a position such as Figure 1 The side end of the scanning device 1 shown enables the camera 3 to capture the bed 2 directly below it and the area within sixty degrees to the side and front of it.
[0150] Camera 3 uses a distortion-free camera and performs anti-electromagnetic interference processing on camera 3 to prevent the image and data transmission collected by camera 3 from being affected. At the same time, a high-computing power main control board is installed on camera 3 to perform edge algorithm processing, such as RK3588.
[0151] S2, image acquisition and processing, image acquisition and processing includes image acquisition and image processing, specifically:
[0152] Image acquisition: start the scanning device 1 to move the bed 2 at a constant speed, and enable the camera 3 to acquire a full-body image of the patient, and generate an original full-body image of the patient, the original full-body image of the patient including an original full-body image of the patient in a supine position and an original full-body image of the patient in a prone position;
[0153] Image Processing:
[0154] Organ labeling processing: labeling each organ in the original patient's whole body image;
[0155] Format conversion processing, which converts the format of the original patient's whole body image and performs data normalization, and normalizes the patient's whole body image;
[0156] Image enhancement processing performs color transformation, advanced transformation and combined enhancement on the normalized patient whole body image to obtain an enhanced patient whole body image.
[0157] S3, model construction: build an accurate positioning model based on the enhanced patient's whole body image;
[0158] The precise positioning model constructed includes:
[0159] 1) Input layer;
[0160] Input image format: The input layer accepts fixed-size images, which are normalized to the same size and resolution through preprocessing;
[0161] Input channel: Usually the input image contains three channels, namely width channel, height channel and color channel;
[0162] 2) Convolutional layer;
[0163] Feature extraction: The convolution layer extracts features through convolution kernels, extracts low-level edge and texture features, and then extracts high-level semantic features layer by layer;
[0164] Convolution kernel size: Use 3x3 or 5x5 convolution kernel;
[0165] Activation function: Use nonlinear activation function to increase the nonlinear representation ability of the network;
[0166] 3) Pooling layer;
[0167] Downsampling: The pooling layer downsamples the feature map through maximum pooling or average pooling operations to reduce the amount of calculation and retain important features;
[0168] Pooling window: Use a 2x2 pooling window to compress the spatial size of the feature map by gradually downsampling;
[0169] 4) Skip connection;
[0170] Residual connection: Add residual connections between specific layers, directly adding the output of the previous layer to the input of the next layer;
[0171] 5) Fully connected layer;
[0172] Feature mapping: Several fully connected layers map the features extracted by the convolutional layer to a specific output space to generate the predicted regions of each organ;
[0173] Output dimension: Determine the output dimension of the fully connected layer based on the number of bounding boxes that need to be predicted and the parameters of each bounding box;
[0174] 6) Output layer;
[0175] Bounding box prediction: The output layer predicts the specific location and range of each organ through multiple bounding box predictors. The bounding box is usually expressed in the form of (x, y, w, h), where (x, y) is the center coordinate and (w, h) is the width and height;
[0176] Confidence score: Each bounding box also outputs a confidence score, indicating the credibility of the prediction;
[0177] Category probability: Output the probability distribution of the category to which each bounding box belongs to determine the specific organ category.
[0178] S4. Design a loss function for the precise positioning model;
[0179] The loss function includes position loss, classification loss, confidence loss and total loss, specifically:
[0180] 1) Position loss;
[0181] Regression loss: Use the L1 and L2 loss weights to calculate the error between the predicted bounding box and the true bounding box, where the position loss formula is:
[0182] Position loss = ∑i|pi-gi|\text{Position loss
[0183] loss} = \sum_{i}\left|p_i-g_i\right|position loss = ∑i|pi-gi|; or,
[0184] Position loss = ∑i(pi-gi)2\text{Position loss
[0185] Loss} = \sum_{i}\left(p_i-g_i\right)^2 Position loss = ∑i(pi-gi)2, where pip_ipi is the predicted bounding box parameter and gig_igi is the true bounding box parameter;
[0186] 2) Classification loss;
[0187] Cross entropy loss: The error between the predicted category and the true category is calculated using cross entropy loss, which is used to measure the difference between two probability distributions; the classification loss formula is:
[0188] Classification loss = -∑igilog(pi), where pip_ipi is the predicted category probability and gig_igi is the one-hot encoding of the true category;
[0189] 3) Confidence loss;
[0190] Mean square error: The error between the predicted confidence and the true confidence is calculated using the mean square error, which is used to measure the square difference between the predicted value and the true value; the formula for confidence loss is:
[0191] Confidence loss = 1N∑i(pi-gi)2\text{Confidence loss} = \frac{1}{N}\sum_{i}\left(p_i-g_i\right)^2Confidence loss = N1∑i(pi-gi)2, where pip_ipi is the predicted confidence, gig_igi is the true confidence, and NNN is the number of samples;
[0192] 4) Total loss;
[0193] Weighted total loss: The position loss, classification loss and confidence loss are combined to form a total loss. The position loss, classification loss and confidence loss are weighted to guide the optimization of the model. The total loss can be a weighted combination of the losses of each part according to specific needs; the formula for the total loss is:
[0194] Total loss = α position loss + β classification loss + γ confidence loss\text{Total loss} = \alpha\cdot\text{position loss} + \beta\cdot\text{classification loss} + \gamma\cdot\text{confidence loss}Total loss = α position loss + β classification loss + γ confidence loss, where α\alphaα, β\betaβ and γ\gammaγ are weighting coefficients that adjust the weight of each part of the loss in the total loss.
[0195] S5. Train the precise positioning model to obtain a precise positioning training model;
[0196] Training the precise positioning model consists of the following steps:
[0197] 1) Data preprocessing;
[0198] Image normalization: normalize the pixel values of the input image to the range of 0-1, standardize the image data, and ensure input consistency;
[0199] Data augmentation: During training, images are randomly cropped, rotated, flipped, and scaled to increase data diversity.
[0200] 2) Batch training;
[0201] Small batch training: Divide the training data into several small batches, input each batch into the model for forward propagation and back propagation, and update the precise positioning model parameters;
[0202] Batch size: Choose an appropriate batch size to make efficient use of the GPU / CPU during training;
[0203] 3) Optimization algorithm;
[0204] Stochastic Gradient Descent: Use the SGD optimization algorithm to minimize the loss function and optimize the parameters of the precision positioning model by gradually adjusting the weights and biases of the precision positioning model;
[0205] Momentum method: introduce momentum terms, combine the directions of previous gradients, smooth gradient updates, reduce oscillations, and accelerate convergence;
[0206] Adam optimizer: Use the Adam optimizer combined with momentum and adaptive learning rate adjustment;
[0207] 4) Hyperparameter adjustment;
[0208] Learning rate: Choose an appropriate learning rate to accurately locate the model parameters and update them stably;
[0209] Regularization: Regularization terms are introduced to prevent overfitting of the precise positioning model and enhance the generalization ability of the precise positioning model;
[0210] Batch size: Adjust the batch size to balance training speed and stability so that the precision positioning model can be trained effectively;
[0211] Get the accurate positioning training model.
[0212] S6. Verify the precise positioning training model. Verifying the precise positioning training model includes the following steps:
[0213] 1) Validation dataset;
[0214] Independent validation set: Use a data set independent of the training set to evaluate the performance of the precise positioning training model, so that the precise positioning training model has good generalization ability on unseen data;
[0215] Verification data preparation: Perform the same preprocessing and normalization operations on the verification data as the training data to ensure the consistency of the input data;
[0216] 2) Index evaluation;
[0217] Accuracy: measures the accuracy of the prediction results of the precision positioning training model and calculates the proportion of correct predictions;
[0218] Recall rate: measures the recall ability of the precision positioning training model for real positive samples and calculates the proportion of correctly predicted positive samples;
[0219] F1 score: A comprehensive indicator that combines precision and recall to measure the overall performance of the precision positioning training model;
[0220] IOU intersection-over-union ratio: Calculates the ratio of the overlapping area to the joint area between the predicted bounding box and the true bounding box to measure the accuracy of the bounding box prediction;
[0221] 3) Early stopping strategy;
[0222] Validation loss monitoring: monitor the change in loss on the validation set. If the validation loss no longer decreases after several rounds of training, terminate the training early to prevent overfitting of the precise positioning training model.
[0223] Model saving: Save the model parameters when the validation loss is the lowest to ensure the best performance of the final accurate positioning training model;
[0224] After the model training is completed, the parameters of the precise positioning training model are quantized using quantization technology. The quantization technology includes the following steps:
[0225] 1) Static quantization: After the model training is completed, the model parameters are quantized;
[0226] Quantization step: convert floating-point weights and activation values into integer representations. Common methods include linear quantization and nonlinear quantization.
[0227] Quantization calibration: Use the calibration data set to calculate the quantization parameters to ensure that the accuracy of the quantized model is not significantly affected;
[0228] 2) Dynamic quantization: During the model inference process, the model parameters are dynamically quantized;
[0229] Dynamic quantization step: During the inference process, floating-point parameters are dynamically converted to low-precision representation to reduce memory usage and computational complexity;
[0230] Applicable scenarios: Dynamic quantization is applicable to the model parts with large computational workload during the inference process, including fully connected layers and RNN layers;
[0231] 3) Quantization-aware training (QAT): quantization operations are added during model training to adapt the model to the low-precision representation after quantization.
[0232] During training, the quantization operation is simulated to convert floating-point weights and activation values into low-precision representations, and the loss is calculated for backpropagation.
[0233] S7, optimizing the precise positioning training model, pruning the precise positioning training model, and obtaining a precise positioning optimization model;
[0234] Optimizing the precise positioning training model includes the following steps:
[0235] 1) Weight pruning: Prune the weights according to their importance and remove the weights with low importance;
[0236] Weight importance evaluation: Calculate the absolute value or square value of each weight to measure the impact of the absolute value or square value on the model output;
[0237] Pruning strategy: set a pruning threshold, reset weights below the threshold to zero, and retain weights of high importance;
[0238] Fine-tuning training: After pruning, fine-tune the model to restore model performance and ensure that pruning has minimal impact on model accuracy;
[0239] 2) Filter pruning: According to the importance of the convolution kernel, the filter of the convolution layer is pruned to reduce the computational complexity of the convolution operation;
[0240] Filter importance evaluation: Calculate the L1 norm or L2 norm of each filter to measure the contribution of L1 norm or L2 norm to feature extraction;
[0241] Pruning strategy: set the pruning ratio, remove filters with smaller contributions, and retain filters with main features;
[0242] Fine-tuning training: After pruning, fine-tune the model to restore model performance and ensure that pruning has minimal impact on feature extraction capabilities;
[0243] S8. Use the precise positioning optimization model to process and analyze the collected original patient whole body images to accurately locate the patient's organs.
[0244] Specifically, preprocessing is performed based on the acquired images;
[0245] Gaussian filtering and mean filtering are used to denoise the image. Gaussian filtering removes random noise in the image through convolution operations while retaining the edge and detail information of the image.
[0246] Enhance the contrast and brightness of the image through histogram equalization and contrast stretching. Histogram equalization adjusts the grayscale distribution of the image to make the contrast of the image more uniform; contrast stretching further improves the contrast of the image by stretching the grayscale range of the image.
[0247] Extract features from images;
[0248] The SIFT algorithm is used to detect and describe local feature points in the image. SIFT features are scale-invariant and rotation-invariant, and can remain stable under different viewing angles and lighting conditions. SIFT detects key points through Gaussian difference (DoG) and generates feature vectors through main direction assignment and local feature description.
[0249] Adopt FLANN (Fast Library for Approximate Nearest Neighbors): FLANN is a fast approximate nearest neighbor search algorithm that can efficiently find matching point pairs in large-scale data sets. FLANN quickly finds approximate nearest neighbors by constructing multiple random trees and performing recursive searches.
[0250] Estimating the transformation of the image;
[0251] Use perspective transformation: According to the estimated homography matrix, the image is perspective transformed and projected onto the same plane to ensure image alignment and stitching.
[0252] Fusion of images;
[0253] The perspective transformed images are stitched together and the overlapping areas are weighted averaged to reduce stitching gaps and artifacts. The weighted average process generates a smooth transition effect by weighted summing the pixel values in the overlapping areas.
[0254] Perform color adjustment on the stitched images to ensure the consistency of hue and brightness and improve the visual effect of the image. Color adjustment adjusts the grayscale distribution and color value of the image through histogram matching and color balance technology.
[0255] Optimize the image results;
[0256] Global optimization is used to further optimize the stitching results and reduce the cumulative error through global optimization algorithms such as gradient descent and simulated annealing. Global optimization minimizes the overall error of the stitched image, adjusts the transformation parameters and fusion methods, and ensures the accuracy and stability of the stitching results.
[0257] The stitching gap removal is adopted, and the image patching technology is used to remove the stitching gap to ensure the integrity and naturalness of the stitching image. The image patching technology includes texture synthesis and image patching algorithms, which generate a continuous image effect by filling the pixel values of the gap area.
[0258] Ultimately, the patient's organs can be accurately located.
[0259] The embodiment of the present application further discloses a precise positioning system based on gesture recognition, and the precise positioning system based on gesture recognition is applied to the precise positioning method based on gesture recognition disclosed in the above embodiment.
[0260] The precise positioning system based on gesture recognition includes the following modules:
[0261] Equipment with scanning function and movable bed, used for collecting whole body images of patients while lying on the bed, such as X-ray machine, gamma ray machine and magnetic resonance imaging equipment.
[0262] Camera, used to collect the patient's whole body image and generate the original patient's whole body image for uploading;
[0263] An image processing module, used for performing organ and format conversion and image enhancement processing on the original patient's whole body image;
[0264] The image processing module includes:
[0265] The organ labeling unit is used to label the various organs in the original patient's whole body image.
[0266] A format conversion unit, used for successively performing format conversion and data normalization on an original patient whole body image, and normalizing the patient whole body image;
[0267] The image enhancement unit is used for performing color transformation, advanced transformation and combined enhancement on the normalized patient whole body image to obtain an enhanced patient whole body image.
[0268] Model building module, used to build precise positioning module;
[0269] The precise positioning model constructed includes:
[0270] 1) Input layer;
[0271] Input image format: The input layer accepts fixed-size images, which are normalized to the same size and resolution through preprocessing;
[0272] Input channel: Usually the input image contains three channels, namely width channel, height channel and color channel;
[0273] 2) Convolutional layer;
[0274] Feature extraction: The convolution layer extracts features through convolution kernels, extracts low-level edge and texture features, and then extracts high-level semantic features layer by layer;
[0275] Convolution kernel size: Use 3x3 or 5x5 convolution kernel;
[0276] Activation function: Use nonlinear activation function to increase the nonlinear representation ability of the network;
[0277] 3) Pooling layer;
[0278] Downsampling: The pooling layer downsamples the feature map through maximum pooling or average pooling operations to reduce the amount of calculation and retain important features;
[0279] Pooling window: Use a 2x2 pooling window to compress the spatial size of the feature map by gradually downsampling;
[0280] 4) Skip connection;
[0281] Residual connection: Add residual connections between specific layers, directly adding the output of the previous layer to the input of the next layer;
[0282] 5) Fully connected layer;
[0283] Feature mapping: Several fully connected layers map the features extracted by the convolutional layer to a specific output space to generate the predicted regions of each organ;
[0284] Output dimension: Determine the output dimension of the fully connected layer based on the number of bounding boxes that need to be predicted and the parameters of each bounding box;
[0285] 6) Output layer;
[0286] Bounding box prediction: The output layer predicts the specific location and range of each organ through multiple bounding box predictors. The bounding box is usually expressed in the form of (x, y, w, h), where (x, y) is the center coordinate and (w, h) is the width and height;
[0287] Confidence score: Each bounding box also outputs a confidence score, indicating the credibility of the prediction;
[0288] Category probability: Output the probability distribution of the category to which each bounding box belongs to determine the specific organ category.
[0289] The loss function module is used to design the loss function for the precise positioning module;
[0290] The loss function module includes:
[0291] A position loss calculation unit, used for calculating the position loss;
[0292] The position loss uses regression loss and uses the loss weights of L1 and L2 to calculate the error between the predicted bounding box and the true bounding box. The formula for the position loss is:
[0293] Position loss = ∑i|pi-gi|\text{Position loss
[0294] loss} = \sum_{i}\left|p_i-g_i\right|position loss = ∑i|pi-gi|; or,
[0295] Position loss = ∑i(pi-gi)2\text{Position loss
[0296] Loss} = \sum_{i}\left(p_i-g_i\right)^2 Position loss = ∑i(pi-gi)2, where pip_ipi is the predicted bounding box parameter and gig_igi is the true bounding box parameter;
[0297] Classification loss calculation unit, used to calculate classification loss;
[0298] Classification loss uses cross entropy loss: The error between the predicted category and the true category is calculated using cross entropy loss. Cross entropy loss is used to measure the difference between two probability distributions. The formula for classification loss is:
[0299]
[0300] Classification loss = -∑igilog(pi), where pip_ipi is the predicted category probability and gig_igi is the one-hot encoding of the true category;
[0301] A confidence calculation unit, used for calculating confidence;
[0302] Confidence loss uses mean square error: The error between the predicted confidence and the true confidence is calculated using mean square error, which is used to measure the square difference between the predicted value and the true value; the formula for confidence loss is:
[0303] Confidence loss = 1N∑i(pi-gi)2\text{Confidence loss
[0304] Confidence loss = N1∑i(pi-gi)2, where pip_ipi is the predicted confidence, gig_igi is the true confidence, and NNN is the number of samples;
[0305] The total loss calculation unit is used to calculate the total loss.
[0306] The total loss is a weighted total loss: comprehensive position loss, classification loss and confidence loss, weighted position loss, classification loss and confidence loss to form a total loss, which is used to guide the optimization of the model. The total loss can be a weighted combination of the losses of each part according to specific needs; the formula for the total loss is:
[0307] Total loss = α position loss + β classification loss + γ confidence loss\text{Total loss}
[0308] =\alpha\cdot\text{position loss}+\beta\cdot\text{classification loss}+\gamma\cdot\text{confidence loss}Total loss = α position loss + β classification loss + γ confidence loss, where α\alphaα, β\betaβ and γ\gammaγ are weighting coefficients, which adjust the weight of each part of the loss in the total loss;
[0309] A model training module is used to train the precise positioning model to obtain a precise positioning training model;
[0310] A data preprocessing unit, used for preprocessing images, including image normalization and data enhancement;
[0311] Image normalization: normalize the pixel values of the input image to the range of 0-1, standardize the image data, and ensure input consistency;
[0312] Data augmentation: During training, images are randomly cropped, rotated, flipped, and scaled to increase data diversity.
[0313] Batch training unit, used to perform batch training on images. Batch training includes small batch training and selecting a suitable batch size.
[0314] Small batch training: Divide the training data into several small batches, input each batch into the model for forward propagation and back propagation, and update the precise positioning model parameters;
[0315] Batch size: Choose an appropriate batch size to make efficient use of the GPU / CPU during training;
[0316] Optimization algorithm unit, used for optimization algorithms, including stochastic gradient descent and momentum method;
[0317] Stochastic Gradient Descent: Use the SGD optimization algorithm to minimize the loss function and optimize the parameters of the precision positioning model by gradually adjusting the weights and biases of the precision positioning model;
[0318] Momentum method: introduce momentum terms, combine the directions of previous gradients, smooth gradient updates, reduce oscillations, and accelerate convergence;
[0319] Adam optimizer: Use the Adam optimizer combined with momentum and adaptive learning rate adjustment;
[0320] Hyperparameter tuning unit, used for hyperparameter tuning, which includes selecting an appropriate learning rate, introducing regularization terms, and adjusting the batch size.
[0321] Learning rate: Choose an appropriate learning rate to accurately locate the model parameters and update them stably;
[0322] Regularization: Regularization terms are introduced to prevent overfitting of the precise positioning model and enhance the generalization ability of the precise positioning model;
[0323] Batch size: Adjust the batch size to balance training speed and stability so that the precision positioning model can be trained effectively.
[0324] The quantization technology module is used to perform static quantization, dynamic quantization and quantization-aware training QAT on the parameters of the precise positioning training model.
[0325] The quantitative technology modules include:
[0326] Static quantization unit, used for static quantization;
[0327] Static quantization: After the model training is completed, the model parameters are quantized;
[0328] Quantization step: convert floating-point weights and activation values into integer representations. Common methods include linear quantization and nonlinear quantization.
[0329] Quantization calibration: Use the calibration data set to calculate the quantization parameters to ensure that the accuracy of the quantized model is not significantly affected;
[0330] Dynamic quantization unit, used for dynamic quantization;
[0331] Dynamic quantization: Dynamically quantize model parameters during model inference;
[0332] Dynamic quantization step: During the inference process, floating-point parameters are dynamically converted to low-precision representation to reduce memory usage and computational complexity;
[0333] Applicable scenarios: Dynamic quantization is applicable to the model parts with large computational workload during the inference process, including fully connected layers and RNN layers;
[0334] Quantization-aware training unit, used for quantization-aware training;
[0335] Quantization-aware training (QAT): quantization operations are added during model training to adapt the model to the low-precision representation after quantization.
[0336] During training, the quantization operation is simulated to convert floating-point weights and activation values into low-precision representations, and the loss is calculated for backpropagation.
[0337] The model verification module is used to verify the precise positioning training model and obtain the precise positioning verification model;
[0338] A data set verification unit, used to verify the data set;
[0339] Independent validation set: Use a data set independent of the training set to evaluate the performance of the precise positioning training model, so that the precise positioning training model has good generalization ability on unseen data;
[0340] Verification data preparation: Perform the same preprocessing and normalization operations on the verification data as the training data to ensure the consistency of the input data;
[0341] An indicator evaluation unit, used to evaluate the indicator;
[0342] Accuracy: measures the accuracy of the prediction results of the precision positioning training model and calculates the proportion of correct predictions;
[0343] Recall rate: measures the recall ability of the precision positioning training model for real positive samples and calculates the proportion of correctly predicted positive samples;
[0344] F1 score: A comprehensive indicator that combines precision and recall to measure the overall performance of the precision positioning training model;
[0345] IOU intersection-over-union ratio: Calculates the ratio of the overlapping area to the joint area between the predicted bounding box and the true bounding box to measure the accuracy of the bounding box prediction;
[0346] Early stopping strategy unit, used for strategic early stopping.
[0347] Validation loss monitoring: monitor the change in loss on the validation set. If the validation loss no longer decreases after several rounds of training, terminate the training early to prevent overfitting of the precise positioning training model.
[0348] Model saving: Save the model parameters when the validation loss is the lowest to ensure the best performance of the final accurate positioning training model;
[0349] Model optimization module, used to optimize the precise positioning training module and obtain the precise positioning optimization model;
[0350] A weight pruning unit, used to prune weights;
[0351] Weight pruning: Prune weights according to their importance and remove weights of low importance;
[0352] Weight importance evaluation: Calculate the absolute value or square value of each weight to measure the impact of the absolute value or square value on the model output;
[0353] Pruning strategy: set a pruning threshold, reset weights below the threshold to zero, and retain weights of high importance;
[0354] Fine-tuning training: After pruning, fine-tune the model to restore model performance and ensure that pruning has minimal impact on model accuracy;
[0355] The filter pruning unit prunes the filters of the convolutional layer.
[0356] Filter pruning: Prune the filters of the convolution layer according to the importance of the convolution kernel to reduce the computational complexity of the convolution operation;
[0357] Filter importance evaluation: Calculate the L1 norm or L2 norm of each filter to measure the contribution of L1 norm or L2 norm to feature extraction;
[0358] Pruning strategy: set the pruning ratio, remove filters with smaller contributions, and retain filters with main features;
[0359] Fine-tuning training: After pruning, fine-tune the model to restore model performance and ensure that pruning has minimal impact on feature extraction capabilities;
[0360] The model precise positioning module is used to accurately locate the patient's organs based on the precise positioning optimization model.
[0361] Perform preprocessing based on the acquired images;
[0362] Gaussian filtering and mean filtering are used to denoise the image. Gaussian filtering removes random noise in the image through convolution operations while retaining the edge and detail information of the image.
[0363] Enhance the contrast and brightness of the image through histogram equalization and contrast stretching. Histogram equalization adjusts the grayscale distribution of the image to make the contrast of the image more uniform; contrast stretching further improves the contrast of the image by stretching the grayscale range of the image.
[0364] Extract features from images;
[0365] The SIFT algorithm is used to detect and describe local feature points in the image. SIFT features are scale-invariant and rotation-invariant, and can remain stable under different viewing angles and lighting conditions. SIFT detects key points through Gaussian difference (DoG) and generates feature vectors through main direction assignment and local feature description.
[0366] Adopt FLANN (Fast Library for Approximate Nearest Neighbors): FLANN is a fast approximate nearest neighbor search algorithm that can efficiently find matching point pairs in large-scale data sets. FLANN quickly finds approximate nearest neighbors by constructing multiple random trees and performing recursive searches.
[0367] Estimating the transformation of the image;
[0368] Use perspective transformation: According to the estimated homography matrix, the image is perspective transformed and projected onto the same plane to ensure image alignment and stitching.
[0369] Fusion of images;
[0370] The perspective transformed images are stitched together and the overlapping areas are weighted averaged to reduce stitching gaps and artifacts. The weighted average process generates a smooth transition effect by weighted summing the pixel values in the overlapping areas.
[0371] Perform color adjustment on the stitched images to ensure the consistency of hue and brightness and improve the visual effect of the image. Color adjustment adjusts the grayscale distribution and color value of the image through histogram matching and color balance technology.
[0372] Optimize the image results;
[0373] Global optimization is used to further optimize the stitching results and reduce the cumulative error through global optimization algorithms such as gradient descent and simulated annealing. Global optimization minimizes the overall error of the stitched image, adjusts the transformation parameters and fusion methods, and ensures the accuracy and stability of the stitching results.
[0374] The stitching gap removal is adopted, and the image patching technology is used to remove the stitching gap to ensure the integrity and naturalness of the stitching image. The image patching technology includes texture synthesis and image patching algorithms, which generate a continuous image effect by filling the pixel values of the gap area.
[0375] Ultimately, the patient's organs can be accurately located.
[0376] An electronic device is provided in an embodiment of the present application, and the electronic device includes: a processor and a memory. The processor and the memory are connected, such as through a bus. Optionally, the electronic device may also include a transceiver. It should be noted that in actual applications, the transceiver is not limited to one, and the structure of the electronic device does not constitute a limitation on the embodiment of the present application.
[0377] The processor may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.
[0378] The bus may include a path to transmit information between the above components. The bus may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc.
[0379] The memory can be ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or it can be EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to this.
[0380] The memory is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor. The processor is used to execute the application code stored in the memory to implement the content shown in the embodiment of the above-mentioned precise positioning method based on gesture recognition.
[0381] The electronic devices include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and fixed terminals such as digital TVs, desktop computers, etc. It can also be a server, etc.
[0382] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer can execute the corresponding content in the aforementioned embodiment of a precise positioning method based on gesture recognition.
[0383] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A precise positioning method based on posture recognition, It is characterized in that The following steps are involved: S1. Equipment installation: Set up equally spaced marking points on both sides of the bed, and set up a camera directly above the bed so that the camera can capture the area directly below and diagonally below the bed; S2. Image acquisition and processing: Image acquisition, moving the bed at a constant speed, so that the camera acquires a full-body image of the patient, and generates an original full-body image of the patient, wherein the original full-body image of the patient includes an original full-body image of the patient in a supine position and an original full-body image of the patient in a prone position; Image Processing: Organ labeling: labeling each organ in the original patient's whole body image; Format conversion, performing format conversion and data normalization on the original patient whole body image, and normalizing the patient whole body image; Image enhancement, performing color transformation, advanced transformation and combined enhancement on the normalized patient whole body image to obtain an enhanced patient whole body image; S3, model construction: build an accurate positioning model based on the enhanced patient's whole body image; Precision positioning models include: 1) Input layer; Input image format: The input layer accepts fixed-size images, which are normalized to the same size and resolution through preprocessing; Input channel: Usually the input image contains three channels, namely width channel, height channel and color channel; 2) Convolutional layer; Feature extraction: The convolution layer extracts features through convolution kernels, extracts low-level edge and texture features, and then extracts high-level semantic features layer by layer; Convolution kernel size: Use 3x3 or 5x5 convolution kernel; Activation function: Use nonlinear activation function to increase the nonlinear representation ability of the network; 3) Pooling layer; Downsampling: The pooling layer downsamples the feature map through maximum pooling or average pooling operations to reduce the amount of calculation and retain important features; Pooling window: Use a 2x2 pooling window to compress the spatial size of the feature map by gradually downsampling; 4) Skip connection; Residual connection: Add residual connections between specific layers, directly adding the output of the previous layer to the input of the next layer; 5) Fully connected layer; Feature mapping: Several fully connected layers map the features extracted by the convolutional layer to a specific output space to generate the predicted regions of each organ; Output dimension: Determine the output dimension of the fully connected layer based on the number of bounding boxes that need to be predicted and the parameters of each bounding box; 6) Output layer; Bounding box prediction: The output layer predicts the specific location and range of each organ through multiple bounding box predictors. The bounding box is usually expressed in the form of (x, y, w, h), where (x, y) is the center coordinate and (w, h) is the width and height; Confidence score: Each bounding box also outputs a confidence score, indicating the credibility of the prediction; Category probability: Output the probability distribution of the category to which each bounding box belongs and determine the specific organ category; S4. Design a loss function for the precise positioning model; The loss functions include: 1) Position loss; Regression loss: Use the L1 and L2 loss weights to calculate the error between the predicted bounding box and the true bounding box, where the position loss formula is: Position loss = ∑i|pi-gi|\text{Position loss loss}=\sum_{i}\left|p_i-g_i\right|position loss=∑i|pi-gi|; or, Position loss = ∑i(pi-gi)2\text{Position loss Loss} = \sum_{i}\left(p_i-g_i\right)^2 Position loss = ∑i(pi-gi)2, where pip_ipi is the predicted bounding box parameter and gig_igi is the true bounding box parameter; 2) Classification loss; Cross entropy loss: The error between the predicted category and the true category is calculated using cross entropy loss, which is used to measure the difference between two probability distributions; the formula for classification loss is: Classification loss = -∑igilog(pi), where pip_ipi is the predicted category probability and gig_igi is the one-hot encoding of the true category; 3) Confidence loss; Mean square error: The error between the predicted confidence and the true confidence is calculated using the mean square error, which is used to measure the square difference between the predicted value and the true value; the formula for confidence loss is: Confidence loss = 1N∑i(pi-gi)2\text{Confidence loss Confidence loss = N1∑i(pi-gi)2, where pip_ipi is the predicted confidence, gig_igi is the true confidence, and NNN is the number of samples; 4) Total loss; Weighted total loss: Comprehensive position loss, classification loss and confidence loss, weight the position loss, classification loss and confidence loss to form a total loss, which is used to guide the optimization of the model. The total loss can be a weighted combination of the losses of each part according to specific needs; the formula for the total loss is: Total loss = α position loss + β classification loss + γ confidence loss\text{Total loss} =\alpha\cdot\text{position loss}+\beta\cdot\text{classification loss} Total loss = α position loss + β classification loss + γ confidence loss, where α, β and γ are weighting coefficients to adjust the weight of each part of the loss in the total loss; S5. Train the precise positioning model to obtain a precise positioning training model; Training the precise positioning model consists of the following steps: 1) Data preprocessing; Image normalization: normalize the pixel values of the input image to the range of 0-1, standardize the image data, and ensure input consistency; Data augmentation: During training, images are randomly cropped, rotated, flipped, and scaled to increase data diversity. 2) Batch training; Small batch training: Divide the training data into several small batches, input each batch into the model for forward propagation and back propagation, and update the precise positioning model parameters; Batch size: Choose an appropriate batch size to make efficient use of the GPU / CPU during training; 3) Optimization algorithm; Stochastic Gradient Descent: Use the SGD optimization algorithm to minimize the loss function and optimize the parameters of the precision positioning model by gradually adjusting the weights and biases of the precision positioning model; Momentum method: introduce momentum terms, combine the directions of previous gradients, smooth gradient updates, reduce oscillations, and accelerate convergence; Adam optimizer: Use the Adam optimizer combined with momentum and adaptive learning rate adjustment; 4) Hyperparameter adjustment; Learning rate: Choose an appropriate learning rate to accurately locate the model parameters and update them stably; Regularization: Regularization terms are introduced to prevent overfitting of the precise positioning model and enhance the generalization ability of the precise positioning model; Batch size: Adjust the batch size to balance training speed and stability so that the precision positioning model can be trained effectively; Get the accurate positioning training model; S6. Verify the precise positioning training model. Verifying the precise positioning training model includes the following steps: 1) Validation dataset; Independent validation set: Use a data set independent of the training set to evaluate the performance of the precise positioning training model, so that the precise positioning training model has good generalization ability on unseen data; Verification data preparation: Perform the same preprocessing and normalization operations on the verification data as the training data to ensure the consistency of the input data; 2) Index evaluation; Accuracy: measures the accuracy of the prediction results of the precision positioning training model and calculates the proportion of correct predictions; Recall rate: measures the recall ability of the precision positioning training model for real positive samples and calculates the proportion of correctly predicted positive samples; F1 score: A comprehensive indicator that combines precision and recall to measure the overall performance of the precision positioning training model; IOU intersection-over-union ratio: Calculates the ratio of the overlapping area to the joint area between the predicted bounding box and the true bounding box to measure the accuracy of the bounding box prediction; 3) Early stopping strategy; Validation loss monitoring: monitor the change in loss on the validation set. If the validation loss no longer decreases after several rounds of training, terminate the training early to prevent overfitting of the precise positioning training model. Model saving: Save the model parameters when the validation loss is the lowest to ensure the best performance of the final accurate positioning training model; S7, optimizing the precise positioning training model, pruning the precise positioning training model, and obtaining a precise positioning optimization model; Optimizing the precise positioning training model includes the following steps: 1) Weight pruning: Prune the weights according to their importance and remove the weights with low importance; Weight importance evaluation: Calculate the absolute value or square value of each weight to measure the impact of the absolute value or square value on the model output; Pruning strategy: set a pruning threshold, reset weights below the threshold to zero, and retain weights of high importance; Fine-tuning training: After pruning, fine-tune the model to restore model performance and ensure that pruning has minimal impact on model accuracy; 2) Filter pruning: According to the importance of the convolution kernel, the filter of the convolution layer is pruned to reduce the computational complexity of the convolution operation; Filter importance evaluation: Calculate the L1 norm or L2 norm of each filter to measure the contribution of L1 norm or L2 norm to feature extraction; Pruning strategy: set the pruning ratio, remove filters with smaller contributions, and retain filters with main features; Fine-tuning training: After pruning, fine-tune the model to restore model performance and ensure that pruning has minimal impact on feature extraction capabilities; S8. Use the precise positioning optimization model to process and analyze the collected original patient whole body images to accurately locate the patient's organs.
2. According to claim 1, a precise positioning method based on posture recognition, It is characterized in that The training of the precise positioning model further includes adopting a quantization technique, and the quantization technique includes the following steps: 1) Static quantization: After the model training is completed, the model parameters are quantized; Quantization step: convert floating-point weights and activation values into integer representations. Common methods include linear quantization and nonlinear quantization. Quantization calibration: Use the calibration data set to calculate the quantization parameters to ensure that the accuracy of the quantized model is not significantly affected; 2) Dynamic quantization: During the model inference process, the model parameters are dynamically quantized; Dynamic quantization step: During the inference process, floating-point parameters are dynamically converted to low-precision representation to reduce memory usage and computational complexity; Applicable scenarios: Dynamic quantization is applicable to the model parts with large computational workload during the inference process, including fully connected layers and RNN layers; 3) Quantization-aware training (QAT): quantization operations are added during model training to adapt the model to the low-precision representation after quantization. During training, the quantization operation is simulated to convert floating-point weights and activation values into low-precision representations, and the loss is calculated for backpropagation.
3. A precise positioning system based on gesture recognition, characterized in that it is applied to the precise positioning method based on gesture recognition as claimed in claim 1 or 2, include: Equipment with scanning function and movable bed, for patients lying on the bed to collect full-body images of the patients; Camera, used to collect the patient's whole body image and generate the original patient's whole body image for uploading; An image processing module, used for performing organ and format conversion and image enhancement processing on the original patient's whole body image; Model building module, used to build precise positioning module; The loss function module is used to design the loss function for the precise positioning module; A model training module is used to train the precise positioning model to obtain a precise positioning training model; The model verification module is used to verify the precise positioning training model and obtain the precise positioning verification model; Model optimization module, used to optimize the precise positioning training module and obtain the precise positioning optimization model; The model precise positioning module is used to accurately locate the patient's organs based on the precise positioning optimization model.
4. According to claim 3, a precise positioning system based on gesture recognition, It is characterized in that The image processing module comprises: The organ labeling unit is used to label the various organs of the original patient's whole body image. A format conversion unit, used for successively performing format conversion and data normalization on an original patient whole body image, and normalizing the patient whole body image; The image enhancement unit is used for performing color transformation, advanced transformation and combined enhancement on the normalized patient whole body image to obtain an enhanced patient whole body image.
5. According to claim 3, a precise positioning system based on gesture recognition, It is characterized in that The loss function module includes: A position loss calculation unit, used for calculating the position loss; Classification loss calculation unit, used to calculate classification loss; A confidence calculation unit, used for calculating confidence; The total loss calculation unit is used to calculate the total loss.
6. According to claim 3, a precise positioning system based on gesture recognition, It is characterized in that The model training module includes: A data preprocessing unit, used for preprocessing images, including image normalization and data enhancement; Batch training unit, used to perform batch training on images. Batch training includes small batch training and selecting a suitable batch size. Optimization algorithm unit, used for optimization algorithms, including stochastic gradient descent and momentum method; Hyperparameter tuning unit, used for hyperparameter tuning, which includes selecting an appropriate learning rate, introducing regularization terms, and adjusting the batch size.
7. According to claim 3, a precise positioning system based on gesture recognition, It is characterized in that The model verification module includes: A data set verification unit, used to verify the data set; An indicator evaluation unit, used to evaluate the indicator; Early stopping strategy unit, used for strategic early stopping.
8. According to claim 3, a precise positioning system based on gesture recognition, It is characterized in that The model optimization module includes: A weight pruning unit, used to prune weights; The filter pruning unit prunes the filters of the convolutional layer.
9. According to claim 3, a precise positioning system based on gesture recognition, It is characterized in that It also includes a quantization technology module, which is used to perform static quantization, dynamic quantization and quantization-aware training QAT on the parameters of the precise positioning training model.
Citation Information
Cited By
Instrument tracking method for surgical navigation
CN119523628A
