AI assistance-based parathyroid gland identification method and device

Abstract

The invention discloses a parathyroid gland identification method and device based on AI assistance. The parathyroid gland identification method comprises the following steps: S1, constructing a training data set of an important anatomical structure; s2, preprocessing the training data set of the important anatomical structure; s3, processing a plurality of segmentation models based on the preprocessed important anatomical structure training data set, and outputting a pixel probability graph of each segmentation model; s4, pixel values in each pixel probability graph are multiplied by corresponding weights, and a summation result is calculated; and S5, based on the addition result, performing post-processing and generating a parathyroid gland segmentation map. According to the method, a comprehensive probability graph containing advantages of different models is obtained based on processing of a plurality of segmentation models and weighted fusion processing. Compared with a traditional detection method, the accuracy and flexibility of recognition are remarkably improved. The method model is integrated into the device, and real-time positioning and recognition of important anatomical structures in an operation are achieved.

Description

Technical Field

[0001] The present invention relates to the field of medical detection, and in particular to an AI-assisted parathyroid gland identification method and device. Background Art

[0002] In thyroid cancer surgery, permanent hypoparathyroidism caused by total thyroidectomy is a serious complication that can cause postoperative hypocalcemia, tetany, and affect normal life. Therefore, it is generally believed in the industry that the preservation of the parathyroid glands is of great significance to the patient's quality of life after surgery.

[0003] At present, there are several mainstream methods for locating and identifying parathyroid glands, but each of them has certain limitations: intraoperative identification by the doctor's naked eye is highly subjective, has limited accuracy and is prone to misjudgment; radionuclide imaging technology requires special equipment, has false positive and false negative results, is easily interfered by thyroid lesions, and has poor imaging of small or non-hyperfunctioning parathyroid glands; ultrasound examination relies on the operator's experience, and it is difficult to display normal-sized parathyroid glands and is greatly affected by surrounding structures; rapid intraoperative parathyroid hormone testing indirectly reflects the situation, is affected by multiple factors and has time delays; near-infrared fluorescence imaging technology has poor fluorescent dye specificity, high cost and complex operation. Summary of the invention

[0004] In view of this, the present invention discloses an AI-assisted parathyroid identification method, which can accurately identify important anatomical structures during surgery, such as parathyroid glands, recurrent laryngeal nerves, internal jugular veins, etc., and integrate this identification method into a device to achieve real-time positioning and identification of important anatomical structures during surgery, so as to improve the parathyroid retention rate.

[0005] The technical solution of the present invention is a parathyroid gland identification method based on AI assistance, comprising the following steps: S1: Construct a training dataset of important anatomical structures; S2: preprocessing the training data set of the important anatomical structure; S3: Based on the preprocessed important anatomical structure training data set, multiple segmentation model processing is performed to output a pixel probability map of each segmentation model, where the probability value of each pixel point represents the probability size of the pixel belonging to the parathyroid gland; S4: Calculate the weight of each segmentation model based on the performance evaluation index, multiply the pixel value in each pixel probability map by the corresponding weight, and calculate the sum of all pixel probability maps after the multiplication; S5: Based on the summing result, post-processing is performed to generate a parathyroid segmentation map.

[0006] According to a preferred embodiment, the training data set of the important anatomical structures in S1 includes a data set of the parathyroid gland, the recurrent laryngeal nerve, and the internal jugular vein.

[0007] Furthermore, the preprocessing in S2 includes standardization, denoising and enhancement processing.

[0008] According to a preferred embodiment, the segmentation model processing in S3 includes: Feature extraction, extracting feature information of the important anatomical structure through different filters; Category probability calculation, mapping the feature information to the category of the important anatomical structure through a fully connected layer, and calculating the category probability; The activation function is applied to normalize the data of the fully connected layer based on the activation function and calculate the probability value of each pixel.

[0009] According to a preferred embodiment, for the Bayesian image segmentation model, the probability value of each pixel is calculated by calculating the prior probability and the posterior probability, wherein the prior probability refers to the evaluation of the probability of occurrence of the parathyroid gland based on previous data or experience, and the posterior probability refers to the probability obtained based on the feature extraction.

[0010] Furthermore, the performance evaluation indicators in step S4 include accuracy, recall and F1 value.

[0011] According to a preferred embodiment, the post-processing in step S5 includes: Thresholding processing, setting a threshold N, converting the probability map into a binary map, wherein pixels with probability values ​​higher than the threshold N are determined to be parathyroid areas, and pixels with probability values ​​lower than the threshold N are determined to be non-parathyroid areas; Morphological operation, processing the binary image by dilation and erosion; Smoothing processing is to smooth the binary image after the morphological operation.

[0012] According to a preferred embodiment, the method further includes adjusting the learning rate, batch size and regularization parameters, using cross-validation on the training set and modifying the loss function to improve the recognition accuracy.

[0013] According to a preferred embodiment, the method further includes performing model evaluation based on image segmentation and target detection.

[0014] Correspondingly, in order to implement the above method, the present invention also proposes an AI-assisted parathyroid identification device, comprising an optical system, an imaging system and a processing system, wherein the optical system provides a light source for the body cavity into which the parathyroid identification device enters, and the imaging system converts the received light signal into an image signal and sends it to the processing system for identification of important anatomical structures. The processing system is used to pre-process the image signal and process the segmentation model, output the pixel probability map of each segmentation model, calculate the weight of each segmentation model based on the performance evaluation index, multiply the pixel value and the corresponding weight in each pixel probability map, and calculate the sum of all pixel probability maps after the multiplication process, and then perform post-processing based on the sum result to generate a parathyroid segmentation map. The present invention is based on AI technology and can learn the characteristics of the parathyroid glands from a large number of surgical images. Based on the processing of multiple segmentation models and weighted fusion processing, a comprehensive probability map that includes the advantages of different models is obtained. Compared with the traditional subjective naked eye recognition method, the accuracy of recognition is significantly improved. The method model is integrated into the device to achieve real-time positioning and recognition of important anatomical structures during surgery, without the need for other additional detection equipment and complex operating procedures, which significantly improves the safety and accuracy of thyroid cancer surgery and is flexible and convenient to use. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic diagram of the steps of an AI-assisted parathyroid gland identification method of the present invention; Figure 2 It is a schematic diagram of the composition of an AI-assisted parathyroid gland identification device of the present invention; Figure 3 It is a test result diagram of the present invention. DETAILED DESCRIPTION

[0016] In thyroid cancer surgery, permanent hypoparathyroidism caused by total thyroidectomy is a serious complication that can cause postoperative hypocalcemia, tetany, and affect normal life. Therefore, it is generally believed in the industry that the preservation of the parathyroid glands is of great significance to the patient's quality of life after surgery.

[0017] At present, there are several mainstream methods for locating and identifying parathyroid glands, but each of them has certain limitations: intraoperative identification by the doctor's naked eye is highly subjective, has limited accuracy and is prone to misjudgment; radionuclide imaging technology requires special equipment, has false positive and false negative results, is easily interfered by thyroid lesions, and has poor imaging of small or non-hyperfunctioning parathyroid glands; ultrasound examination relies on the operator's experience, and it is difficult to display normal-sized parathyroid glands and is greatly affected by surrounding structures; rapid intraoperative parathyroid hormone testing indirectly reflects the situation, is affected by multiple factors and has time delays; near-infrared fluorescence imaging technology has poor fluorescent dye specificity, high cost and complex operation.

[0018] In view of this, the present invention discloses an AI-assisted parathyroid identification method, which can accurately identify important anatomical structures during surgery, such as parathyroid glands, recurrent laryngeal nerves, internal jugular veins, etc., and integrate this identification method into a device to achieve real-time positioning and identification of important anatomical structures during surgery, so as to improve the parathyroid retention rate.

[0019] like Figure 1 As shown, the technical solution of the present invention is a parathyroid gland identification method based on AI assistance, comprising the following steps: S1: Construct a training dataset of important anatomical structures; S2: preprocessing the training data set of the important anatomical structure; S3: Based on the preprocessed important anatomical structure training data set, multiple segmentation model processing is performed to output a pixel probability map of each segmentation model, where the probability value of each pixel point represents the probability size of the pixel belonging to the parathyroid gland; S4: Calculate the weight of each segmentation model based on the performance evaluation index, multiply the pixel value in each pixel probability map by the corresponding weight, and calculate the sum of all pixel probability maps after the multiplication; S5: Based on the summed result, post-processing is performed to generate a visual para-gland segmentation map.

[0020] The present invention is based on AI technology and can learn the characteristics of the parathyroid glands from a large number of surgical images. Based on the processing of multiple segmentation models and weighted fusion processing, a comprehensive probability map that includes the advantages of different models is obtained. Compared with the traditional subjective naked eye recognition method, the accuracy of recognition is significantly improved. The method model is integrated into the device to achieve real-time positioning and recognition of important anatomical structures during surgery, without the need for other additional detection equipment and complex operating procedures, which significantly improves the safety and accuracy of thyroid cancer surgery and is flexible and convenient to use.

[0021] The following is a detailed description of the above steps: First of all, it should be pointed out that the present invention does not limit the selection of specific models. However, according to a preferred embodiment, the U-Net architecture can be used, which uses skip connections and upsampling layers to maintain the spatial hierarchy of the image, thereby accurately locating the anatomical structure and having high effectiveness and accuracy in the field of medical image segmentation. In addition, models such as YoloV8, ResNet, ConvNeXt, MAE, PoolFormer, SegNeXt, etc. can also be used for comparison and comprehensive evaluation.

[0022] In step S1, a large number of high-quality endoscopic and robotic thyroid surgery videos can be collected, video clips showing important anatomical structures can be captured, the video clips can be converted into images, and manually labeled to construct a data set for AI model training and testing. Among them, 80% of the image data can be used as a training set, and the remaining 20% ​​of the image data can be used as a test set.

[0023] The training data set of important anatomical structures in S1 includes data sets of parathyroid glands, recurrent laryngeal nerves, and internal jugular veins. Damage to the parathyroid glands may lead to permanent hypoparathyroidism, damage to the recurrent laryngeal nerves may lead to hoarseness, and damage to the internal jugular veins may lead to serious complications such as bleeding. The recurrent laryngeal nerves and internal jugular veins are easily accessible during resection surgery, so their accurate identification is equally important.

[0024] Furthermore, the preprocessing in S2 includes standardization, denoising and enhancement processing.

[0025] Among them, the standardization operation includes image size unification: since surgical images may come from different devices or be captured under different conditions, their sizes and resolutions may be different. In order for the model to process these images consistently, all images need to be resized to a uniform size, such as resizing all images to 256x256 pixels. For the same reason, color space conversion is also included, converting all images to a uniform color space, and normalizing the pixel values ​​of the image, scaling the pixel values ​​to between 0 and 1, to reduce numerical instability during model training.

[0026] Among them, denoising can adopt Gaussian filtering, median filtering or non-local mean denoising to reduce the noise in the image and improve the image quality.

[0027] Among them, the enhancement processing increases the diversity of parathyroid image data through various operation methods such as rotation, flipping, and scaling.

[0028] Furthermore, the segmentation model processing in S3 includes: Feature extraction: Each segmentation model slides over the preprocessed parathyroid image data through a series of convolutional layers. Different convolution kernels act as different filters to capture local patterns such as lines and edges in different directions and scales in the image, as well as parathyroid-specific texture information. As the number of convolutional layers increases, the model can extract more abstract and representative high-level features, laying the foundation for subsequent judgment of whether a pixel belongs to a parathyroid gland.

[0029] Category probability calculation, after feature extraction, the fully connected layer in the model maps these extracted features to a specific number of categories, such as the parathyroid gland, recurrent laryngeal nerve, and internal jugular vein mentioned above. Each category has a predicted probability, which is calculated based on the feature information learned by the model from the image, indicating the probability that the pixel belongs to that category.

[0030] Application of activation function: In the output layer of the model, an activation function such as softmax is used to convert the output into a probability distribution. The Softmax function normalizes the raw scores output by the fully connected layer so that the sum of the probabilities of all categories is 1. In this way, each pixel has a clear probability value for each category, which clearly indicates the possibility that the pixel belongs to a specific category, providing a standardized probability representation for subsequent probability map generation and processing.

[0031] According to a preferred embodiment, for some specific models, such as the Bayesian image segmentation model, the probability value of each pixel is calculated by calculating the prior probability and the posterior probability, wherein the prior probability refers to the evaluation of the probability of the parathyroid gland based on previous data or experience, and the posterior probability refers to the probability obtained based on the feature extraction. The prior probability and the posterior probability are combined through the Bayesian formula to more accurately determine the final category of the pixel, further improving the accuracy of segmentation.

[0032] After each segmentation model processes the data, it outputs the corresponding probability map. These probability maps are the model's quantitative representation of the possibility that each pixel in the parathyroid image belongs to the parathyroid gland. The value of each pixel represents the probability that the pixel belongs to the parathyroid gland, providing a basis for subsequent fusion processing.

[0033] Furthermore, after the segmentation model processing is completed, the weight of each segmentation model is calculated based on the performance evaluation index, and these weights reflect the relative importance of each model in the entire segmentation process, ensuring that in the fusion process, the model with better performance contributes more to the final result; The pixel value and the corresponding weight in each pixel probability map are multiplied, and the sum of the pixel values ​​of all pixel probability maps after the multiplication is output. In this way, the output of each segmentation model is comprehensively considered to obtain a comprehensive probability map that integrates the advantages of multiple models, thereby improving the accuracy and reliability of parathyroid identification.

[0034] Among them, the performance evaluation indicators include accuracy, recall and F1 value. Accuracy indicates the proportion of samples correctly classified by the model to the total samples, accuracy = (true positive + true negative) / (true positive + false positive + true negative + false negative). Recall indicates the proportion of positive samples correctly identified by the model to all actual positive samples, recall = true positive / (true positive + false negative). F1 value is the harmonic mean of accuracy and recall, which comprehensively considers the balance between accuracy and recall and is a more comprehensive performance evaluation indicator. F1 value = 2 * (accuracy * recall) / (accuracy + recall).

[0035] By using accuracy, recall, and F1 value as performance evaluation indicators, the AI-assisted parathyroid identification method can be comprehensively evaluated and optimized to ensure that the application of the model in actual surgery can achieve the best performance balance.

[0036] Furthermore, the post-processing in step S5 includes: Thresholding: Thresholding is performed on the probability map obtained by weighted addition. By setting a suitable threshold N, the probability map is converted into a binary map. Pixels above the threshold N are judged to belong to the parathyroid area, and pixels below the threshold N are judged to belong to the non-parathyroid area, thereby clearly distinguishing the parathyroid glands from the surrounding tissues.

[0037] Morphological operations: Use morphological operations such as dilation and erosion to process the binary image. The dilation operation can fill the small holes that may exist in the parathyroid region to make the parathyroid region more complete; the erosion operation can remove small noise points or discontinuous parts at the boundary of the parathyroid gland, further optimize the segmentation results, and improve the clarity and accuracy of the parathyroid boundary.

[0038] Smoothing: Smoothing the segmentation results after morphological operations. Through appropriate smoothing algorithms, the noise and discontinuity in the image are reduced, making the segmentation map of the parathyroid gland smoother and more natural, more in line with the actual anatomical structure, and improving the quality of the final segmentation results.

[0039] After post-processing, the final parathyroid gland segmentation map is obtained. This map clearly shows the location and outline of the parathyroid glands in the image, providing key visual guidance for the identification and protection of the parathyroid glands during thyroid surgery, helping to improve the accuracy and safety of the surgery.

[0040] Furthermore, the AI-assisted parathyroid identification method also includes improving the recognition accuracy by adjusting the learning rate, batch size and regularization parameters, using cross-validation on the training set and modifying the loss function.

[0041] Model evaluation is performed based on image segmentation and target detection. According to a preferred implementation, two evaluation indicators commonly used in image segmentation tasks are used: average Dice coefficient (Dice) and average intersection over union (IoU); four evaluation indicators widely used in the field of target detection are used, namely S-measure (Sα), F-measure (Fwβ), Emean and precision-recall curve. These indicators are selected to comprehensively evaluate the model from the perspectives of image segmentation and target detection.

[0042] Among them, the Dice coefficient is an indicator to measure the similarity between two samples, which is suitable for image segmentation tasks in binary classification problems. In parathyroid identification, the Dice coefficient can help evaluate the accuracy of the model's segmentation of the parathyroid region; the intersection over union (IoU) is another indicator to measure the similarity between two samples. It evaluates the accuracy of segmentation by calculating the ratio of the intersection to the union of two samples to provide an intuitive metric for evaluating the degree of overlap between the parathyroid region predicted by the model and the true annotation; S-measure is an indicator to measure the performance of target detection, which comprehensively considers the precision and recall rate and weights them to reflect the importance of different categories; F-measure is the harmonic mean of precision and recall rate, providing a balanced measure; Emean can help evaluate the accuracy of the model in detecting parathyroid glands, especially when considering position errors; the precision-recall curve is a graphical evaluation tool that shows the relationship between the precision and recall rate of the model at different thresholds.

[0043] By using these evaluation indicators, the performance of the AI-assisted parathyroid gland recognition method can be comprehensively evaluated to ensure the effectiveness and reliability of the model in practical applications. This helps to optimize model parameters, improve recognition accuracy, and provide more accurate parathyroid gland recognition results in actual surgery.

[0044] Accordingly, to implement the above method, Figure 2 As shown, the present invention also discloses an AI-assisted parathyroid identification device, comprising an optical system, an imaging system and a processing system, wherein the optical system provides a light source for the body cavity into which the parathyroid identification device enters, and the imaging system converts the received light signal into an image signal and sends it to the processing system for identification of important anatomical structures. Among them, the processing system is used to preprocess and segmentation model process the image signal, output the pixel probability map of each segmentation model, and calculate the weight of each segmentation model based on the performance evaluation index, multiply the pixel value and corresponding weight in each pixel probability map, and output the sum of the pixel values ​​of all pixel probability maps after the multiplication, and then post-process based on the sum result to generate a parathyroid segmentation map.

[0045] According to a preferred embodiment, the optical system and the imaging system may be implemented by, for example, an endoscope or other devices capable of detecting images, connected to a processing system and a display device, to provide a reference for the physician during the operation.

[0046] like Figure 3 The "Precision-Recall Curve" is shown as the precision-recall curve (the vertical axis "Precision" represents the precision, and the horizontal axis "Recall" represents the recall rate). In a test result, "all classes 0.954 mAP@0.5" means: the average precision for the parathyroid gland reaches 0.954, which means that the model can identify and locate the parathyroid gland in the image with extremely high accuracy when the Mask_IoU threshold is 0.5.

[0047] The above is a detailed introduction to an AI-assisted parathyroid identification method and device provided by an embodiment of the present invention. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can be referenced to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

[0048] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0049] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

Claims

1. A parathyroid gland identification method based on AI assistance, characterized in that: The following steps are involved: S1: Construct a training dataset of important anatomical structures; S2: preprocessing the training data set of the important anatomical structure; S3: Based on the preprocessed training data set of important anatomical structures, multiple segmentation models are processed to output a pixel probability map of each segmentation model, where the probability value of each pixel represents the probability that the pixel belongs to the parathyroid gland; S4: Calculate the weight of each segmentation model based on the performance evaluation index, multiply the pixel value in each pixel probability map by the corresponding weight, and calculate the sum of all pixel probability maps after the multiplication; S5: Based on the summing result, post-processing is performed to generate a parathyroid segmentation map.

2. The AI-assisted parathyroid gland identification method according to claim 1, characterized in that: The training data sets of important anatomical structures in S1 include parathyroid gland, recurrent laryngeal nerve, and internal jugular vein data sets.

3. The AI-assisted parathyroid gland identification method according to claim 1, characterized in that: The preprocessing in S2 includes normalization, denoising and enhancement.

4. The AI-assisted parathyroid gland identification method according to claim 1, characterized in that: The segmentation model processing in S3 includes: Feature extraction, extracting feature information of the important anatomical structure through different filters; Category probability calculation, mapping the feature information to the category of the important anatomical structure through a fully connected layer, and calculating the category probability; The activation function is applied to normalize the data of the fully connected layer based on the activation function and calculate the probability value of each pixel.

5. The AI-assisted parathyroid gland identification method according to claim 4, characterized in that: For the Bayesian image segmentation model, the probability value of each pixel is calculated by calculating the prior probability and the posterior probability, wherein the prior probability refers to the evaluation of the probability of the occurrence of the parathyroid gland based on previous data or experience, and the posterior probability refers to the probability obtained based on the feature extraction.

6. The AI-assisted parathyroid gland identification method according to claim 1, characterized in that: The performance evaluation indicators in step S4 include accuracy, recall and F1 value.

7. The AI-assisted parathyroid gland identification method according to claim 1, characterized in that: The post-processing in step S5 includes: Thresholding processing, setting a threshold N, converting the probability map into a binary map, wherein pixels with probability values ​​higher than the threshold N are determined to be parathyroid areas, and pixels with probability values ​​lower than the threshold N are determined to be non-parathyroid areas; Morphological operation, processing the binary image by dilation and erosion; Smoothing processing is to smooth the binary image after the morphological operation.

8. The AI-assisted parathyroid gland identification method according to claim 1, characterized in that: It also includes using cross-validation on the training set and modifying the loss function by adjusting the learning rate, batch size, and regularization parameters.

9. The AI-assisted parathyroid gland identification method according to claim 1, characterized in that: It also includes model evaluation based on image segmentation and object detection.

10. An AI-assisted parathyroid gland identification device, characterized in that: The device comprises an optical system, an imaging system and a processing system, wherein the optical system provides a light source for the body cavity into which the parathyroid gland identification device enters, and the imaging system converts the received light signal into an image signal and sends it to the processing system for identification of important anatomical structures. Among them, the processing system is used to preprocess and segmentation model process the image signal, output the pixel probability map of each segmentation model, and calculate the weight of each segmentation model based on the performance evaluation index, multiply the pixel value and corresponding weight in each pixel probability map, and calculate the sum of all pixel probability maps after multiplication, and then post-process based on the sum result to generate a parathyroid segmentation map.

Images