Infrared thermal imaging foot region automatic segmentation method

Abstract

The invention provides an infrared thermal imaging foot region automatic segmentation method, which can be operated in two different segmentation modes to adapt to different operation and training conditions, and specifically comprises a supervised learning mode and an unsupervised zero sample segmentation mode. The supervisory mode includes an efficient lightweight image encoder and a dual-branch up-sampling decoder generated by split knowledge distillation training. According to the unsupervised mode, an image is divided into two parts through a clustering algorithm according to pixel intensity values, then the mass center of a high-pixel-intensity area is calculated, the mass center serves as a space prompt to be input into the SAM model, and the SAM model can utilize the prompts to focus on segmentation of a foot area and effectively ignore other areas. According to the method, the foot region in the infrared thermogram can be efficiently and accurately extracted in real time without any preprocessing, the inherent challenges caused by low contrast, high noise and visual discontinuity (malformation, partial cut-off and the like) of the foot boundary of the infrared image are solved, and meanwhile, the calculation burden is remarkably reduced.

Description

Technical Field

[0001] The invention belongs to the technical field of infrared imaging, and in particular relates to an automatic segmentation method for a foot area of ​​infrared thermal imaging. Background Art

[0002] Infrared thermal imaging technology can sensitively detect changes in body surface thermal radiation caused by foot microcirculation and nerve damage in diabetic patients, and convert them into visual and digital information. Even for diabetic patients with amputation, the temperature of the remaining limb can provide sufficient information to support early prevention and screening of diabetic foot.

[0003] Accurate and efficient image segmentation is the basis for risk assessment and diagnosis of diabetic foot using infrared thermal imaging technology. Advanced semantic segmentation methods can accurately identify and extract plantar areas, thereby significantly reducing environmental interference and improving analysis accuracy and data processing reliability. However, the low contrast and high noise levels of infrared images pose significant challenges to achieving accurate segmentation. In addition, the visual discontinuity of foot boundaries caused by deformities or partial amputations further affects the robustness of the algorithm. The dynamic changes in foot size and posture during continuous monitoring also increase the need for real-time adaptability.

[0004] CN113096811B proposes a diabetic foot image processing and risk warning device based on infrared thermal imaging, which discloses an infrared thermal imaging processing method, which uses an infrared image acquisition device and a color image acquisition device to respectively acquire the patient's foot infrared thermal image and visible light image. And the color image acquisition device is configured to have the same field of view and shooting angle as the infrared image acquisition device to generate a color image registered with the infrared image. Then, the color image is segmented to distinguish the foot image from the surrounding environment; and then the foot image segmented from the color image is registered with the infrared image to obtain an accurate infrared image of the sole of the foot. This method is complicated to operate, poses a potential threat to patient privacy, and is difficult to meet clinical needs.

[0005] CN114888870B proposes an insole cutting method for artificial intelligence diabetic foot ulcer identification based on thermal imaging, which discloses a computer image processing method, in which the original infrared thermal imaging images of diabetic foot ulcers are processed into an image data set, and medical staff diagnose and annotate the collected infrared thermal imaging images, organize the infrared thermal imaging images of foot ulcers of all patients, and the medical staff confirm the image quality and determine the ulcer area in the image; this method is greatly affected by the subjective influence of medical staff, and has poor robustness and generalization.

[0006] Traditional machine learning methods such as histogram-based methods, clustering algorithms, entropy methods, object attribute analysis, genetic algorithm Snakes, region growing, active contour methods, level sets, and graph cuts. When the temperature of the foot area is close to the surrounding environment during segmentation, the performance of this method decreases, and it involves overly complex parameter adjustments, which reduces its robustness and real-time performance.

[0007] With the remarkable success of CNN and ViT in the field of vision, many studies have attempted to develop deep learning-based DF segmentation methods. In the prior art, a UNet-based foot segmentation network PFSnet was proposed, which requires a two-stage adaptive gamma transform for data preprocessing, compromising real-time performance. In the prior art, a dual encoder-ResUnet (DE-ResUnet) architecture for foot image segmentation was proposed, which relies on the corresponding visible light images of infrared thermal images, which are not always available in clinical settings. In addition, the use of color thermal imagers instead of simple infrared imagers may increase the risk of patient privacy.

[0008] There are two problems with existing infrared thermal imaging segmentation methods: 1. Deep learning-based segmentation methods mostly use visible light and infrared for registration, which poses a potential threat to patient privacy and has high computational costs, high requirements for infrared equipment, or strict restrictions on shooting angles and postures that do not meet clinical application requirements. 2. Segmentation based on traditional methods is subject to the subjective influence of operators, has poor robustness, and is inefficient. Summary of the invention

[0009] In response to a series of current problems, the present invention proposes a new lightweight segmentation method, which includes a supervised learning mode and an unsupervised zero-sample segmentation mode. The supervised mode includes a lightweight encoder distilled from separation knowledge and a dual-branch upsampling decoder. Finally, dice-97.62% was achieved, the global accuracy was 0.99, the number of parameters was no more than 6.16 million, and the number of frames processed per second was 80. The unsupervised mode divides the image into two parts through a clustering algorithm, and then calculates the part with high pixel intensity (foot area) as the centroid, and uses the centroid as a spatial cue input to SAM. SAM uses these spatial cues to effectively segment the foot area and ignore other areas.

[0010] The present invention proposes an automatic segmentation method for foot area of ​​infrared thermal imaging, which specifically includes: a supervised learning mode and an unsupervised zero-sample segmentation mode; In the supervised learning mode, the method specifically includes the following steps: Step (1): Obtain infrared thermal imaging images of the soles of the feet; Step (2): input the infrared thermal imaging image in step (1) into the efficient and lightweight image encoder generated by the separable knowledge distillation training to generate a feature map; Step (3): Input the feature map into the dual-branch upsampling decoder; Step (4): output the segmented infrared thermal imaging image; In the unsupervised zero-sample segmentation mode, the method specifically includes the following steps: Step (1): Obtain infrared thermal imaging images of the soles of the feet; Step (2): Divide the infrared thermal imaging image in step (1) into two parts in the middle; Step (3): Each part is divided into background and foreground according to the pixel intensity value using a clustering algorithm; Step (4): Calculate the centroid coordinates of each part using the formula: ; in, represents half of the infrared image, Represents the centroid coordinates of the area with high pixel intensity values, represents the centroid of all identified clusters; Step (5): Using the centroid coordinates as a hint, call the SAM model to perform segmentation and accurately segment the foot area without any specific training for foot segmentation. The formula is as follows: ; in, is the segmentation mask, is the input infrared image, is the centroid coordinate, are the parameters of the SAM model.

[0011] The present invention has the following beneficial technical effects: The lightweight encoder is trained through a separate knowledge distillation process, which maintains the high-fidelity feature representation of the original SAM heavy encoding while significantly reducing the computational burden. The dual-branch upsampling encoder has the following advantages: The decoder architecture includes two complementary branches, a fine-grained branch for detail recovery and a coarse-grained branch for extensive feature extraction. By stacking this dual-branch module, the segmentation performance is effectively enhanced at multiple granularity levels. The unsupervised segmentation mode identifies salient areas near the foot in infrared images through unsupervised clustering techniques. These areas are then used as hints for zero-sample segmentation. This mode is particularly advantageous in the case of limited or no labeled data, facilitating rapid model deployment and adaptation. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1A schematic diagram of a flow chart of an automatic segmentation method for foot area using infrared thermal imaging according to the present invention; Figure 2 This is a flow chart of the supervised learning mode of the present invention; Figure 3 It is a schematic diagram of the encoder of the present invention; Figure 4 A schematic diagram of a decoder of the present invention; Figure 5 This is the flowchart for the unsupervised zero-shot segmentation mode. DETAILED DESCRIPTION

[0013] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in each embodiment of the present invention described below can be combined with each other as long as they do not conflict with each other. To achieve the above-mentioned purpose, the present invention adopts the following technical scheme.

[0014] The present invention proposes an automatic segmentation method for the foot area of ​​infrared thermal imaging, such as Figure 1 As shown, this method can be run in two different segmentation modes: supervised learning mode and unsupervised zero-shot segmentation mode.

[0015] In supervised learning mode, such as Figure 2 As shown, the method specifically comprises the following steps: Step (1): Obtain infrared thermal imaging images of the soles of the feet; Step (2): input the infrared thermal imaging image in step (1) into the efficient and lightweight image encoder generated by the separable knowledge distillation training to generate a feature map; Step (3): Input the feature map into the dual-branch upsampling decoder; Step (4): output the segmented infrared thermal imaging image; Furthermore, the step (2) trains the SAM encoder through the separate knowledge distillation process to obtain a lightweight image encoder such as Figure 3 , so that it maintains high-fidelity feature representation while significantly reducing the computational burden to meet the needs of clinical high-throughput screening and in-depth diagnostic evaluation. The formula is as follows: ; in, represents the distillation loss, represents the Euclidean norm, Indicates SAM's own heavy-duty encoder, represents the lightweight encoder after knowledge distillation, and I represents the input infrared image.

[0016] Furthermore, the dual-branch upsampling decoder of step (3) is composed of two dual-branch modules stacked together. By stacking the dual-branch upsampling decoders, the segmentation performance can be effectively improved at multiple granularity levels, such as Figure 4 shown.

[0017] Furthermore, the dual-branch module is composed of a fine-grained branch, a coarse-grained branch and a fused dual-branch module, and is specifically implemented as follows: In the fine-grained branch, the feature map input to the decoder is denoted as , where C is the number of channels, H and W are the height and width of the feature map, respectively. The fine-grained branch uses deconvolution to spatially upsample the feature map and refine the channel dimension. The activation function is Gaussian Error Linear Unit (GELU), followed by normalization. represents the Gaussian Error Linear Unit (GELU) activation function, The representation layer is normalized, and the formula is as follows: ; Indicates that it has parameters This operation effectively doubles the spatial dimension of F and reduces the channel dimension to ,get ; In the coarse-grained branch, the coarse-grained branch adjusts the channel dimension through point convolution and aligns the spatial resolution using bilinear interpolation. This branch ensures that the necessary broad features are retained and aligned with the upsampled fine-grained features. The operation is defined as follows: ; in, Indicates that it has parameters Point by point (1 1) Convolution, L is a bilinear interpolation function, which adjusts its input to The size of .

[0018] The dual-branch fusion module combines the outputs of the fine-grained and coarse-grained branches through element-wise addition to synthesize a refined feature map. The formula is as follows: ; in, This fusion enables the dual-branch upsampling decoder to effectively enhance both global and local features, allowing for detailed and accurate segmentation in consecutive stages.

[0019] Furthermore, the unsupervised zero-shot segmentation mode specifically includes the following steps: Figure 5 As shown: Step (1): Obtain infrared thermal imaging images of the soles of the feet; Step (2): Divide the infrared thermal imaging image in step (1) into two parts in the middle; Step (3): using a clustering algorithm, each part is divided into background and foreground according to the pixel intensity value; Step (4): Calculate the centroid of each part using the formula: ; in, represents half of the infrared image, represents the centroid coordinates of the area with high pixel intensity values ​​(foot), represents the centroid of all identified clusters; Step (5): Use the centroid coordinates as a spatial cue and input them into SAM. SAM uses this cue to focus on the foot region segmentation and effectively ignores other regions. It can accurately segment the foot region without any specific training for foot segmentation. The formula is as follows: ; in, is the segmentation mask, For a complete infrared image, are the parameters of the SAM model.

Claims

1. A method for automatic segmentation of foot area in infrared thermal imaging, characterized in that: The method specifically includes: supervised learning mode and unsupervised zero-sample segmentation mode; In the supervised learning mode, the method specifically includes the following steps: Step (1): Obtain infrared thermal imaging images of the soles of the feet; Step (2): input the infrared thermal imaging image in step (1) into an efficient and lightweight image encoder generated by separable knowledge distillation training to generate a feature map; Step (3): Input the feature map into the dual-branch upsampling decoder; Step (4): output the segmented infrared thermal imaging image; In the unsupervised zero-sample segmentation mode, the method specifically includes the following steps: Step (1): Obtain infrared thermal imaging images of the soles of the feet; Step (2): Divide the infrared thermal imaging image in step (1) into two parts in the middle; Step (3): Each part is divided into background and foreground according to the pixel intensity value using a clustering algorithm; Step (4): Calculate the centroid coordinates of each part using the formula: ； in, represents half of the infrared image, Represents the centroid coordinates of the area with high pixel intensity values, represents the centroid of all identified clusters; Step (5): Using the centroid coordinates as a hint, call the SAM model to perform segmentation and accurately segment the foot area without any specific training for foot segmentation. The formula is as follows: ； in, is the segmentation mask, is the input infrared image, is the centroid coordinate, are the parameters of the SAM model.

2. The method for automatic segmentation of foot area using infrared thermal imaging according to claim 1, characterized in that: The lightweight image encoder is trained through the separated knowledge distillation process, and the formula is as follows: ； in, represents the distillation loss, represents the Euclidean norm, represents the heavy encoder of the SAM model itself, Represents the lightweight encoder after knowledge distillation.

3. The method for automatic segmentation of foot area using infrared thermal imaging according to claim 1, characterized in that: The dual-branch upsampling decoder consists of two dual-branch modules stacked together.

4. The method for automatic segmentation of foot area using infrared thermal imaging according to claim 3, characterized in that: The dual-branch module consists of a fine-grained branch, a coarse-grained branch and a fused dual-branch module, wherein: In the fine-grained branch, the feature map input to the decoder is denoted as , where C is the number of channels, H and W are the height and width of the feature map, respectively. The fine-grained branch uses deconvolution to spatially upsample the feature map and refine the channel dimension. The activation function is the Gaussian error linear unit, followed by normalization. Let σ represent the Gaussian error linear unit activation function and Λ represent the layer normalization. The formula is as follows: ； Indicates that it has parameters The deconvolution doubles the spatial dimension of the feature map F and reduces the channel dimension to ,get ; In the coarse-grained branch, the coarse-grained branch adjusts the channel dimension through point convolution and uses bilinear interpolation to align the spatial resolution. This branch is used to retain and align the broad features with the upsampled fine-grained features. The operation is defined as follows: ； in, Indicates that it has parameters Point by point (1 1) Convolution, L is a bilinear interpolation function, which adjusts its input to The size of ; In the fusion of two branches, the outputs of the fine-grained and coarse-grained branches are combined by element-wise addition to synthesize the feature map. The formula is as follows: ； in, .

Images