Infrared Thermal Image Biped Region Segmentation Method Based on Relative Temperature Difference

Abstract

The present invention relates to a method for segmenting the bipedal region of an infrared thermal image based on relative temperature difference, belonging to the field of computer vision technology. The method includes: S1: Establishing a global average temperature-time curve of the biped; S2: Determining the moment t at which the temperature difference thermal image segmentation needs to be performed in the thermal image sequence; S3: Determining the reference moment t0; S4: Obtaining the bipedal edges of the temperature difference thermal image at moment t; S5: Using the U2-Net network to extract the complete edges of the biped in the infrared thermal image. The present invention can achieve the segmentation of the bipedal region with small difference or poor contrast between the bipedal region and the background in the active infrared thermal image sequence.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and relates to a method for segmenting infrared thermal map bipedal regions based on relative temperature difference. Background Art

[0002] The temperature change of the foot ulcer prone area of diabetic foot is higher than that of the non-ulcer area. Conventional methods for evaluating the skin, including inspection and palpation, although valuable methods, usually cannot detect changes in skin integrity before ulcer formation. On the contrary, infrared imaging is a technology that can evaluate skin integrity and its multi-layers, but sometimes it includes part of the ankle area. To improve the image segmentation accuracy, many institutions have designed and adopted occlusion devices, and the obtained thermal maps usually only contain the bipedal regions and the occluded background. When the contrast between the bipedal regions and the background is good, threshold methods, edge-based methods, watershed methods, etc. are all relatively easy to obtain good segmentation effects.

[0003] There are mainly two forms of applying infrared thermal imaging technology to diabetic foot detection: passive and active. The passive form is to directly use an infrared thermal imager to observe the temperature distribution of the sole of the foot, while the active form usually cools the sole of the foot first and then uses an infrared thermal imager to observe and record the sole heating process. For active infrared thermal imaging, during the experiment, the room temperature can usually be regarded as constant, and the sole temperature generally drops below the room temperature and then rises slowly. During this process, there will be a certain time range when the sole temperature is close to the background temperature at room temperature, and the image contrast is very poor. For passive infrared images, there may also be a situation where the contrast is very poor, and there is currently no way to solve the problem of bipedal region segmentation in this case well. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a method for segmenting infrared thermal map bipedal regions based on relative temperature difference, which can achieve the segmentation of bipedal regions with small difference or poor contrast between the bipedal regions and the background in the active infrared thermal map sequence.

[0005] To achieve the above purpose, the present invention provides the following technical solutions:

[0006] A method for segmenting infrared thermal map bipedal regions based on relative temperature difference, specifically including the following steps:

[0007] S1: Establish a global average temperature-time curve of the biped;

[0008] S2: Determine the moment t for which the temperature difference thermal map segmentation is required in the thermal map sequence;

[0009] S3: Determine the reference moment t0;

[0010] S4: Obtain the bipedal edge of the temperature difference heat map at time t;

[0011] S5: Use the U2-Net network to extract the complete edge of the infrared heat map of the biped.

[0012] Furthermore, step S1 specifically includes: Select the region of interest to obtain the average temperature value of each frame, and then subtract the average temperature value of the first frame from the average temperature value of each frame; Multiply the result by the ratio of the number of pixels in the region of interest to the number of pixels in the foot, and then add the average temperature value inside the foot of the first frame, that is, a relatively accurate global average temperature-time heating curve of the biped is obtained.

[0013] Furthermore, step S2 specifically includes: Establish a standard deviation-time curve of the heat map sequence, use the Sobel algorithm to extract the edge of the original biped heat map, and then classify it, divided into two categories: can be segmented and cannot be segmented. Use these two categories as the ordinate, and at the same time introduce the standard deviation as the abscissa; And use the Sigmoid function as the decision boundary curve. The abscissa value corresponding to the ordinate of 0.5 in the Sigmoid function is defined as the threshold σ0 for the temperature difference heat map segmentation required. Greater than the threshold σ0 means it can be segmented, and less than the threshold σ0 means it cannot be segmented. That is, the time period when the standard deviation in the heat map sequence is less than the threshold σ0 is the time t when the temperature difference heat map segmentation is required.

[0014] Furthermore, step S3 specifically includes: Assign different temperature differences ΔT to the inside of the foot and the background region, and then add Gaussian noise with different standard deviations; Then obtain the edge through the Sobel algorithm, establish the relationship between the temperature difference and the noise standard deviation by analyzing whether the edge can be correctly extracted; Calculate the noise of the background uniform position to determine the noise value of the temperature difference heat map, and then obtain the temperature difference from the relationship between the temperature difference and the noise standard deviation, so as to determine the reference time t0 according to the global average temperature-time curve of the biped established in step S1.

[0015] Furthermore, in step S3, the expression of the temperature difference heat map is:

[0016] f(x,y) = |T(x,y,t) - αT(x,y,t0)| t0 < t, 0 ≤ α ≤ 1 (1)

[0017] Among them, f(x,y) is the difference between the heat maps at t0 and t, T(x,y,t) is the obtained heat recovery sequence, (x,y) is the coordinate value, and α is the coefficient; There are four cases for the temperature difference result: sole minus background, background minus sole, sole minus sole, and background minus background.

[0018] Further, in step S3, establishing the relationship between the temperature difference and the noise standard deviation specifically includes: Assuming that in the original thermal image, the sole temperature is T1 and the background temperature is T0; currently, an infrared camera is used for acquisition. Due to noise interference during the camera acquisition process, at time t0, the sole temperature is T1 + n1 and the background temperature is T0 + n1; at time t1, the sole temperature is T2 + n2 and the background temperature is T0 + n2; n1 is the noise at time t0, and n2 is the noise at time t1; using σ1 and σ2 to represent the noise standard deviations at the two times respectively, then there is the following relationship: σ1 = σ2;

[0019] Using formula (1) for temperature difference processing, the background noise in the temperature difference result is:

[0020] T0 + n2 - α(T0 + n1) = (1 - α)T0 + n2 - αn1 (2)

[0021] Through simulation, it is found that the standard deviation of the temperature difference background noise has nothing to do with T0. When α is known, the relationship between the standard deviation σ of the temperature difference background noise and n1 or n2 is:

[0022]

[0023] Further, step S4 specifically includes: Selecting a thermal image that requires the temperature difference thermal image segmentation method from step S2, then determining the reference time t0 from step S3, and further obtaining the temperature value of the thermal image at the reference time. Subtracting the temperature value of the thermal image with a large contrast difference from the reference time, and then linearly converting the obtained difference from 0 to 255 to convert it into a picture; then performing Sobel algorithm and binarization operations on the temperature difference thermal image and the reference thermal image at time t0 respectively; in order to correctly obtain the edges of the two feet at time t, it is necessary to subtract the edges of the temperature difference thermal image from the edges of the reference thermal image at time t0, and the edge signal of the two feet at time t in the obtained result is stronger than that of the two feet edges of the thermal image at time t0.

[0024] Further, step S5 specifically includes: After performing the temperature difference processing of steps S1 to S4 on all experimental data, two-thirds of the overall data set is selected as the training set, and the remaining one-third is used as the test set; performing annotation processing on the training set, annotating the feature regions, converting the generated json file after annotation into a Mask map, and training through the U2-Net network to generate a U2-Net model; using the test set to test the obtained image, which is the result of the segmentation processing, and the target region is the region to be segmented.

[0025] The beneficial effects of the present invention are as follows: Based on the infrared temperature difference thermal image, the present invention realizes the segmentation of the two-foot region with a small difference or a large contrast difference between the two-foot region and the background in the active infrared thermal image sequence.

[0026] Other advantages, objects, and features of the present invention will be set forth in part in the following description, and in part will be obvious to those skilled in the art from a study of the following, or may be learned from practice of the invention. The objects and other advantages of the invention may be realized and attained by means of the instrumentalities and combinations particularly pointed out hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to make the objectives, technical solutions, and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings, where:

[0028] Figure 1 is the overall flowchart of the relative temperature difference segmentation method adopted by the present invention;

[0029] Figure 2 is the flowchart of the heat map temperature difference analysis;

[0030] Figure 3 is the biped histogram with different contrasts;

[0031] Figure 4 is the standard deviation - time curve;

[0032] Figure 5 is the curve graph of the relationship between the scatter plot and the Sigmoid function;

[0033] Figure 6 is the flowchart of obtaining the complete edge time interval;

[0034] Figure 7 is the curve graph of the temperature difference ΔT and the standard deviation;

[0035] Figure 8 is the global temperature - time curve;

[0036] Figure 9 is the heat map with sporadic edges;

[0037] Figure 10 is the segmentation graph with poor contrast. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0038] The following specific examples illustrate the embodiments of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the drawings provided in the following examples only illustrate the basic concept of the present invention schematically. Without conflict, the following examples and the features in the examples can be combined with each other.

[0039] Please refer toFigures 1 to 10 , the present invention provides a segmentation method based on a temperature difference heat map. Assume that T(x, y, t) is the obtained heat recovery sequence, where (x, y) are coordinate values and t is the corresponding time. The temperature difference heat map f(x, y) is the difference between the heat maps corresponding to two different moments, that is, the temperature value corresponding to the coordinate (x, y) at time t minus the temperature value corresponding to the coordinate at time t0 multiplied by a coefficient α, to obtain the difference between the two moments:

[0040] f(x, y) = |T(x, y, t) - αT(x, y, t0)| where t0 < t and 0 ≤ α ≤ 1 (1)

[0041] When the contrast of the heat map is poor, that is, when the temperature difference between the sole area and the background temperature is small, it is difficult to correctly segment the sole area. The original heat map mainly includes two regions: the sole and the background. Therefore, there are four cases for the temperature difference result: sole minus background, background minus sole, sole minus sole, and background minus background.

[0042] Assume that the reference time is t0, the background temperature value is T0, and the sole temperature value is T1. The time t is the moment with poor contrast, the background temperature value is T0, and the sole temperature value is T2. In the temperature difference heat map obtained by the method of formula (1), the temperature differences of the above four results relative to the fourth result (i.e., background minus background) are:

[0043] 1) Sole minus background: (T2 - αT0) - (T0 - αT0) = T2 - T0;

[0044] 2) Background minus sole: (T0 - αT1) - (T0 - αT0) = α(T0 - T1);

[0045] 3) Sole minus sole: (T2 - αT1) - (T0 - αT0) = T2 - T0 + α(T0 - T1);

[0046] 4) Background minus background: (T0 - αT0) - (T0 - αT0) = 0;

[0047] In the heat map at time t, the temperature difference between the inside of the foot and the background is T2 - T0. After being processed by formula 1), the obtained temperature difference becomes the above four cases. The present invention proposes to use the temperature difference method instead of using the original image to segment the two-foot area. Then, the temperature differences in the above cases should be greater than the temperature difference in the original image, that is, T2 - T0. Compare the results of the four cases in the obtained temperature difference heat map with T2 - T0:

[0048] 1) Sole minus background: (T2 - T0) - (T2 - T0) = 0;

[0049] 2) Background minus sole: α(T0 - T1) - (T2 - T0) = (T0 - T2) + α(T0 - T1);

[0050] 3) Sole minus sole: (T2 - T0 + α(T0 - T1)) - (T2 - T0) = α(T0 - T1);

[0051] 4) Background minus background: As the background reference temperature for the first three cases, it does not need to be considered further.

[0052] Whether it is the original image or the temperature difference thermal image, since the image of the region of interest is mainly composed of the background and the two feet, the image contrast is determined by the absolute value of the temperature difference between the two feet and the background. For the first case, since the result is 0, the temperature difference and the contrast in the original image remain unchanged. For the second case, it is necessary to subtract the sole at time t0 from the background at time t of the temperature difference. The obtained temperature difference signal is about the sole at time t0. Since the present invention focuses on the sole region at time t, the improvement of its temperature difference is not discussed. For the third case, the difference is α(T0 - T1). If the selected time is relatively early, T0 > T1, the difference is greater than 0, and the contrast will be improved. If T0 < T1, then α(T0 - T1) < 0, and the contrast is worse. During the experiment, the background temperature remains unchanged, and the initial temperature in the foot region is lower than the background temperature, then it rises slowly and finally reaches equilibrium. For the sole region at time t that the present invention is interested in, it is composed of the results of the first and third cases in the temperature difference thermal image. At the same time, during the experiment, the two feet basically remain stationary, only with small angular rotations or small movements. Therefore, the sole region in the temperature difference thermal image is mainly composed of the third case. Based on the above analysis, the temperature difference thermal image based on Equation 1) has better contrast than the original thermal image. Therefore, the present invention proposes an infrared thermal image two-foot region segmentation method based on relative temperature difference, and the overall algorithm flow is as Figure 1 shown, specifically including the following steps:

[0053] Step 1: Establish the global (average) temperature-time curve of the two feet.

[0054] In the entire experimental image, the background remains unchanged, and what changes is the two-foot region. Whether the two feet move or not has no impact on the temperature rise of the image. Therefore, the temperature rise curve of the entire image is the average temperature rise curve of the two-foot region. Select the region of interest ROI and find the average temperature value of each frame, then subtract the average temperature value of the first frame from the average temperature value of each frame. Multiply the result by the ratio of the number of pixels in the ROI region to the number of pixels in the foot region, and then add the average temperature value of the foot region in the first frame to obtain a more accurate two-foot temperature-time average temperature rise curve. Based on this curve, when the interval time t for which the temperature difference analysis needs to be performed and the corresponding temperature difference are determined, the reference time t0 can be obtained, and then the temperature difference result can be obtained for further processing.

[0055] Step 2: Determine at which moments the thermal images need to be segmented using the segmentation method based on the temperature difference thermal images.

[0056] For the infrared thermal image bipedal region segmentation method using relative temperature difference, it is first necessary to determine which moments of the thermal images can be directly processed using traditional methods and which moments of the thermal images need to be processed using the infrared thermal image segmentation method based on relative temperature difference proposed in the present invention. For which moments of the thermal images need the segmentation method based on the temperature difference thermal images, its algorithm flow is as Figure 2 shown.

[0057] For the thermal image at a specific moment, first calculate its histogram. For the thermal image at an earlier moment, the sole temperature is lower than the background temperature. Due to the large temperature difference between the sole and the background, its histogram shows a typical bimodal shape. As time increases, the sole temperature slowly rises while the background temperature remains unchanged, and the histogram no longer has obvious bimodality, that is, the two peaks will gradually overlap, and there is a process from bimodality to unimodality. When the acquisition time is long enough, as time further increases, the sole temperature further rises and is higher than the background temperature, and the unimodal histogram will gradually become a bimodal histogram. During actual detection, different experimental environments may not necessarily have the complete process from bimodality to unimodality and then to bimodality as described. As Figure 3 shown, when the image histogram shows a typical bimodal shape ( Figure 3 (a)), the temperature difference between the sole and the background is large, the image contrast is good, and the standard deviation is large; while when it shows a unimodal shape ( Figure 3 (b)), the temperature difference between the sole and the background is small, the image contrast is poor, and the standard deviation is small.

[0058] For the thermal image sequence data obtained from the experiment, calculate the standard deviation of all moments of the thermal images. Taking time as the abscissa and the standard deviation as the ordinate, a standard deviation-time curve can be obtained. As Figure 4 shown, due to the influence of noise, there are certain fluctuations in this curve. After polynomial fitting, the relationship between the standard deviation and time can be obtained:

[0059]

[0060] where a i represents the coefficient after polynomial fitting; for different experimental sequence data, the polynomial parameters in formula (2) are different. A complete standard deviation-time curve is a "U"-shaped curve, with a smaller central standard deviation, which also corresponds to poor image contrast.

[0061] To determine the interval for temperature difference processing, observe the standard deviation - time curve. It is known that the larger the standard deviation, the better the contrast, and the corresponding thermal images of the two feet are easier to segment. Conversely, the smaller the standard deviation, the worse the contrast of the thermal image, and it is more difficult to segment the two feet. To determine the value range of t, first use the Sobel algorithm to extract the edges of the original image. According to the continuity of the edges, the images of the two feet are divided into two categories: can be segmented (1) and cannot be segmented (0). Taking these two categories as the ordinate and introducing the standard deviation as the abscissa at the same time. Since the standard deviation value is small and not convenient to observe, the value is scaled to -1 to 1. After determining the threshold, its value needs to be restored to the standard deviation, as Figure 5 shown. To better determine the threshold, the Sigmoid function is introduced here as the decision boundary curve. The function curve is smooth in extreme cases and almost linear in the middle, reducing the error of misclassification. The abscissa value corresponding to the ordinate of 0.5 in the Sigmoid function is the scaled value, and it needs to be restored to the original standard deviation. The restored value is defined as the threshold σ0 required for temperature difference analysis. The time period in the thermal image sequence with a standard deviation smaller than this threshold is the time t required for temperature difference analysis.

[0062] Step 3: Determine the reference time t0.

[0063] Given the time t required for temperature difference analysis, it is also necessary to determine the selection of its reference image, that is, the selection of t0. The processing flow is as Figure 6 shown. The general idea is: when there is no noise, there is a temperature difference between the inside of the foot and the background, and theoretically, the edges should be correctly obtained. However, the experimental data has noise, and different thermal imagers have different levels of noise. Therefore, under the interference of noise, the temperature difference between the inside of the foot and the background needs to reach a certain level to correctly obtain the edges using the image processing algorithm. The present invention uses Sobel for processing as in Step 2. The noise of the thermal imager usually follows a Gaussian distribution. Different temperature differences ΔT are given to the inside of the foot and the background regions, and then Gaussian noise with different standard deviations is added. Then, the edges are obtained through the Sobel algorithm, and the relationship between the temperature difference and the noise standard deviation is established by analyzing whether the edges can be correctly extracted, as Figure 7 shown.

[0064] For the original thermal image, through the Figure 8 established relationship, given the noise, the required temperature difference can be obtained to correctly segment the sole area. For the experimental data, the noise standard deviation can be calculated by selecting a relatively uniform background area. Since the present invention is based on relative temperature difference for subsequent processing, it is necessary to analyze the relationship between noise and temperature difference in the temperature difference thermal image. From the perspective of image processing, this relationship is the same in the original image and the temperature difference thermal image. However, in the temperature difference thermal image, the temperatures of the background and the inside of the foot have both changed.

[0065] Assume that in the original heat map, the plantar temperature is T1 and the background temperature is T0. Now, an infrared camera is used for acquisition. Due to noise interference during the camera acquisition process, at time t0, the plantar temperature is T1 + n1 and the background temperature is T0 + n1. At time t1, the plantar temperature is T2 + n2 and the background temperature is T0 + n2. Among these two moments, n1 is the noise at time t0, expressed as n1(x, y), and n2 is the noise at time t1, expressed as n2(x, y). In fact, the noise values at different positions are different, that is, n1(x, y) ≠ n2(x, y). Let σ1 and σ2 represent the noise standard deviations at the two moments respectively, then there is the following relationship: σ1 = σ2, that is, the standard deviations are the same.

[0066] Perform temperature difference processing using formula (1). The background noise in the temperature difference result is:

[0067] T0 + n2 - α(T0 + n1) = (1 - α)T0 + n2 - αn1 (3)

[0068] Through simulation, it is found that the standard deviation of the temperature difference background noise has nothing to do with T0. When α is known, the relationship between the standard deviation σ of the temperature difference background noise and n1 or n2 is:

[0069]

[0070] After being processed by formula (1), the plantar region at time t is divided into two parts. One part is the plantar minus the background. Since its temperature difference is the same as that between the plantar and the background in the original image, it cannot be correctly segmented in the original image and also cannot be correctly segmented in the temperature difference heat map. The other part, which is also the main part, is the plantar minus the plantar. According to the previous analysis, the temperature difference between it and the temperature difference background is T2 - T0 + α(T0 - T1). In this expression, except for T1, the other parameters are known or can be obtained from experimental data. Using the relationship between ΔT and the noise standard deviation obtained as Figure 8 and calculating the temperature difference background noise using formula (4), T1 can be obtained. Finally, using the global average temperature-time curve established in step 1, the time t0 corresponding to T1 can be obtained.

[0071] Step 4: Find the edge of the temperature difference heat map.

[0072] First, select a thermal image from the time t for which the temperature difference analysis is to be performed, and then combine it with the thermal image at the reference time t0. Subtract the temperature value corresponding to each pixel in the thermal image for which the temperature difference analysis is to be performed from the temperature value of each pixel in the reference time. Then, linearly transform the obtained values from 0 to 255 to convert them into an image. Next, perform the Sobel algorithm and binarization operation on the temperature difference thermal image and the reference thermal image at time t0 respectively. For the binarization operation, a simple threshold function is used for binarization, and the threshold is set to 80. All values greater than 80 will be processed as 255, and the remaining values will be processed as 0. The edge of the binarized temperature difference thermal image not only contains the edges of the two feet at time t but may also contain partial edges of the two feet at time t0. To correctly obtain the edges of the two feet at time t, subtract the edges of the two feet in the thermal image at time t0 from it. The signal of the edges of the two feet at time t in the obtained result is stronger than that of the edges of the two feet in the thermal image at time t0, and the edges of the two feet at time t can be correctly extracted through subsequent deep learning.

[0073] Step 5: Use the U2-Net network to extract the complete edge.

[0074] After performing the temperature difference processing on all experimental data using the above method, two-thirds of the total is selected as the training set, and the remaining one-third is used as the test set. The data set is labeled to mark the feature regions, and the generated json file after labeling is converted into a Mask image and trained through the U2-Net network to generate a U2-Net model. Using the test set to test the obtained images is the result of the segmentation process, where the target region is the desired region.

[0075] The theoretical basis of the present invention is based on a process in which the two feet automatically warm up after active cooling, which is a dynamic process. The thermal imager performs non-uniformity correction every once in a while, and after the correction, there is no obvious difference in the standard deviation of the noise temperature at each stage and at different times within the same time period. When selecting pictures, pay attention not to select the first few pictures to avoid errors caused by uneven cooling. Specific embodiments:

[0077] To facilitate the understanding of the present invention, this embodiment illustrates the method for segmenting the relative temperature difference of the two-foot region in the infrared thermal image of the present invention through specific experiments. In this embodiment, 14-degree cold water is used to directly cool the two feet for 30 seconds, the room temperature is 25.5 °C, the humidity is 72%, and 900 thermal image sequences corresponding to the data collected at 1 Hz for 15 minutes are processed.

[0078] For this experimental data, first, it is necessary to determine which moments require the segmentation method based on the temperature difference thermal image. During the invention stage, it has been determined that the standard deviation of the region of interest in the thermal image with a poor contrast is smaller than that in the thermal image with a good contrast. A time curve of the standard deviation of the two feet for the entire thermal image sequence is established, and then by establishingFigure 5 Regarding the relationship, it is found that when the standard deviation is lower than 0.02, the contrast of its heat map is poor and temperature difference analysis is required. In this embodiment, only the standard deviation of the region of interest needs to be calculated. If its value is less than 0.02, then the bipedal heat map requires temperature difference analysis.

[0079] Next, it is necessary to determine the reference time t0. First, select a heat map from the times t for which temperature difference analysis is required. Calculate the temperature standard deviation of a region with uniform background as 0.01 and use it as the background noise. The temperature fluctuation coefficient α is 0.99. According to formula (4), the noise value of the temperature difference map can be calculated as 0.014. Refer to Figure 7 The curve graph of temperature difference ΔT and standard deviation σ. The temperature difference ΔT corresponding to the noise value of the temperature difference map in this embodiment is 0.6617 °C.

[0080] At this time, it is necessary to establish a global temperature-time curve corresponding to this embodiment, as Figure 8 shown. First, use infrared software to obtain the temperature rise curve graph within the region of interest. The region of interest (ROI) includes the complete bipedal part. The temperature rise curve heat map obtained at this time is the average temperature rise curve including the background. Subtract the average temperature value of the first frame from this temperature rise curve and prepare the curve graph in place. Then multiply the values on this curve by the ratio of the pixel values in the ROI region to the pixel values within the foot in the first frame, and add the average temperature value of the foot within the first frame when preparing in place. At this time, an accurate bipedal temperature rise heat map curve is obtained. Select a heat map with poor contrast in the interval where temperature difference analysis is required. The temperature value at this time can be obtained from Figure 8 and according to formula (1) and the obtained temperature difference ΔT, the temperature at the reference time t0 can be deduced, thereby finding the time corresponding to this temperature.

[0081] Given the reference heat map and the heat map with poor contrast, subtract the temperature corresponding to each pixel value in the heat map for which temperature difference analysis is required from the temperature value of each pixel in the reference time according to formula (1). Linearly convert the temperature difference result from 0 to 255 into grayscale values and thus convert it into a picture. Then perform the Sobel algorithm and binarization operation on the temperature difference heat map and the reference heat map at time t0 respectively. The edge of the binarized temperature difference heat map not only contains the bipedal edges at the moment with poor contrast, but may also contain part of the bipedal edges at the reference time t0. In order to correctly obtain the bipedal edges at time t, subtract the bipedal edges in the heat map at time t0 from it. The bipedal edges at time t in the obtained result have a stronger signal compared to the bipedal edges in the heat map at time t0, as Figure 9 shown.

[0082] A series of corresponding thermogram edges are obtained by using the method of thermogram segmentation based on temperature difference. Among these thermogram edges, two-thirds of the total are selected as the training set, and the remaining one-third is used as the test set. The data set is labeled to mark the feature regions. The generated json file after labeling is converted into a Mask graph, and the U2-Net network is used for training to generate the U2-Net model. Finally, the biped can be segmented using this model, as Figure 10 shown.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the purpose and scope of the present technical solution, and they should all be covered within the scope of the claims of the present invention.

Claims

1. An infrared thermal image bipedal region segmentation method based on relative temperature difference, characterized in that The method specifically includes the following steps: S1: Establish the global average temperature-time curve of the two feet; S2: Determine the moment t at which the thermal map difference needs to be segmented in the thermal map sequence; S3: Determine the reference moment t0, specifically including: assigning different temperature differences ΔT to the foot interior and the background region, and then adding Gaussian noise with different standard deviations; then obtaining the edges through the Sobel algorithm, and establishing the relationship between the temperature difference and the noise standard deviation by analyzing whether the edges can be correctly extracted; calculating the noise of the background uniform position to determine the noise value of the thermal map difference, and then obtaining the temperature difference from the relationship between the temperature difference and the noise standard deviation, so as to determine the reference moment t0 according to the global average temperature-time curve of the two feet established in step S1; The expression of the thermal map difference is: Among them, is the corresponding difference with the heat map at time t, T(x, y, t) is the obtained heat recovery sequence, (x, y) are coordinate values, is a coefficient; there are four cases for the temperature difference result: sole minus background, background minus sole, sole minus sole, and background minus background; S4: Obtain the two-foot edges of the thermal map difference at moment t; S5: Use the U2-Net network to extract the complete edges of the infrared thermal map of the two feet.

2. The infrared thermal image bipedal region segmentation method according to claim 1, wherein Step S1 specifically includes: selecting the region of interest to obtain the average temperature value of each frame, and then subtracting the average temperature value of the first frame from the average temperature value of each frame; multiplying the result by the ratio of the number of pixels in the region of interest to the number of pixels in the foot interior, and then adding the average temperature value of the foot interior of the first frame, that is, obtaining the global average temperature-time heating curve of the two feet.

3. The infrared thermal image bipedal region segmentation method according to claim 1, characterized in that Step S2 specifically includes: establishing the standard deviation-time curve of the thermal map sequence, using the Sobel algorithm to extract the edges of the original two-foot thermal map picture, and then classifying it, dividing it into two categories: can be segmented and cannot be segmented. Taking these two categories as the ordinate, and introducing the standard deviation as the abscissa at the same time; and using the Sigmoid function as the decision boundary curve, and setting the abscissa value corresponding to the ordinate of 0.5 in the Sigmoid function as the threshold σ0 for the thermal map difference segmentation. If it is greater than the threshold σ0, it can be segmented, and if it is less than the threshold σ0, it cannot be segmented. That is, the time period with a standard deviation less than the threshold σ0 in the thermal map sequence is the moment t at which the thermal map difference needs to be segmented.

4. The infrared thermal image bipedal region segmentation method according to claim 1, characterized in that In step S3, the relationship between the temperature difference and the noise standard deviation is established, specifically including: Assume that in the original thermal image, the plantar temperature is T1 and the background temperature is T0. Now, when using an infrared camera to collect data, due to noise interference during the camera collection process, there will be At a certain moment, the plantar temperature is T1 + , and the background temperature is T0 + ; At another moment, the plantar temperature is T2 + , and the background temperature is T0 + ; is the moment noise, is the moment noise; Let and represent the noise standard deviations at the two moments respectively, then there is the following relationship: ;​​​ Perform the temperature difference processing using formula (1), and the background noise in the temperature difference result is: When α is known, the relationship between the standard deviation σ of the temperature difference background noise and or is as follows: 。 5. The infrared thermal image bipedal region segmentation method according to claim 1, characterized in that Step S4 specifically includes: selecting a thermal image that requires the use of the temperature difference thermal image segmentation method from step S2, and then determining the reference time by step S3 , thereby obtaining the temperature value of the thermal image at the reference time, subtracting the temperature value of the thermal image with poor contrast from the reference time, and then linearly converting the obtained difference from 0 to 255 to convert it into a picture; then performing Sobel algorithm and binarization operations on the temperature difference thermal image and the reference thermal image at the moment respectively; subtracting the edge of the temperature difference thermal image from the edge of the reference thermal image at the moment, and the edge of the two feet at time t in the obtained result is stronger compared to the edge signal of the two feet in the thermal image at the moment.

6. The infrared thermal image bipedal region segmentation method according to claim 1, characterized in that Step S5 specifically includes: after performing the temperature difference processing of steps S1 to S4 on all experimental data, selecting two-thirds of the overall data set as the training set, and the remaining one-third as the test set; performing annotation processing on the training set, marking the feature regions, converting the generated json file after annotation into a Mask map, and training through the U2-Net network to generate a U2-Net model; using the test set to test the obtained images, and the target region is the region to be segmented.

Images