A self-enhancement method based on Gamma transformation

Abstract

The present invention relates to a self-enhancement method based on Gamma transformation, belonging to the field of infrared thermal imaging. This method first performs contrast enhancement on the biped thermal map using piecewise Gamma transformation. Its mechanism is based on the analysis of the cumulative distribution function of pixels to obtain the relative temperature difference at different time periods. According to the relative temperature difference, the enhancement factor γ is obtained to achieve contrast enhancement of the biped thermal map and make the edges of the biped thermal map clear. The improved U<supgt;2< / supgt>-Net network image segmentation mechanism is based on the U<supgt;2< / supgt>-Net network structure. By replacing the ordinary convolution module with the reference cyclic residual convolution module structure, replacing the Relu activation function with the Elu activation function to eliminate the gradient vanishing problem, and increasing the dilated convolution to enlarge the receptive field, the biped thermal map enhanced by piecewise Gamma transformation is used as the data set, which can quickly and accurately segment the foot area.

Description

Technical Field

[0001] The present invention belongs to the field of infrared thermal imaging and relates to a self-enhancement method based on Gamma transformation. Background Art

[0002] The most common mechanism of foot ulcers is the cumulative effect of unrecognized trauma under pressure points on the soles of the feet. The areas where ulcers may occur are related to areas with elevated local skin temperature caused by inflammation and tissue enzyme autolysis. This inflammation has five characteristics: redness, heat, swelling, pain, and loss of function. Some signs are difficult for clinicians to objectively evaluate; however, temperature measurement can provide quantitative data, demonstrating that impending ulcers can be predicted. Traditional non-invasive methods for evaluating foot skin integrity, including inspection and palpation, may be valuable diagnostic tools, but they usually do not detect changes in skin integrity before skin rupture occurs. In addition, the manual infrared thermometers used in clinics can only provide the average value of a large area of the foot. In this search study, thermal imaging technology is used to monitor the temperature distribution on the skin. However, there is no standard distribution of the skin surface temperature of healthy feet because skin temperature may be affected by many factors, such as environmental and internal thermal conditions, age, gender, weight, etc. Currently, common methods for segmenting the thermal images of both feet include the watershed method, genetic algorithm, Otsu segmentation method based on local histogram equalization, etc.

[0003] The watershed algorithm is an image region segmentation method. During the segmentation process, it takes the similarity between adjacent pixels as an important reference basis, and thus connects pixel points that are close in spatial position and have similar gray values (calculate the gradient) to form a closed contour. The common operation steps of the watershed algorithm: grayscale the color image, then calculate the gradient map, and finally perform the watershed algorithm on the gradient map to obtain the edge line of the segmented image. However, when collecting data for the thermal images of both feet, as the infrared camera shooting time lengthens, the heat in the diseased area may spread to the surrounding colder areas, causing the temperature of the entire foot to gradually increase, resulting in a lower contrast. Therefore, in real thermal images of both feet, due to the existence of noise points or other interference factors, the watershed algorithm often has the phenomenon of over-segmentation, and there will be many very small local extreme points. This leads to incomplete foot features or the inclusion of noise points in the feature map.

[0004] The genetic algorithm and the Otsu method adaptively control the search process to obtain the optimal threshold solution, so as to achieve the most perfect image cutting effect. However, in the thermal images of both feet, the diseased area often transfers heat to the surrounding area, making the temperature of the surrounding area close to that of the diseased part. In this way, the foot features cannot be completely segmented, and noise points often appear.

[0005] Morphology is a relatively common method in the early processing of biped thermal maps. First, the foot area is defined as the region of interest and separately segmented. Then, erosion or dilation is performed on the region of interest to complement the foot or eliminate noise. Next, the Otsu method or other methods are used to obtain the optimal threshold to obtain the most complete shape. However, this method is rather cumbersome. When selecting the region of interest, it is necessary to ensure the integrity of the edge. When the edge is not clear, it cannot be selected.

[0006] With the development of deep learning, there are gradually more methods of using deep learning for biped thermal map segmentation. However, most deep learning methods for segmenting biped thermal maps may consume a relatively long time and have low accuracy, and cannot completely cut out the shape of the foot. With the 2 proposal of the U-Net network, the segmentation of biped thermal maps can be fast and accurate. However, when the contrast is poor or there is too much noise, the shape of the foot cannot be accurately segmented either. Therefore, based on the U- 2 Net network, improvements are made to enable it to quickly and accurately segment biped thermal maps. This method includes two steps: contrast enhancement and image segmentation.

[0007] In image analysis and recognition, generally, the given image needs to be segmented first, then the segmented regions are appropriately described, and then some analysis can be performed on the image. Image segmentation is an important processing step before image analysis and is a basic prerequisite for visual analysis and pattern recognition of images. The image segmentation based on U- 2 Net is a new type of segmentation technology. U- 2 Net is based on a stacked U-shaped structure to deepen the network for SOD (Salient Object Detection). It is a simple and powerful deep network architecture and a two-layer nested U-shaped structure. Summary of the Invention

[0008] In view of this, the purpose of the present invention is to provide a self-enhancement method based on Gamma transformation.

[0009] To achieve the above purpose, the present invention provides the following technical solutions:

[0010] A self-enhancement method based on Gamma transformation, the method comprising the following steps:

[0011] S1: Obtain a sequence of thermal maps of the surface of the measured foot using an infrared thermal imaging device and store the sequence of thermal maps in a general-purpose memory;

[0012] S2: Image the data in the memory. The data in the memory is radiation value. First, convert the radiation value to temperature, perform image gray-scale conversion on the temperature according to a fixed temperature window, and then perform normalization processing on the pixel gray-scale value to compress the pixel value to between 0 and 1;

[0013] S3: Obtain the histogram of each image, then calculate its cumulative distribution function, and normalize its ordinate. On the obtained normalized cumulative distribution function curve, select the pixel value X0 corresponding to the median value of the ordinate, that is, 0.5;

[0014] S4: Determine the position of the pixel value X0 corresponding to the median value of the ordinate, that is, 0.5, in the histogram, and calculate the distance between X0 and the median value of the abscissa, that is:

[0015] f = 0.5 - X0 (1)

[0016] S5: Based on the value of f, calculate the Gamma enhancement factor according to the change of the foot temperature;

[0017] The specific content of S5 is as follows:

[0018] S51: The judgment standard for the relative temperature value is:

[0019]

[0020] S52: When the relative temperature value is small, use formula (3) to calculate the Gamma enhancement factor:

[0021]

[0022] S53: When the relative temperature value is large, use formula (4) to calculate the Gamma enhancement factor:

[0023]

[0024] S54: Gamma transformation:

[0025]

[0026] S6: Calculate β according to X0 and X1 by formula (2), and then perform segmented Gamma transformation processing according to the value of β;

[0027] S7: When β < 1, the difference between X0 and X1 is small, and the overall image is brightened or darkened. The enhancement method is as follows: For all pixel points with gray values between 0 and X0 in the original image, perform Gamma transformation: Calculate f0 according to formula (5) and calculate the enhancement factor γ1 according to formula (3), and substitute them into formula (1) to obtain the new gray values between 0 and X0;

[0028] S8: For all pixel points with gray values between X0 and 1 in the original image, perform Gamma transformation: Calculate the distance f1 between X1 and the abscissa and then substitute it into formula (3) to calculate the enhancement factor γ2, and substitute it into formula (1) to obtain the new gray values between X0 and 1;

[0029] S9: When β > 1, the difference between X0 and X1 is large. Due to the fixed temperature window, the background is relatively dark. To make the foot area clear and the background darker, the enhancement method is as follows. For all pixel points with gray values between 0 and X0 in the original image, perform Gamma transformation: Calculate f0 according to formula (5) and the enhancement factor γ1 according to formula (4), and substitute them into formula (1) to obtain the new gray values from 0 to X0;

[0030] S10: For all pixel points with gray values between X0 and 1 in the original image, perform Gamma transformation: Calculate the distance f1 between X1 and the abscissa according to formula (2), then substitute it into formula (4) to calculate the enhancement factor γ2, and substitute it into formula (1) to obtain the new gray values from X0 to 1;

[0031] S11: Invoke the cyclic residual convolution module and train the biped heat map with high complexity by specifying the number of convolutional layers in each convolutional module;

[0032] S12: Increase the number of dilated convolutions and deepen the sixth layer of the decoding of the U 2 -Net network to obtain higher-dimensional feature information;

[0033] S13: Replace the relu activation function with the elu activation function;

[0034] S14: Enhance the dataset through piecewise Gamma transformation; After image enhancement of all test data, select two-thirds of the total as the training set and the remaining one-third as the test set; Perform annotation processing on the dataset, mark the feature regions, and convert the generated json file after annotation into a Mask map;

[0035] S15: Train the training set with the network described in S11~S14 to obtain the improved U 2 -Net segmentation model, use the test set data for verification, generate a mask map, that is, a binary map, extract features from the test set through the mask map, and finally obtain the foot area.

[0036] The beneficial effects of the present invention are as follows: The theoretical basis of the present invention is based on the process of segmenting images by the U 2 -Net network. Therefore, this algorithm is not only applicable to biped heat maps, but also applicable to the segmentation processing of heat maps obtained from other heat conduction processes that can be approximately represented by internal heat sources.

[0037] Other advantages, objectives, and features of the present invention will be described to some extent in the subsequent specification, and to some extent, will be obvious to those skilled in the art based on the study of the following text, or can be learned from the practice of the present invention. The objectives and other advantages of the present invention can be achieved and obtained through the following specification. Brief Description of the Drawings

[0038] In order to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be described in detail preferably with reference to the accompanying drawings, where:

[0039] Figure 1 For the flowchart of image segmentation based on piecewise Gamma transform and improved U 2 -Net network;

[0040] Figure 2 For the thermal image of the temperature rise of the test foot;

[0041] Figure 3 For the cumulative distribution function curve of the image;

[0042] Figure 4 For the cyclic residual convolution module;

[0043] Figure 5 For the result of piecewise Gamma transform;

[0044] Figure 6 For the improved U 2 -Net network structure

[0045] Figure 7 For the improved U 2 -Net network structure result. Detailed Embodiments

[0046] The following specific examples illustrate the embodiments of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the drawings provided in the following embodiments only illustrate the basic concept of the present invention schematically, and the following embodiments and the features in the embodiments can be combined with each other without conflict.

[0047] Among them, the attached drawings are only for illustrative purposes, showing only schematic diagrams rather than physical diagrams, and should not be construed as a limitation to the present invention; for better illustration of the embodiments of the present invention, some components in the attached drawings will be omitted, enlarged or reduced, which do not represent the dimensions of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the attached drawings may be omitted.

[0048] In the attached drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "rear", etc. indicating the orientation or positional relationship, they are based on the orientation or positional relationship shown in the attached drawings. This is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, the terms describing the positional relationship in the attached drawings are only for illustrative purposes and should not be construed as a limitation to the present invention. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to specific circumstances.

[0049] In view of the above problems, the present invention provides a new method for self-selection of Gamma transformation parameters and an improvement to the U 2 -Net network to achieve fast and accurate bipedal heatmap segmentation. The flowchart is shown in Figure 1 . The algorithm can be divided into two parts, namely image enhancement and segmentation.

[0050] I. Adaptive Gamma transformation

[0051] In order to segment the foot area image in the obtained heatmap sequence, first, data is collected by an infrared camera. The data collected is infrared radiation (radiation value). Therefore, it is necessary to convert the collected infrared radiation data into a picture format. First, the radiation value is converted into temperature, and the temperature is subjected to image gray conversion according to a fixed temperature window, and then the pixel values are normalized to compress the pixel values between 0 and 1.

[0052] After normalization, the grayscale image may have unclear edges, or the foot area may be too bright or too dark. As Figure 2 shown, the left figure has a relatively large temperature relative value β = 1.32, and the right figure has a relatively small temperature relative value β = 0.74; this results in a very poor contrast of the entire picture. An adaptive Gamma transformation is proposed for image enhancement processing. Through non-linear transformation, Gamma transformation makes the linear response of the image to the exposure intensity closer to the response perceived by the human eye, that is, it corrects the overexposed (excessive camera exposure) or underexposed (insufficient exposure) pictures, so that the output image gray value and the input image gray value have an exponential relationship. The calculation formula of Gamma transformation is shown as follows:

[0053]

[0054] (1) In the formula, V out is the pixel value of the output of the Gamma transformation, V in is the input pixel value of the Gamma transformation, and γ is the enhancement factor. In order to find a suitable γ (enhancement factor), a method for automatically obtaining γ (enhancement factor) according to the picture itself is proposed. After obtaining the grayscale image, the histogram of each picture is obtained, and then its cumulative distribution function is calculated, and its ordinate is normalized. As Figure 3 shown, on the obtained normalized cumulative distribution function curve, select the pixel grayscale value X0 corresponding to the median value of the ordinate (i.e., 0.5), and then calculate the distance f between it and the median value of the abscissa:

[0055] f = 0.5 - X0 (2)

[0056] The judgment criterion for the relative temperature value is:

[0057]

[0058] X0 corresponds to the pixel grayscale value corresponding to the ordinate 0.5 of the cumulative distribution function, X1 corresponds to the pixel grayscale value corresponding to the ordinate 0.75 of the cumulative distribution function, and this value is equivalent to the data in the ordinate range 0.5 - 1 of the original cumulative distribution function being further normalized from 0 to 1, and then the abscissa value corresponding to its 0.5. When β > 1, X1 and X0 are far apart, and the relative temperature difference is large, which only occurs in the case of being relatively dark. When it is relatively bright, X0 is greater than 0.5, and it is impossible for β to be greater than 1; when β < 1, X1 and X0 are adjacent, and the relative temperature difference is small.

[0059] Based on f in formula (2), the γ value is obtained through the following method:

[0060] 1. When β < 1:

[0061]

[0062] 2. When β > 1:

[0063]

[0064] Since the method of using a fixed temperature window is adopted when converting the original temperature data into a grayscale image, and at the same time, the experimental scheme adopts an active refrigeration method and the experimental environment temperature is relatively fixed, therefore, for an obtained experimental data sequence, it can generally be divided into three segments: in the front-segment data, the sole temperature is lower than the background, the relative temperature value is relatively large, the contrast is good, the image of the region of interest is overall darker, and β < 1; in the middle-segment data, the sole temperature is close to the background temperature, the relative temperature value is small, the contrast is relatively poor, and β < 1; in the back-segment data, the sole temperature is higher than the background temperature, the relative temperature value is large, the contrast is good, the image of the region of interest is overall brighter, and β > 1. To obtain the enhancement factor γ of each picture, first obtain X0 and X1 according to Figure 3 as shown, then calculate β by formula (3), and then perform segmented Gamma transformation processing according to the size of the β value:

[0065] 1. When β < 1, X0 and X1 are relatively close, and the overall image is brightened or darkened. The enhancement method is as follows:

[0066] ① Perform Gamma transformation on all pixel points with grayscale values from 0 to X0 in the original image: calculate f0 according to formula (2) and calculate the enhancement factor γ1 according to formula (4), and substitute it into formula (1) to obtain the new grayscale values from 0 to X0.

[0067] ② Perform Gamma transformation on all pixel points with grayscale values from X0 to 1 in the original image: as Figure 3 shown by the cumulative distribution function of the normalized histogram from X0 to 1 on the right, calculate the distance f1 between X1 and the abscissa according to formula (2), then substitute it into formula (4) to calculate the enhancement factor γ2, and substitute it into formula (1) to obtain the new grayscale values from X0 to 1.

[0068] 2. When β > 1, X0 and X1 are relatively far apart. Due to the fixed temperature window, the background is dark. To make the foot area clear and the background darker, the enhancement method is as follows:

[0069] ① Perform Gamma transformation on all pixel points with grayscale values from 0 to X0 in the original image: calculate f0 according to formula (2) and calculate the enhancement factor γ1 according to formula (5), and substitute it into formula (1) to obtain the new grayscale values from 0 to X0.

[0070] ② Perform Gamma transformation on all pixel points with grayscale values from X0 to 1 in the original image: as Figure 3 shown by the cumulative distribution function of the normalized histogram from X0 to 1 on the right, calculate the distance f1 between X1 and the abscissa according to formula (2), then substitute it into formula (5) to calculate the enhancement factor γ2, and substitute it into formula (1) to obtain the new grayscale values from X0 to 1.

[0071] II. Image Segmentation Based on the Improved U 2 -Net Model

[0072] The U 2 -Net model is based on the U-Net network, proposes a residual convolutional module structure, fuses the features of receptive fields at different scales, increases the depth of the entire architecture without significantly increasing the computational cost. However, when segmenting the bipedal heat map, there are cases where accurate segmentation cannot be achieved. To solve this problem, the following improvements are made:

[0073] 1) Introduce the cyclic residual convolutional module, and train the bipedal heat map with high complexity by specifying the number of convolutional layers in each convolutional module.

[0074] The classic U 2 -Net network architecture is widely used for the segmentation of medical images. However, due to the high complexity of the bipedal heat map processed, the noise contained is fused with the biped, and the temperatures in different regions of the sole are different, sometimes the sole region cannot be completely segmented. To enable a model with low complexity to achieve higher accuracy and reach a balance between accuracy and complexity, so as to extract more complex features. We are based on the classic U 2 -Net network architecture, introduce the cyclic residual convolutional network. The cyclic convolutional network contains several cyclic residual convolutional modules, and use the cyclic residual convolutional modules to replace the ordinary convolutional modules in the original network, so as to deepen the depth of the U 2 -Net network and extract more complex features.

[0075] The cyclic residual convolutional network contains cyclic residual convolutional modules and a max pooling layer (MPL). Each cyclic residual convolutional unit contains a cyclic convolutional layer, and a single cyclic convolutional layer contains multiple cyclic subsequences. By defining the number of cycles T, determine the number of convolutional modules in each cyclic residual convolutional module to extract more complex features.

[0076] The structures of the ordinary convolutional layer, cyclic residual convolutional module, and cyclic subsequence are as Figure 4 shown, Figure 4 where T represents the number of cycles, and its calculation formula is:

[0077]

[0078] where: k is the feature map sequence in the cyclic convolutional layer, l is the serial number of the cyclic convolutional layer in the cyclic residual convolutional module, x is the input feature map, is the weight of the previous cycle output in the kth feature map, is the weight of x in the k-th feature map, f is the activation function, F is the output feature of the Recurrent Convolutional Layer (RCL), O is the feature map output by the recurrent subsequence, T is the number of loops, which determines how many convolutional modules are included in each loop residual convolution module, and b k is the bias compensation.

[0079] Compared with the convolutional module in the classic U 2 -Net network architecture, the loop residual convolution module not only retains the characteristic of fewer original network structure parameters, but also enables the network to extract more complex features.

[0080] 2) Increase the number of dilated convolutions to deepen the sixth layer of the decoder of the U 2 -Net network to obtain higher-dimensional feature information;

[0081] For infrared biped thermal images with extremely poor contrast, as the U 2 -Net architecture network deepens, sometimes the segmentation effect is not obvious, and there is still some noise that cannot be removed. To address this issue, dilated convolutions with a dilation rate of 8 and dilated convolutions with a dilation rate of 16 are added to the sixth layer of the network decoder to increase the receptive field and obtain more complex information.

[0082] The calculation method of the dilated convolution is as follows:

[0083]

[0084] where u is the feature map input to the dilated convolution, v is the convolution coefficient, and R is the dilation rate (the stride of the convolution).

[0085] The classic U 2 -Net architecture network uses pooling layers and convolutional layers to increase the receptive field, but at the same time reduces the size of the feature map. Then, it uses upsampling to restore the image size. The process of reducing and then enlarging the feature map causes a loss of accuracy. Therefore, using dilated convolutions can increase the receptive field while ensuring that the features are not missing.

[0086] 3) Replace the relu activation function with the elu activation function;

[0087] The Relu activation function is defined as:

[0088] relu(x) = max(0, x) (8)

[0089] Since relu is a linear relationship, the calculation speed is very fast, and there is no problem of gradient saturation when the input is positive. However, once the input is negative, it will not be activated, which may lead to the inactivation of neurons. Although this problem has little impact on the forward propagation, during the backpropagation process, the gradient will become 0.

[0090] The elu function is an improved version of the Relu function, and its definition is as follows:

[0091]

[0092] Compared with the Relu function, the elu function can also output values when the input is negative, and has a certain anti-interference ability, which can eliminate the problem of neuron inactivation caused by Relu and improve the segmentation accuracy.

[0093] Suppose the original input image is single-channel 1×320×320. After training with the improved U 2 -Net network, finally, the right decoder of the U-shaped structure outputs 6 groups of feature maps. After processing, 6 Mask binary maps are output with resolutions of: 1×288×288, 1×144×144, 1×72×72, 1×36×36, 1×18×18, 1×9×9. Then, upsampling is performed respectively with upsampling ratios of 1, 2, 4, 8, 16, and 32 to obtain 6 feature maps of 1×288×288. They are fused together to obtain a 6×288×288 feature map, and finally, a convolution is used to convert it into a 1×288×288 convolution.

[0094] The dataset is enhanced by piecewise Gamma transformation. Figure 2 The enhancement results are shown in Figure 5 . On the left, the relative temperature value is larger, β = 1.32, and on the right, the relative temperature value is smaller, β = 0.74. After image enhancement of all experimental data using the above method, two-thirds of the total is selected as the training set, and the remaining one-third is used as the test set. The dataset is labeled to mark the feature regions, and the generated json file after labeling is converted into a Mask map. Through the improved U 2 -Net network (as shown in Figure 6 ) for training to generate an improved U 2 -Net model. Figure 7 For the structure result of the improved U 2 -Net network, on the left is the picture segmented by the improved model, and on the right is the picture segmented by the unimproved model.

[0095] Generally speaking, the technical solution adopted by the present invention includes the following steps:

[0096] 1. Set the acquisition frequency and time length of the thermal imager. Under specific experimental conditions, use an infrared thermal imaging device to obtain a thermal image sequence of the measured foot area, and store the thermal image sequence in a general-purpose memory.

[0097] 2. Visualize the data in the memory. The data in the memory is the radiation value. First, convert the radiation value to temperature, then perform image grayscale conversion according to a fixed temperature window, and then normalize the pixel values so that the pixel values are compressed between 0 and 1.

[0098] 3. Perform piecewise Gamma transformation on the grayscale image according to formula (3) to enhance the contrast. When β < 1, X0 and X1 are relatively close. For overall brightening or darkening processing, perform Gamma transformation on all pixel points with grayscale values between 0 and X0 in the original image: Calculate f0 according to formula (2) and the enhancement factor γ1 according to formula (4), and substitute them into formula (1) to obtain the new grayscale values between 0 and X0. For all pixel points with grayscale values between X0 and 1 in the original image, perform Gamma transformation: As shown in the right part, calculate the cumulative distribution function of the histogram between X0 and 1 and normalize it to obtain X1. Calculate the distance f1 between it and the abscissa according to formula (2), and calculate the enhancement factor γ2 according to formula (4), and substitute them into formula (1) to obtain the new grayscale values between X0 and 1. Figure 3 As shown in the right part, calculate the cumulative distribution function of the histogram between X0 and 1 and normalize it to obtain X1. Calculate the distance f1 between it and the abscissa according to formula (2), and calculate the enhancement factor γ2 according to formula (4), and substitute them into formula (1) to obtain the new grayscale values between X0 and 1. 7. For all pixel points with grayscale values between X0 and 1 in the original image, perform Gamma transformation: As shown in the right part, calculate the cumulative distribution function of the histogram between X0 and 1 and normalize it to obtain X1. Calculate the distance f1 between it and the abscissa according to formula (2), and calculate the enhancement factor γ2 according to formula (4), and substitute them into formula (1) to obtain the new grayscale values between X0 and 1.

[0099] 4. When β > 1, X0 and X1 are relatively far apart. Due to the fixed temperature window, the background is relatively dark. To make the foot area clear and the background darker, perform Gamma transformation on all pixel points with grayscale values between 0 and X0 in the original image: Calculate f0 according to formula (2) and the enhancement factor γ1 according to formula (5), and substitute them into formula (1) to obtain the new grayscale values between 0 and X0. For all pixel points with grayscale values between X0 and 1 in the original image, perform Gamma transformation: As shown in the right part, calculate the cumulative distribution function of the histogram between X0 and 1 and normalize it to obtain X1. Calculate the distance f1 between it and the abscissa according to formula (2), and calculate the enhancement factor γ2 according to formula (5), and substitute them into formula (1) to obtain the new grayscale values between X0 and 1. Figure 3 As shown in the right part, calculate the cumulative distribution function of the histogram between X0 and 1 and normalize it to obtain X1. Calculate the distance f1 between it and the abscissa according to formula (2), and calculate the enhancement factor γ2 according to formula (5), and substitute them into formula (1) to obtain the new grayscale values between X0 and 1. 7. For all pixel points with grayscale values between X0 and 1 in the original image, perform Gamma transformation: As shown in the right part, calculate the cumulative distribution function of the histogram between X0 and 1 and normalize it to obtain X1. Calculate the distance f1 between it and the abscissa according to formula (2), and calculate the enhancement factor γ2 according to formula (5), and substitute them into formula (1) to obtain the new grayscale values between X0 and 1.

[0100] 5. Select two-thirds of the enhanced thermal image sequence as the training set, and the remaining one-third as the test set. Perform annotation processing on the training set data and convert it into a Mask map. Train through the improved U 2 -Net network to generate the improved U 2 -Net model.

[0101] 6. After generating the improved U 2 -Net model, use the test set data for verification. After verification, generate a mask map (i.e., a binary map), extract features from the test set through the mask map, and finally only retain the foot area.

[0102] After being processed through the above steps, the segmented target region is the foot region. When processing the dataset sequence according to the above method, although there may be significant differences in comparison pairs, positions, etc. in these foot regions, the temperature rise differences of the feet may vary greatly. However, in the obtained black-and-white images, all foot regions are correctly segmented into target regions, and at the same time, they correctly reflect the shape of the foot region.

[0103] The following will illustrate the image segmentation processing process of the foot region in the obtained thermal map sequence for realizing the biped thermal map segmentation in combination with embodiments. In this embodiment, the feet are sampled.

[0104] First, cool the patient's plantar surface, and use an infrared thermal imager to record the change of the surface temperature field of the plantar surface in real time, that is, the heating process. The computer collects the thermal map data obtained by the infrared thermal imager to obtain a thermal map sequence of the plantar surface temperature field. As the temperature of the plantar surface continuously rises, a distinct temperature difference is formed with the diseased area. The collected thermal map data is infrared radiation (i.e., radiation value), and the radiation value needs to be converted into temperature. Then, according to a fixed temperature window, the pixel values are normalized, and the pixel values are compressed between 0 and 1.

[0105] After determining the grayscale map sequence, calculate the histogram of each image, and then statistically calculate its cumulative distribution function. Calculate the enhancement factor γ of each image. First, obtain X0 and X1 as shown in Figure 3 , then calculate β according to formula (3), and then perform segmented Gamma transformation processing according to the value of β:

[0106] When β < 1, X0 and X1 are relatively close, and the overall image is brightened or darkened. The enhancement method is as follows: For all pixel points with grayscale values between 0 and X0 in the original image, perform Gamma transformation: Calculate f0 according to formula (2) and the enhancement factor γ1 according to formula (4), and substitute them into formula (1) to obtain the new grayscale values between 0 and X0. For all pixel points with grayscale values between X0 and 1 in the original image, perform Gamma transformation: As shown in the upper right figure Figure 3 for the cumulative distribution function of the normalized histogram from X0 to 1, calculate the distance f1 between X1 and the abscissa , and then substitute it into formula (4) to calculate the enhancement factor γ2, and substitute it into formula (1) to obtain the new grayscale values from X0 to 1.

[0107] When β > 1, X0 and X1 are relatively far apart. Due to the fixed temperature window, the background is dark. To make the foot region clear and the background darker, the enhancement method is as follows:

[0108] Perform Gamma transformation on all pixel points in the original image with gray values ranging from 0 to X0: Calculate f0 according to formula (2) and the enhancement factor γ1 according to formula (5), and substitute them into formula (1) to obtain the new gray values from 0 to X0. Perform Gamma transformation on all pixel points in the original image with gray values ranging from X0 to 1: As Figure 3 shown in the normalized histogram cumulative distribution function from X0 to 1 in the upper right figure above, calculate the distance f1 between X1 and the abscissa axis according to formula (2), then substitute it into formula (5) to calculate the enhancement factor γ2, and substitute it into formula (1) to obtain the new gray values from X0 to 1.

[0109] Test the enhanced gray image sequence through the improved U 2 -Net network model to generate a mask image (i.e., a binary image), extract features from the data set through the mask image, and finally only retain the foot area.

[0110] Perform similar infrared experiments on the data set using the same methods and steps as above, and process the obtained gray image sequence. The results show that: all the segmented images can correctly classify the diseased areas into the target areas and correctly reflect the shape of the feet at the same time. Since all the parameters in the above steps are automatically calculated, therefore, this method can achieve automatic processing.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the purpose and scope of the present technical solution, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A self-enhancement method based on Gamma transformation, characterized in that: The method includes the following steps: S1: Obtain a sequence of thermal images of the surface of the foot to be measured using an infrared thermal imaging device, and store the sequence of thermal images in a general-purpose memory; S2: Image the data in the memory. The data in the memory is radiation value. First, convert the radiation value to temperature, perform image grayscale conversion on the temperature according to a fixed temperature window, and then normalize the pixel grayscale values so that the pixel values are compressed between 0 and 1; S3: Obtain the histogram of each picture, then calculate its cumulative distribution function, and normalize its ordinate. On the obtained normalized cumulative distribution function curve, select the pixel value X0 corresponding to the median value of the ordinate, that is, 0.5; S4: Judge the position of the pixel value X0 corresponding to the median value of the ordinate, that is, 0.5, in the histogram, and calculate the distance between X0 and the median value of the abscissa, that is: f = 0.5 - X0 (1) S5: Based on the value of f, calculate the Gamma enhancement factor according to the change of the foot temperature; The specific content of S5 is as follows: S51: The judgment standard for the relative temperature value is: S52: When the relative temperature value is small, use formula (3) to calculate the Gamma enhancement factor: S53: When the relative temperature value is large, use formula (4) to calculate the Gamma enhancement factor: S54: Gamma transformation: S6: Calculate β according to formula (2) based on X0 and X1, and then perform segmented Gamma transformation processing according to the value of β; S7: When β < 1, the difference between X0 and X1 is small, and the overall image is brightened or darkened. The enhancement method is as follows: For all pixel points with grayscale values between 0 and X0 in the original image, perform Gamma transformation: Calculate f0 according to formula (5) and calculate the enhancement factor γ1 according to formula (3), and substitute them into formula (1) to obtain the new grayscale values between 0 and X0; S8: Perform Gamma transformation on all pixel points with gray values ranging from X0 to 1 in the original image: Calculate the distance f1 between X1 and the abscissa according to formula (5), then substitute it into formula (3) to calculate the enhancement factor γ2, and substitute it into formula (1) to obtain the new gray values from X0 to 1; between, and then substitute it into formula (3) to calculate the enhancement factor γ2, and substitute it into formula (1) to obtain the new gray values from X0 to 1; S9: When β > 1, the difference between X0 and X1 is large. Due to the fixed temperature window, the background is dark. To make the foot area clear and the background darken, the enhancement method is as follows. For all pixel points with grayscale values between 0 and X0 in the original image, perform Gamma transformation: Calculate f0 according to formula (5) and calculate the enhancement factor γ1 according to formula (4), and substitute them into formula (1) to obtain the new grayscale values between 0 and X0; S10: Perform Gamma transformation on all pixel points in the original image with gray values ranging from X0 to 1: Calculate the distance f1 between X1 and the abscissa according to formula (2), then substitute it into formula (4) to calculate the enhancement factor γ2, and substitute it into formula (1) to obtain the new gray values from X0 to 1; between, and then substitute it into formula (4) to calculate the enhancement factor γ2, and substitute it into formula (1) to obtain the new gray values from X0 to 1; S11: Invoke the cyclic residual convolution module, and train the biped thermal image with high complexity by specifying the number of convolutional layers included in each convolutional module; S12: Increase the number of dilated convolutions to deepen the U-Net network's sixth decoding layer and obtain higher-dimensional feature information; 2 ​ S13: Replace the relu activation function with the elu activation function; S14: Enhance the data set through segmented Gamma transformation; After image enhancement of all test data, select two-thirds of the total as the training set, and the remaining one-third as the test set; Perform annotation processing on the data set, mark the feature areas, and convert the generated json file after annotation into a Mask map; S15: The training set is trained using the network described in S11 - S14 to obtain an improved U - Net segmentation model. The test set data is used for verification to generate a mask image, that is, a binary image. Feature extraction is performed on the test set through the mask image, and finally the foot region is obtained. 2 -Net segmentation model, the test set data is used for verification to generate a mask image, that is, a binary image. Feature extraction is performed on the test set through the mask image, and finally the foot region is obtained.

Images