Foot wound surface detection method and system based on image recognition

Abstract

The invention discloses a foot wound surface detection method and system based on image recognition, and relates to the technical field of medical image processing and analysis, and the method comprises the steps: extracting foot wound surface features of high-resolution foot wound surface images under different scales through a multi-scale feature fusion network, constructing pyramid hierarchies, and obtaining a multi-scale feature fusion network; feature fusion is carried out from top to bottom through the feature pyramid structure, and a multi-scale foot wound feature map is generated; inputting the multi-scale foot wound feature map into a segmentation network for cross-modal interaction and fusion, obtaining a regional physiological characteristic map and a regional probability map by using a deformable convolutional network and a decoding network, and comparing the regional physiological characteristic map and the regional probability map through an adaptive threshold map to generate a binary mask foot wound image; according to the method, the segmentation accuracy is improved, and detailed wound surface information is also provided, so that the accuracy and efficiency of clinical diagnosis are greatly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing and analysis, in particular to a foot wound detection method and system based on image recognition. BACKGROUND

[0002] In recent years, with the deep integration of medical imaging technology and artificial intelligence, image recognition-based wound detection methods have shown important value in the management of chronic wounds such as diabetic foot and pressure ulcers. Traditional wound assessment mainly relies on visual inspection or two-dimensional measurement tools by clinicians, which has strong subjectivity and poor repeatability. In the prior art, multispectral imaging and near-infrared photography have been applied to the visualization analysis of wound depth and tissue, and the wide application of convolutional neural networks (CNN) in the field of medical image segmentation (such as U-Net, FCN, etc.) has further improved the accuracy of automated detection.

[0003] The limitations of current technology mainly reflect that low-resolution images (such as clinical images captured by mobile phone cameras) are prone to introduce artifacts in the super-resolution reconstruction process, and traditional interpolation algorithms (such as bicubic interpolation) are difficult to preserve the biological tissue characteristics of the wound area, affecting the input quality of the subsequent segmentation model; these problems seriously restrict the practicality and reliability of the wound detection system in primary medical settings. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides a foot wound detection method based on image recognition to solve the problem of biological tissue feature loss caused by artifacts in the low-resolution foot wound image reconstruction process.

[0006] To solve the above technical problems, the present application provides the following technical solutions: In a first aspect, the present application provides a foot wound detection method based on image recognition, which comprises collecting and preprocessing multi-band foot wound images, the multi-band foot wound images including visible light band wound images, near-infrared band wound images, short-wave infrared band wound images and thermal infrared band wound images; Based on a super-resolution convolutional neural network model and historical multi-band foot wound images, a super-resolution reconstruction model is constructed, the preprocessed multi-band foot wound images are separated by band, fast up-sampling is performed using bicubic interpolation, and the super-resolution reconstruction model is inputted for super-resolution reconstruction to obtain high-resolution foot wound images; Through a multi-scale feature fusion network, foot wound features of high-resolution foot wound images at different scales are extracted to construct a pyramid level, and through a feature pyramid structure, feature fusion is performed from top to bottom to generate multi-scale foot wound feature maps; The multiscale foot wound feature map is input into the segmentation network for cross-modal interaction and fusion, a region physiological characteristic map and a region probability map are obtained by using a deformable convolution network and a decoding network, and comparison is performed through a self-adaptive threshold map to generate a binary mask foot wound image; The binary mask foot wound image is superimposed with the high-resolution foot wound image through OpenCV, and a red contour is marked on the wound area through a contour detection algorithm to obtain a foot wound area annotation map.

[0007] As a preferred scheme of the foot wound detection method based on image recognition, the preprocessing includes image denoising, multi-band registration, radiation normalization and band alignment.

[0008] As a preferred scheme of the foot wound detection method based on image recognition, the super-resolution convolutional neural network model is used as a basic model, and a super-resolution reconstruction model is constructed, and the specific steps are as follows, The super-resolution convolutional neural network model is used as a basic model. The extended 4-channel input layer receives historical multi-band foot wound images; The feature extraction layer extracts local features through a small convolution kernel; The non-linear mapping layer compresses the feature dimension and fuses the cross-band information; The cross-band feature interaction layer dynamically calculates the weight of each band and highlights important band features; The image reconstruction layer integrates global context information by using a large convolution kernel to output a high-resolution image. The super-resolution reconstruction model is constructed.

[0009] As a preferred scheme of the foot wound detection method based on image recognition, the high-resolution foot wound image is obtained, and the specific steps are as follows, The preprocessed multi-band foot wound image is separated by band, and bicubic interpolation is used for fast up-sampling to preliminarily enlarge each band image to the target size, and the up-sampled 4-band images are merged into a 4-channel tensor; The 4-channel tensor is input into the super-resolution reconstruction model, and is processed in sequence through the feature extraction layer, the non-linear mapping layer, the cross-band feature interaction layer and the image reconstruction layer to output a 4-channel high-resolution tensor image; The 4-channel high-resolution tensor image is subjected to cross-band gradient alignment, and the wound edge is strengthened through adaptive sharpening filtering to obtain a high-resolution foot wound image.

[0010] As a preferred scheme of the foot wound detection method based on image recognition, the multiscale foot wound feature map is generated, and the specific steps are as follows, Based on the high-resolution foot wound image, high-resolution epidermis features, medium-resolution subcutaneous features and low-resolution necrosis features are extracted respectively; Based on the high-resolution epidermis features, medium-resolution subcutaneous features and low-resolution necrosis features, a feature pyramid level is constructed; The feature pyramid level is fused from top to bottom to generate a multi-scale foot wound feature map.

[0011] As a preferred scheme of the foot wound detection method based on image recognition, a binary mask foot wound image is generated, and the specific steps are as follows, The multi-scale foot wound feature map is cross-modally interacted with the 4-channel high-resolution tensor image after super-resolution reconstruction, the waveband weight is dynamically allocated and fused through the waveband correlation attention mechanism; Based on the multi-modal fusion feature map, edge guidance and region proposal are performed through a deformable convolution network to generate a wound region proposal, local features are extracted and decoded to obtain a region physiological property map; The region physiological property map is up-sampled to the original image size through the decoding network, and a region probability map is generated through Sigmoid activation; An adaptive threshold map is generated according to the region physiological property map, and is compared with the region probability map, the region probability map greater than the adaptive threshold map is binarized to generate a binary mask foot wound image.

[0012] As a preferred scheme of the foot wound detection method based on image recognition, a foot wound region annotation map is obtained, and the specific steps are as follows, All closed contours in the binary mask foot wound image are detected using OpenCV, and small contours are filtered through a wound area threshold; The high-resolution foot wound image is converted into a BGR three-channel format, and the drawContours function is used to fill the filtered contours in red, and a foot wound region annotation map with red contour annotation is output.

[0013] In a second aspect, the present application provides an image recognition-based foot wound detection system, comprising a data acquisition module configured to acquire multi-band foot wound images and perform preprocessing, wherein the multi-band foot wound images comprise visible light band wound images, near-infrared band wound images, short-wave infrared band wound images and thermal infrared band wound images; an ultra-resolution reconstruction module configured to construct an ultra-resolution reconstruction model based on an ultra-resolution convolutional neural network model and historical multi-band foot wound images, separate the preprocessed multi-band foot wound images by band, perform fast up-sampling using bicubic interpolation, and input the same into the ultra-resolution reconstruction model for ultra-resolution reconstruction to obtain high-resolution foot wound images; a fusion module configured to extract foot wound features of the high-resolution foot wound images at different scales to construct a pyramid level through a multi-scale feature fusion network, and perform feature fusion from top to bottom through a feature pyramid structure to generate multi-scale foot wound feature maps; an accurate segmentation module configured to input the multi-scale foot wound feature maps into a segmentation network for cross-modal interaction and fusion, acquire regional physiological property maps and regional probability maps using a deformable convolution network and a decoding network, and compare the same through an adaptive threshold map to generate a binary mask foot wound image; and a labeling module configured to superimpose the binary mask foot wound image and the high-resolution foot wound image through OpenCV, and mark a wound region with a red contour through a contour detection algorithm to obtain a foot wound region labeling map.

[0014] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and wherein the computer program, when executed by the processor, implements any step of the image recognition-based foot wound detection method according to the first aspect of the present application.

[0015] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any step of the image recognition-based foot wound detection method according to the first aspect of the present application.

[0016] The present application has the following beneficial effects: the ultra-resolution reconstruction model constructed based on the ultra-resolution convolutional neural network model realizes high-quality up-sampling of the multi-band foot wound images, significantly improves the resolution of low-quality images and the understanding ability of complex wound conditions, and lays a foundation for subsequent accurate segmentation; meanwhile, the multi-scale feature fusion network is used to extract and fuse features at different scales, fully captures various detail features of the wound region, improves the accuracy of segmentation, provides detailed wound information, helps to develop individualized treatment plans, and thus greatly improves the accuracy and efficiency of clinical diagnosis. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0018] Fig. 1 Flow chart of the image recognition-based foot wound detection method.

[0019] Fig. 2 Schematic diagram of the image recognition-based foot wound detection system.

[0020] Fig. 3 Flow chart of generating a multi-scale foot wound feature map.

[0021] Fig. 4 Flow chart of super-resolution reconstruction. DETAILED DESCRIPTION

[0022] In order to make the above objectives, features and advantages of the present application more apparent and comprehensible, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0023] In the following description, a large number of specific details are set forth in order to facilitate a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of the present application, so the present application is not limited to the specific embodiments disclosed below.

[0024] Secondly, the "one embodiment" or "embodiment" referred to herein means that the specific features, structures or characteristics can be included in at least one implementation of the present application. "In one embodiment" appearing in different places in the specification does not mean the same embodiment, nor is it an independent or alternative embodiment that excludes other embodiments.

[0025] Reference Figs. 1-4 The embodiment provides an image recognition-based foot wound detection method, which comprises the following steps: S1, collecting multi-band foot wound images and performing pretreatment, wherein the multi-band foot wound images comprise visible light band wound images, near-infrared band wound images, short-wave infrared band wound images and thermal infrared band wound images.

[0026] S1.1, the pretreatment comprises image denoising, multi-band registration, radiation normalization and band alignment.

[0027] Further, image denoising: non-local mean denoising is respectively performed on the visible light band wound image, the near-infrared band wound image, the short-wave infrared band wound image and the thermal infrared band wound image, for example, the search window is set to 21x21 pixels, the similar block window is 7x7 pixels, and the filtering strength h=10.

[0028] Multi-band registration: using scale invariant feature transform (SIFT) to extract key points of the visible light band wound image, and matching corresponding feature points on the near-infrared band wound image, the short-wave infrared band wound image and the thermal infrared band wound image, calculating the affine transformation matrix, for example, using RANSAC algorithm to remove mis-matching points, and finally completing image registration through bilinear interpolation.

[0029] Radiation normalization: linear correction is performed on the thermal infrared band wound image based on blackbody radiation calibration, for example, using two-point correction method, setting low temperature reference value and high temperature reference value, mapping the original radiation value to the standard temperature range, and performing histogram matching on the visible light band wound image, the near-infrared band wound image and the short-wave infrared band wound image to make the light distribution consistent.

[0030] Band alignment: converting the registered multi-band foot wound image to the same spatial resolution, using bicubic interpolation to uniformly scale to 512x512 pixels, and stacking them in the order of bands to form a 4-channel tensor, ensuring that the pixel positions of each band correspond strictly.

[0031] S2, based on the super-resolution convolutional neural network model and the historical multi-band foot wound image, a super-resolution reconstruction model is constructed.

[0032] S2.1, taking the super-resolution convolutional neural network model as the basic model; The extended 4-channel input layer receives the historical multi-band foot wound image; The feature extraction layer extracts local features through small convolution kernels; The non-linear mapping layer compresses the feature dimension and fuses the cross-band information; The cross-band feature interaction layer dynamically calculates the weight of each band and highlights important band features; The image reconstruction layer integrates global context information using large convolution kernels and outputs high-resolution images; The super-resolution reconstruction model is constructed.

[0033] It should be noted that: Extended 4-channel input layer: stack the preprocessed visible light band wound image, near-infrared band wound image, short-wave infrared band wound image and thermal infrared band wound image by channel dimension to form a 4xHxW input tensor, where H and W are the spatial dimensions of the image (e.g. 512x512 pixels), and the input tensor data type is single-precision floating point type; Feature extraction layer operation: Convolution operation is performed on the input tensor using a 3x3 convolution kernel (e.g. 64 filters, stride 1, padding='same'), followed by a batch normalization layer and a ReLU activation function after each convolution layer, and 3 such convolution blocks are stacked in succession to gradually extract multi-scale local features; Nonlinear mapping layer operation: The feature channel dimension is compressed by a 1x1 convolution kernel (e.g. 32 filters) to map the 64-channel multi-scale local features to a 32-dimensional space, and a ReLU activation function is used to realize nonlinear transformation while preserving the correlation of cross-band features; Cross-band feature interaction layer operation: Global average pooling is performed on each band feature map to obtain a 1x1xC description vector, which is processed by two fully connected layers (e.g. the first fully connected layer outputs C / 16 channels, and the second restores to C channels) to analyze the contribution of each band feature to the current task, and a Sigmoid function is used to generate a band weight matrix in the range of 0-1, and the band with a weight value greater than 0.5 is determined as an important feature (e.g. the thermal infrared band may obtain a weight of 0.9 in the inflammation area), and finally the original feature map is multiplied by the corresponding weight matrix channel by channel to realize the strong expression of important band features; Image reconstruction layer operation: A 9x9 large convolution kernel (e.g. 4 filters) is used to integrate the weighted multi-band features, and PixelShuffle up-sampling (e.g. factor 2) is used to expand the feature map spatial size to the target resolution, outputting a 4-channel high-resolution image with the same band order and data type as the input; Furthermore, the input layer is expanded to 4 channels to fully utilize the complementary advantages of multi-band spectrum, visible light provides high-resolution morphological features, near-infrared detects subcutaneous tissue changes, short-wave infrared reflects tissue biochemical properties, and thermal infrared captures metabolic abnormalities; This multi-dimensional information fusion can comprehensively cover the multi-scale features required for wound diagnosis, such as epidermis, subcutaneous tissue, and metabolism, making the super-resolution reconstruction model have more complete pathological analysis capability and significantly improving the detection accuracy; The input layer is expanded to 4 channels by modifying the channel dimension of the traditional CNN input layer, specifically using a multi-channel convolution kernel (such as Conv2d(in_channels=4)) in frameworks such as PyTorch / TensorFlow, and the registered and aligned visible light, near-infrared, short-wave infrared, and thermal infrared images are used as independent input channels, and a 4D tensor is formed by parallel stacking (torch.cat or tf.concat); The core role of the cross-band feature interaction layer is to dynamically optimize the contribution of different band features, suppress redundant information interference, and enhance the expression of key wound features. This design can effectively maintain the consistency between bands, avoid spectral distortion in the reconstruction process, and at the same time introduce a band correlation constraint; The cross-band feature interaction layer is implemented based on an improved Squeeze-and-Excitation (SE) attention mechanism. The spatial information of each band feature map is compressed by global average pooling to obtain a channel description vector. A bottleneck structure composed of two fully connected layers (the intermediate layer dimension is reduced to 1 / 16) is used to learn the nonlinear relationship between bands. A sigmoid activation is used to generate adaptive weights for channel weighting of the original features. A band correlation loss function (Band Correlation Loss) is added to strengthen the feature collaboration between near-infrared and thermal infrared and other medical-related bands.

[0034] S3, separate the preprocessed multi-band foot wound image by band, use bicubic interpolation for fast upsampling, and input the super-resolution reconstruction model for super-resolution reconstruction to obtain a high-resolution foot wound image.

[0035] S3.1, separate the preprocessed multi-band foot wound image by band, use bicubic interpolation for fast upsampling, and input the super-resolution reconstruction model for super-resolution reconstruction to obtain a high-resolution foot wound image.

[0036] Further, band separation: split the 4-channel multi-band foot wound image into independent single-channel images. Use the split function of OpenCV to decompose the input 4-channel multi-band foot wound image into four independent single-channel images. Extract the visible light band wound image, near-infrared band wound image, short-wave infrared band wound image, and thermal infrared band wound image, respectively, and store them as single-precision floating-point matrices. Bicubic interpolation upsampling: perform bicubic interpolation enlargement on each single-band image. The input image size is 256x256 pixels, and the target size is 512x512 pixels. When calculating the interpolation weight, use a bicubic kernel function (e.g., 16-neighborhood weighting for -0.5≤x≤1.5). Calculate the new pixel value after interpolation pixel by pixel. Merge 4-channel tensor: stack the upscaled visible light band wound image, near-infrared band wound image, short-wave infrared band wound image, and thermal infrared band wound image by channel dimension to form a 4x512x512 floating-point tensor.

[0037] S3.2, input the 4-channel tensor into the super-resolution reconstruction model, sequentially pass through the feature extraction layer, the nonlinear mapping layer, the cross-band feature interaction layer and the image reconstruction layer for processing, and output a 4-channel high-resolution tensor image.

[0038] Further, the 4-channel tensor is input into the feature extraction layer, local features are extracted using a 3x3 convolution kernel (e.g. 64 filters, stride 1, padding='same'), the output is sent to the nonlinear mapping layer, feature dimension reduction is achieved through 1x1 convolution (e.g. compressed to 32 channels), and a ReLU activation function is applied, then in the cross-band feature interaction layer, global average pooling is performed on each band feature map to compress the spatial dimension and generate a 1x1xC channel description vector; the channel description vector is input into a bottleneck structure composed of two fully connected layers, the first fully connected layer reduces the channel number to 1 / 16 (e.g. 4 channels), uses a ReLU activation, and the second fully connected layer restores the original channel number (e.g. 4 channels 64 channels), generates a channel weight in the range of 0-1 through a Sigmoid function, multiplies the channel weight with the original feature map channel by channel, and additionally calculates the inter-band cosine similarity as a regularization term added to the loss function, forcing medical-related bands such as near-infrared and thermal infrared to maintain feature consistency, using a 9x9 large convolution kernel (e.g. 4 filters) in the image reconstruction layer to integrate global information, and outputting a 4-channel high-resolution tensor image through PixelShuffle upsampling (e.g. 2 times magnification), keeping the order of the input and output bands consistent.

[0039] S3.3, align the gradients across bands for the 4-channel high-resolution tensor image, and enhance the wound edge through adaptive sharpening filtering to obtain a high-resolution foot wound image.

[0040] Further, the Sobel gradient amplitude map (using a 3x3 kernel, dx=1, dy=1) of the visible light band wound image, the near-infrared band wound image, the short-wave infrared band wound image and the thermal infrared band wound image is calculated, the gradient direction field of the other three bands is adjusted based on the visible light band gradient (e.g. using the least squares method to solve the transformation matrix), the aligned multi-band gradient map is weighted and fused (e.g. the weight distribution is visible light 0.5, near-infrared 0.3, short-wave infrared 0.1, and thermal infrared 0.1), and finally a local variance-based adaptive sharpening filter is used to enhance the edge in the gradient significant area (e.g. variance greater than 10) using a 3x3 Laplacian kernel, and a final high-resolution foot wound image is output.

[0041] S4, through the multi-scale feature fusion network, the foot wound features of the high-resolution foot wound image at different scales are extracted to construct a pyramid level, and the features are fused from top to bottom through the feature pyramid structure to generate a multi-scale foot wound feature map.

[0042] S4.1, based on high-resolution foot wound image, high-resolution epidermis features, medium-resolution subcutaneous features and low-resolution necrosis features are extracted respectively.

[0043] Further, high-resolution epidermis features are extracted from the high-resolution foot wound image using a 3x3 convolution kernel (step 1, 64 filters), which retains complete spatial details. After down-sampling through 2x2 max pooling (step 2), medium-resolution subcutaneous features are extracted using a 3x3 convolution kernel (step 1, 128 filters), which captures subcutaneous tissue information. After further 2x2 max pooling (step 2), low-resolution necrosis features are extracted using a 3x3 dilated convolution (dilation rate 2, 256 filters), which expands the receptive field to cover deep necrotic areas.

[0044] S4.2, based on high-resolution epidermis features, medium-resolution subcutaneous features and low-resolution necrosis features, a feature pyramid hierarchy is constructed.

[0045] Further, the high-resolution epidermis features (original size) are directly used as the base-level feature map, the medium-resolution subcutaneous features (1 / 2 size) are up-sampled to the original size through bilinear interpolation, and the low-resolution necrosis features (1 / 4 size) are first up-sampled to 1 / 2 size through 2 times bilinear interpolation, and then up-sampled to the original size through 2 times bilinear interpolation, serving as the high-level feature map. The three levels of feature maps maintain the same spatial size, with channel numbers of 64 (base level), 128 (intermediate level) and 256 (high level) respectively, forming a complete feature pyramid structure.

[0046] S4.3, the feature pyramid hierarchy is fused from top to bottom to generate a multi-scale foot wound feature map.

[0047] Further, the low-resolution necrosis features (high level) are processed through 3x3 convolution (256 filters), the output feature map is up-sampled by 2 times through bilinear interpolation, the up-sampled result is added element by element with the medium-resolution subcutaneous features (intermediate level), and then processed through 3x3 convolution (128 filters), the output feature map is again up-sampled by 2 times through bilinear interpolation, and added element by element with the high-resolution epidermis features (base level), and the multi-scale information is integrated through 3x3 convolution (64 filters) to generate a multi-scale foot wound feature map (size HxWx64) containing necrotic tissue, subcutaneous features and epidermis details.

[0048] S5, the multi-scale foot wound feature map is input into the segmentation network for cross-modal interaction and fusion, the regional physiological property map and the regional probability map are obtained using the deformable convolution network and the decoding network, and the adaptive threshold map is compared to generate a binary mask foot wound image.

[0049] S5.1. Cross-modal interaction is performed between the multi-scale foot wound feature map and the super-resolution reconstructed 4-channel high-resolution tensor image. The weights of each band are dynamically allocated and fused through the band correlation attention mechanism.

[0050] Specifically as follows: The multi-scale foot wound feature map (size H×W×64) was adjusted to 64 channels through 1×1 convolution, and the 4-channel high-resolution tensor image (size 4×H×W) after super-resolution reconstruction was expanded to 64 channels through 3×3 convolution. Global average pooling was performed on the multi-scale foot wound feature map to obtain a 1×1×64 query vector, and global average pooling was performed on each band feature of the 4-channel high-resolution tensor image to obtain 4 1×1×64 key vectors. The dot product of the query matrix output by the multi-scale feature pyramid (FPN) and each key vector is calculated and then normalized using Softmax to obtain the weights for four bands (e.g., visible light 0.35, near-infrared 0.25, short-wave infrared 0.20, thermal infrared 0.20). The expression is as follows: ; in, It is the first Dynamic weighting of bands It is the query matrix output by the Multi-Scale Feature Pyramid (FPN). It is the first The key matrix of the band, It is the matrix transpose operator. It is a scaling factor. The band index identifier specifically refers to the first band in the multimodal image. Wavelength bands (such as visible light, near-infrared, thermal infrared, etc.); The feature map of each band of the 4-channel high-resolution tensor image is multiplied by its corresponding band weight, then summed element-wise and fused to output a fused 64-channel feature map (size H×W×64), expressed as: ; in, It is a multimodal fusion feature map. It is the first The original feature map of the band (such as the features corresponding to the visible light / near infrared / thermal infrared bands).

[0051] S5.2 Based on multimodal fusion feature maps, edge guidance and region proposal are performed through deformable convolutional networks to generate wound region proposals, extract local features and decode them to obtain regional physiological characteristic maps.

[0052] Specifically as follows: The multimodal fusion feature map is converted into a single-channel grayscale image by averaging the values ​​of each channel. A Gaussian filter is used for smoothing and denoising. 3×3 horizontal and vertical Sobel convolution kernels are used to calculate the gradient values ​​of the image in the horizontal and vertical directions, respectively. The gradient intensity and direction of each pixel are calculated using the gradient values ​​in these two directions. Non-maximum suppression is performed on the gradient intensity image to retain local maximum gradient values. A high and low dual threshold strategy is used to binarize the image. The high threshold is used to identify strong edges, and the low threshold is used to connect weak edges. The final output is a binary edge mask that contains only edge information. For each channel of the multimodal fusion feature map, the gradient values ​​in the horizontal and vertical directions are calculated separately. During the calculation, convolution operations are performed using horizontal and vertical convolution kernels. The horizontal and vertical gradient values ​​of all channels are averaged to obtain the global horizontal gradient image and vertical gradient image. The gradient images in these two directions are merged into a Sobel gradient field of a two-channel multimodal fusion feature map. The Sobel gradient field of the multimodal fusion feature map is multiplied element-wise with the binary edge mask and then input into a 3×3 convolutional layer to generate the offset of the deformable convolutional kernel. The expression is as follows: ; in, It is the first The offset of each convolutional kernel It is a gradient field of multimodal fusion feature map. It is a binary edge mask. It is the index variable of the convolution kernel; use Adjust the sampling position of the standard convolution kernel, perform deformable convolution (3×3 kernel size, 256 output channels) on the original multimodal fusion feature map, and output a feature map containing wound region candidates. Compress the feature map into a region physiological characteristic map with the number of channels equal to the number of categories through 1×1 convolution.

[0053] S5.3. The regional physiological characteristic map is sampled to the original image size through a decoding network, and a regional probability map is generated by Sigmoid activation.

[0054] It should be noted that the region physiological characteristic map is input into a decoding network composed of two transposed convolution layers, the first transposed convolution layer uses a 3x3 convolution kernel, a step of 2 and 128 output channels, expands the region physiological characteristic map size by 2 times, and the second transposed convolution layer uses the same parameters but the output channel is changed to 64, and is expanded by 2 times again to the original image size. After each upsampling, the corresponding scale region physiological characteristic map (medium resolution subcutaneous features and high resolution epidermis features) is added channel by channel. Finally, a 1x1 convolution kernel is used to compress the channel number to 1, and the output value is mapped to the range of 0-1 through the Sigmoid function, to generate a region probability map with the same size as the input image.

[0055] S5.4, generate an adaptive threshold map according to the region physiological characteristic map, and compare it with the region probability map, binarize the region probability map greater than the adaptive threshold map, and generate a binary mask foot wound image.

[0056] Specifically as follows: The blood oxygen saturation feature and the thermal infrared temperature feature are extracted from the region physiological characteristic map, and are normalized to the range of [0, 1] respectively, and the two features are weighted using the predefined weight parameters =0.6 and =0.4 to generate an adaptive threshold map, the expression is: ; Wherein, is the adaptive threshold (range [0, 1]) at the coordinate , is a Sigmoid function, is the contribution weight of blood oxygen saturation to the threshold (optimized by back propagation), is the contribution weight of temperature to the threshold (optimized by back propagation), is the temperature value at the coordinate (decoded from the thermal infrared band feature, which needs to be normalized to [0, 1]), is the blood oxygen saturation value at the coordinate (decoded from the near-infrared band feature, which needs to be normalized to [0, 1]); The region probability map (numerical range 0-1) is compared with the adaptive threshold map (numerical range 0-1) pixel by pixel; for each pixel position , when the value of in the region probability map is greater than When the value of the binary mask foot wound image is 1, the corresponding position of the binary mask foot wound image is assigned a value of 1, indicating a wound area; otherwise, it is assigned a value of 0, indicating normal tissue; finally, a binary image containing only 0 and 1 is output, in which the white area (value 1) represents the detected wound area, and the black area (value 0) represents normal tissue.

[0057] S6. Superimpose the binary mask foot wound image and the high-resolution foot wound image by OpenCV, and use a contour detection algorithm to mark the wound area with a red contour to obtain a foot wound area annotation map.

[0058] S6.1, use OpenCV to detect all closed contours in the binary mask foot wound image, and filter small contours by a wound area threshold.

[0059] Further, the cv2.findContours function is called to process the binary mask foot wound image, extract the outermost contour and compress redundant contour points. The input image needs to be a single-channel 8-bit binary format, and after execution, the contour coordinate list (N x 1 x 2 NumPy array) and hierarchical relationship are returned. The contour point set obtained is the outer boundary coordinates of all wound areas. By iterating through each contour, the contour area is calculated using cv2.contourArea, and the contour area is compared with the preset wound area threshold (e.g. 100 square pixels). Contours with an area less than the wound area threshold are deleted, and contour point sets that meet the area requirement are retained. The filtered contour list is output. It should be noted that the preset wound area threshold is specifically, the historical wound area is counted, an area distribution histogram is drawn, and the 5% quantile of the historical wound area distribution is calculated as the initial reference value (e.g. a typical value of 85 pixels is measured in a 512 x 512 image). Test multiple candidate thresholds (60 / 85 / 110 pixels) on the validation set, and calculate the number of false negatives (false negatives) and the number of false positives (false positives) for each candidate threshold. Finally, select the threshold that minimizes the sum of false negatives and false positives (e.g. 85 pixels) as the final wound area threshold.

[0060] S6.2, convert the high-resolution foot wound image to BGR three-channel format, and use the drawContours function to fill the filtered contours with red color, and output the foot wound area annotation map with red contour marking.

[0061] Further, the high-resolution foot wound image is processed using the cv2.cvtColor function, the input parameters are the original single-channel grayscale image and the conversion code, and after execution, the single-channel pixel value is copied to the corresponding position of the BGR three-channel (B=G=R=gray value), and the output is a three-channel BGR format image (the size is the same as the input, and the data type remains uint8), which completes the conversion of the grayscale image to the color space. The cv2.drawContours function is used to set the contour line color to red (BGR value 0, 0, 255) and line width to 2 pixels, and the filtered contour is drawn. The cv2.fillPoly function is used to fill the contour inside area with a semi-transparent red color (BGR value 0, 0, 255, transparency alpha=0.3), and the output is a foot wound area annotation image containing the original image information and red annotation. The image size is exactly the same as the input high-resolution foot wound image.

[0062] The embodiment also provides a foot wound detection system based on image recognition, comprising: A data acquisition module is configured to acquire multi-band foot wound images and perform preprocessing. The multi-band foot wound images include visible light band wound images, near-infrared band wound images, short-wave infrared band wound images, and thermal infrared band wound images. An super-resolution reconstruction module is configured to construct a super-resolution reconstruction model based on a super-resolution convolutional neural network model and historical multi-band foot wound images, separate the preprocessed multi-band foot wound images by band, perform fast upsampling by bicubic interpolation, and input the super-resolution reconstruction model for super-resolution reconstruction to obtain high-resolution foot wound images. A fusion module is configured to extract foot wound features of the high-resolution foot wound images at different scales to construct a pyramid level through a multi-scale feature fusion network, and generate a multi-scale foot wound feature map through feature fusion from top to bottom according to a feature pyramid structure. An accurate segmentation module is configured to input the multi-scale foot wound feature map into a segmentation network for cross-modal interaction and fusion, obtain a regional physiological property map and a regional probability map by using a deformable convolutional network and a decoding network, and generate a binary mask foot wound image by comparing an adaptive threshold map. A labeling module is configured to superimpose the binary mask foot wound image and the high-resolution foot wound image by using OpenCV, mark a wound area with a red contour by using a contour detection algorithm, and obtain a foot wound area annotation image.

[0063] The embodiment also provides a computer device suitable for the foot wound detection method based on image recognition, comprising a memory and a processor. The memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions to implement the foot wound detection method based on image recognition as described in the above embodiment.

[0064] The computer device can be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected by a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is configured to perform wired or wireless communication with an external terminal. The wireless communication can be achieved by WIFI, an operator network, NFC (Near Field Communication) or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0065] The embodiment also provides a storage medium having a computer program stored thereon, the program being executed by a processor to implement the foot wound detection method based on image recognition as described in the above embodiment. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic storage, a flash memory, a magnetic disk or an optical disk.

[0066] To sum up, the present application realizes high-quality up-sampling of multi-band foot wound images by using the super-resolution reconstruction model constructed based on the super-resolution convolutional neural network model, significantly improves the resolution of low-quality images and the understanding ability of complex wound conditions, and lays a foundation for subsequent accurate segmentation. Meanwhile, the multi-scale feature fusion network is used to extract and fuse features at different scales, comprehensively capture various detailed features of the wound area, improve the accuracy of segmentation, provide detailed wound information, help to develop personalized treatment plans, and thus greatly improve the accuracy and efficiency of clinical diagnosis.

[0067] It should be noted that the above examples are only used to illustrate the technical solutions of the present application but not limit the present application. Although the present application is described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or equivalently replaced, without departing from the spirit and scope of the technical solutions of the present application, which should be covered in the scope of the claims of the present application.

Claims

1. An image recognition based foot wound detection method, characterized in that: The application relates to a method for generating a binary mask foot wound image based on a multi-band foot wound image. The method comprises the following steps: collecting and preprocessing a multi-band foot wound image, wherein the multi-band foot wound image comprises a visible light band wound image, a near-infrared band wound image, a short-wave infrared band wound image and a thermal infrared band wound image; constructing an ultra-resolution reconstruction model based on an ultra-resolution convolutional neural network model and historical multi-band foot wound images; separating the preprocessed multi-band foot wound image according to bands, performing fast up-sampling by using bicubic interpolation, inputting the multi-band foot wound image into the ultra-resolution reconstruction model for ultra-resolution reconstruction, and obtaining a high-resolution foot wound image; extracting foot wound features of the high-resolution foot wound image at different scales by using a multi-scale feature fusion network, constructing a pyramid level, and performing feature fusion from top to bottom by using a feature pyramid structure to generate a multi-scale foot wound feature map; inputting the multi-scale foot wound feature map into a segmentation network for cross-modal interaction and fusion, obtaining a regional physiological characteristic map and a regional probability map by using a deformable convolution network and a decoding network, comparing the regional physiological characteristic map and the regional probability map by using an adaptive threshold map, and generating a binary mask foot wound image; and superimposing the binary mask foot wound image and the high-resolution foot wound image by using OpenCV, marking a wound region by using a contour detection algorithm, and obtaining a foot wound region annotation image. The preprocessing comprises image denoising, multi-band registration, radiation normalization and band alignment. The ultra-resolution reconstruction model is constructed based on the ultra-resolution convolutional neural network model, and the specific steps are as follows: The ultra-resolution convolutional neural network model is used as a basic model. A 4-channel input layer is expanded to receive historical multi-band foot wound images.

2. The image recognition-based foot wound detection method of claim 1, wherein: A feature extraction layer extracts local features by using a small convolution kernel.

3. The image recognition-based foot wound detection method of claim 1, wherein: A nonlinear mapping layer compresses feature dimensions and fuses cross-band information. A cross-band feature interaction layer dynamically calculates the weight of each band and highlights important band features. An image reconstruction layer integrates global context information by using a large convolution kernel and outputs a high-resolution image. The ultra-resolution reconstruction model is constructed. The high-resolution foot wound image is obtained, and the specific steps are as follows: The preprocessed multi-band foot wound image is separated according to bands, fast up-sampling is performed by using bicubic interpolation, each band image is preliminarily enlarged to a target size, and the up-sampled 4-band images are combined into a 4-channel tensor. The 4-channel tensor is input into the ultra-resolution reconstruction model, and the 4-channel high-resolution tensor image is output by sequentially processing the feature extraction layer, the nonlinear mapping layer, the cross-band feature interaction layer and the image reconstruction layer. The 4-channel high-resolution tensor image is subjected to cross-band gradient alignment, and the high-resolution foot wound image is obtained by strengthening the wound edge through adaptive sharpening filtering.

4. The image recognition-based foot wound detection method of claim 1, wherein: The multi-scale foot wound feature map is generated, and the specific steps are as follows: High-resolution epidermal features, medium-resolution subcutaneous features and low-resolution necrotic features are extracted based on the high-resolution foot wound image. The feature pyramid level is constructed based on the high-resolution epidermal features, the medium-resolution subcutaneous features and the low-resolution necrotic features. The feature pyramid level is subjected to feature fusion from top to bottom to generate the multi-scale foot wound feature map.

5. The image recognition based foot wound detection method of claim 1, wherein: ​ ​ ​ ​ 6. The image recognition-based foot wound detection method of claim 1, wherein: The specific steps are as follows, The multiscale foot wound feature map is cross-modally interacted with the 4-channel high-resolution tensor image after super-resolution reconstruction, the waveband correlation attention mechanism is used to dynamically allocate the weight of each waveband and perform fusion; Based on the multi-modal fusion feature map, the edge guidance and region proposal are performed through the deformable convolution network to generate the wound region proposal, the local features are extracted and decoded to obtain the region physiological characteristic map; The region physiological characteristic map is up-sampled to the original image size through the decoding network, and the region probability map is generated through the Sigmoid activation; According to the region physiological characteristic map, an adaptive threshold map is generated, and compared with the region probability map, the region probability map greater than the adaptive threshold map is binarized to generate a binary mask foot wound image.

7. The image recognition-based foot wound detection method of claim 1, wherein: The specific steps are as follows, All closed contours in the binary mask foot wound image are detected using OpenCV, and small contours are filtered through a wound area threshold; The high-resolution foot wound image is converted to BGR three-channel format, and the drawContours function is used to fill the filtered contours in red, and the foot wound region annotation map with red contour annotation is output.

8. An image recognition-based foot wound detection system based on the image recognition-based foot wound detection method according to any one of claims 1 to 7, characterized by: It comprises, A data acquisition module is used to acquire multi-waveband foot wound images and perform preprocessing, wherein the multi-waveband foot wound images include visible light waveband wound images, near-infrared waveband wound images, short-wave infrared waveband wound images, and thermal infrared waveband wound images; An super-resolution reconstruction module is used to construct a super-resolution reconstruction model based on a super-resolution convolutional neural network model and historical multi-waveband foot wound images, separate the preprocessed multi-waveband foot wound images by waveband, use bicubic interpolation for fast up-sampling, and input the super-resolution reconstruction model for super-resolution reconstruction to obtain high-resolution foot wound images; A fusion module is used to extract foot wound features of high-resolution foot wound images at different scales through a multi-scale feature fusion network to construct a pyramid level, and perform feature fusion from top to bottom through a feature pyramid structure to generate a multiscale foot wound feature map; An accurate segmentation module is used to input the multiscale foot wound feature map into a segmentation network for cross-modal interaction and fusion, use a deformable convolution network and a decoding network to obtain a region physiological characteristic map and a region probability map, and compare them through an adaptive threshold map to generate a binary mask foot wound image; An annotation module is used to superimpose the binary mask foot wound image and the high-resolution foot wound image through OpenCV, and mark the wound area with a red contour through a contour detection algorithm to obtain a foot wound region annotation map. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is characterized in that: The processor executes the computer program to realize the steps of the foot wound detection method based on image recognition in any one of claims 1-7.

10. A computer readable storage medium having stored thereon a computer program, characterized in that: The computer program is executed by the processor to realize the steps of the foot wound detection method based on image recognition in any one of claims 1-7.

Images