Skin tissue layering evaluation method, device, equipment, medium and program product

Abstract

The invention relates to a skin tissue layering evaluation method, device and equipment, a medium and a program product. The method comprises the following steps: acquiring an original gray-scale ultrasonic image of skin tissue layering, and preprocessing the original gray-scale ultrasonic image to obtain a preprocessed ultrasonic image; performing elastic image generation on the preprocessed ultrasonic image through an elastic imaging generation network to obtain an elastic modulus diagram; performing super-resolution image reconstruction on the preprocessed ultrasonic image through a super-resolution reconstruction network to obtain a high-resolution image; performing feature extraction on the preprocessed ultrasonic image, the elastic modulus diagram and the high-resolution diagram to obtain a multi-modal hierarchical feature map; performing cross-modal attention fusion on the multi-modal hierarchical features to obtain a fusion feature map; and according to the fusion feature map, performing layering evaluation on the skin tissue layering to obtain a multi-skin tissue probability map. By adopting the method, the integrated skin tissue layering evaluation of the whole body skin, fascia and muscular tissues can be realized under the limitation of insufficient resolution.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a skin tissue stratification evaluation method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the development of artificial intelligence, ultrasound imaging has emerged as a technology for medical image processing. Ultrasound imaging is currently the only non-invasive, radiation-free method for observing the internal structure of human tissue. When ultrasound waves penetrate the skin, different tissue layers (epidermis, dermis, and subcutaneous fat) reflect echoes due to differences in acoustic impedance, forming an ultrasound image with layered characteristics. However, the range of ultrasound imaging is controlled by frequency. Lower frequencies increase imaging depth but reduce resolution in shallow layers, while higher frequencies increase resolution in shallow layers but limit depth. High-frequency ultrasound (>20MHz) can provide a spatial resolution of 0.05-0.1mm, making it suitable for fine-grained imaging of superficial skin and subcutaneous muscle. However, high-frequency ultrasound places high demands on the hardware system. Limited by cost and size, existing technologies struggle to achieve accurate high-frequency ultrasound within the limited hardware footprint, making it difficult to implement in portable and handheld ultrasound devices. Summary of the Invention

[0003] Based on this, it is necessary to provide a skin tissue stratification evaluation method, device, computer equipment, computer-readable storage medium and computer program product that can overcome the limitations of traditional evaluation methods in terms of high equipment cost, insufficient resolution, difficulty in quantification and multi-tissue collaborative analysis, and realize integrated skin tissue stratification and collagen quantification of skin, fascia and muscle tissue throughout the body.

[0004] In a first aspect, the present application provides a skin tissue stratification assessment method, comprising:

[0005] Acquiring an original grayscale ultrasound image of skin tissue layers, and preprocessing the original grayscale ultrasound image to obtain a preprocessed ultrasound image;

[0006] generating an elastic image of the preprocessed ultrasound image through a target elastic imaging generation network to obtain a target elastic modulus map;

[0007] The preprocessed ultrasound image is super-resolution reconstructed by a target super-resolution reconstruction network to obtain a target high-resolution image; the target elastic imaging generation network and the target super-resolution reconstruction network are trained and generated based on a preset elastic super-resolution joint training strategy;

[0008] performing feature extraction on the preprocessed ultrasound image, the target elastic modulus map, and the target high-resolution map to obtain a multimodal hierarchical feature map;

[0009] Performing cross-modal attention fusion on the multimodal hierarchical features to obtain a fused feature map;

[0010] The skin tissue layers are evaluated according to the fusion feature map to obtain a multi-skin tissue probability map, and the multi-skin tissue probability map is visualized.

[0011] In one embodiment, preprocessing the original grayscale ultrasound image to obtain a preprocessed ultrasound image includes:

[0012] Determining a target region on the original grayscale ultrasound image to obtain a target ultrasound image;

[0013] Denoising the target ultrasound image to obtain a denoised ultrasound image;

[0014] The denoised ultrasonic image is scaled to obtain the preprocessed ultrasonic image.

[0015] In one embodiment, performing cross-modal attention fusion on the multimodal hierarchical feature map to obtain a fused feature map includes:

[0016] Performing an affine transformation on the multimodal hierarchical feature map to obtain a multimodal feature map of equal size;

[0017] Performing channel alignment on the equal-sized multimodal feature maps to obtain equal-channel multimodal feature maps;

[0018] Performing multimodal graph splicing on the equal-channel multimodal feature graph to obtain a spliced ​​multimodal feature graph;

[0019] Performing global average pooling on the concatenated multimodal feature map to obtain channel weights;

[0020] Performing multimodal learning on the channel weights through a preset fully connected layer to obtain a weight vector;

[0021] Performing weighted summation on the concatenated multimodal feature map according to the weight vector to obtain a channel attention feature map;

[0022] Performing a convolution operation on the channel attention feature map to obtain a spatial weight matrix;

[0023] activating the spatial weight matrix through a threshold function to obtain key anatomical boundaries;

[0024] A fusion feature is constructed based on the key anatomical boundary, the spliced ​​multimodal feature map and the channel weight to obtain the fusion feature map.

[0025] In one embodiment, the method further comprises:

[0026] Performing image segmentation on the preprocessed ultrasound image, the target elastic modulus map, and the target high-resolution map to obtain image block data;

[0027] Performing image feature extraction on the image block data to obtain an image feature vector;

[0028] Classifying the image feature vector according to a preset collagen classification model to obtain a binary classification result;

[0029] Calculating collagen quantitative data of the skin tissue layer based on the binary classification results;

[0030] A structured report is generated based on the collagen quantification data and the multiple skin tissue probability map, and the structured report is displayed.

[0031] In one embodiment, the collagen quantitative data includes collagen area ratio, spatial heterogeneity index and elastic modulus gradient distribution; and the step of calculating the collagen quantitative data of the skin tissue stratification based on the binary classification results includes:

[0032] Performing morphological smoothing on the binary classification results through a spatial consistency correction strategy to obtain a morphologically smoothed result;

[0033] Calculating the collagen ratio based on the number of collagen blocks, the area of ​​a single block, and the total skin area of ​​the morphological smoothing result to obtain the collagen area ratio;

[0034] performing heterogeneity calculation based on the collagen block distribution data of the morphological smoothing result to obtain the spatial heterogeneity index;

[0035] Obtaining the elastic modulus gradient distribution of the collagen region based on the binary classification result;

[0036] A cumulative distribution model is generated according to the elastic modulus gradient distribution; the cumulative distribution model is used to quantify the hardness variation trend to obtain the elastic modulus gradient distribution.

[0037] In one embodiment, the method further comprises:

[0038] Acquiring a training ultrasound image of a low-resolution device and a training elastic modulus map of a high-resolution device; wherein the low-resolution device and the high-resolution device are operated synchronously;

[0039] Training a preset original elastic imaging generation network based on the training ultrasound image and the training elastic modulus map to obtain a candidate elastic imaging generation network;

[0040] Training a preset original super-resolution reconstruction network based on the training ultrasound image to obtain a candidate super-resolution reconstruction network; wherein a feature layer association exists between the candidate elastic imaging generation network and the candidate super-resolution reconstruction network;

[0041] Performing alternating joint training on the candidate elastic imaging generation network and the candidate super-resolution reconstruction network, and constructing a joint optimization loss model for the candidate elastic imaging generation network and the candidate super-resolution reconstruction network during training;

[0042] The candidate elastic imaging generation network and the candidate super-resolution reconstruction network are jointly optimized according to the joint optimization loss model to obtain the target elastic imaging generation network and the target super-resolution reconstruction network.

[0043] In a second aspect, the present application further provides a skin tissue layering assessment device, comprising:

[0044] an acquisition module, configured to acquire an original grayscale ultrasound image of skin tissue layers, and preprocess the original grayscale ultrasound image to obtain a preprocessed ultrasound image;

[0045] an elastic image generation module, configured to generate an elastic image of the preprocessed ultrasound image through a target elastic imaging generation network to obtain a target elastic modulus map;

[0046] An image reconstruction module is configured to perform super-resolution image reconstruction on the pre-processed ultrasound image through a target super-resolution reconstruction network to obtain a target high-resolution image; the target elastic imaging generation network and the target super-resolution reconstruction network are trained and generated based on a preset elastic super-resolution joint training strategy;

[0047] a feature extraction module, configured to extract features from the preprocessed ultrasound image, the target elastic modulus map, and the target high-resolution map to obtain a multimodal hierarchical feature map;

[0048] An attention fusion module, configured to perform cross-modal attention fusion on the multimodal hierarchical features to obtain a fused feature map;

[0049] A stratification evaluation module is used to perform stratification evaluation on the skin tissue stratification according to the fusion feature map, obtain a multi-skin tissue probability map, and visualize the multi-skin tissue probability map.

[0050] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0051] Acquiring an original grayscale ultrasound image of skin tissue layers, and preprocessing the original grayscale ultrasound image to obtain a preprocessed ultrasound image;

[0052] generating an elastic image of the preprocessed ultrasound image through a target elastic imaging generation network to obtain a target elastic modulus map;

[0053] The preprocessed ultrasound image is super-resolution reconstructed by a target super-resolution reconstruction network to obtain a target high-resolution image; the target elastic imaging generation network and the target super-resolution reconstruction network are trained and generated based on a preset elastic super-resolution joint training strategy;

[0054] performing feature extraction on the preprocessed ultrasound image, the target elastic modulus map, and the target high-resolution map to obtain a multimodal hierarchical feature map;

[0055] Performing cross-modal attention fusion on the multimodal hierarchical features to obtain a fused feature map;

[0056] The skin tissue layers are evaluated according to the fusion feature map to obtain a multi-skin tissue probability map, and the multi-skin tissue probability map is visualized.

[0057] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0058] Acquiring an original grayscale ultrasound image of skin tissue layers, and preprocessing the original grayscale ultrasound image to obtain a preprocessed ultrasound image;

[0059] generating an elastic image of the preprocessed ultrasound image through a target elastic imaging generation network to obtain a target elastic modulus map;

[0060] The preprocessed ultrasound image is super-resolution reconstructed by a target super-resolution reconstruction network to obtain a target high-resolution image; the target elastic imaging generation network and the target super-resolution reconstruction network are trained and generated based on a preset elastic super-resolution joint training strategy;

[0061] performing feature extraction on the preprocessed ultrasound image, the target elastic modulus map, and the target high-resolution map to obtain a multimodal hierarchical feature map;

[0062] Performing cross-modal attention fusion on the multimodal hierarchical features to obtain a fused feature map;

[0063] The skin tissue layers are evaluated according to the fusion feature map to obtain a multi-skin tissue probability map, and the multi-skin tissue probability map is visualized.

[0064] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the following steps:

[0065] Acquiring an original grayscale ultrasound image of skin tissue layers, and preprocessing the original grayscale ultrasound image to obtain a preprocessed ultrasound image;

[0066] generating an elastic image of the preprocessed ultrasound image through a target elastic imaging generation network to obtain a target elastic modulus map;

[0067] The preprocessed ultrasound image is super-resolution reconstructed by a target super-resolution reconstruction network to obtain a target high-resolution image; the target elastic imaging generation network and the target super-resolution reconstruction network are trained and generated based on a preset elastic super-resolution joint training strategy;

[0068] performing feature extraction on the preprocessed ultrasound image, the target elastic modulus map, and the target high-resolution map to obtain a multimodal hierarchical feature map;

[0069] Performing cross-modal attention fusion on the multimodal hierarchical features to obtain a fused feature map;

[0070] The skin tissue layers are evaluated according to the fusion feature map to obtain a multi-skin tissue probability map, and the multi-skin tissue probability map is visualized.

[0071] The above-mentioned skin tissue stratification evaluation method, device, computer equipment, computer-readable storage medium and computer program product obtain the original grayscale ultrasound image of the skin tissue stratification and preprocess the original grayscale ultrasound image to obtain a preprocessed ultrasound image for subsequent processing and analysis. The target elastic imaging generation network is used to generate an elastic image of the preprocessed ultrasound image, and the target elastic modulus map is efficiently obtained. The preprocessed ultrasound image is super-resolution reconstructed by the target super-resolution reconstruction network to obtain a high-resolution target high-resolution map. The target elastic imaging generation network and the target super-resolution reconstruction network are trained and generated based on a preset elastic super-resolution joint training strategy, and the elastic imaging generation network is used to generate an elastic image of the preprocessed ultrasound image. The detailed features after network reconstruction are embedded in the elastic imaging generation network to improve the recognition accuracy of tiny structures. Feature extraction is performed on the preprocessed ultrasound image, target elastic modulus map and target high-resolution map to obtain a multi-modal hierarchical feature map. The multi-modal hierarchical features are fused with cross-modal attention to obtain a fused feature map. According to the fused feature map, the skin tissue layers are evaluated hierarchically to obtain a multi-skin tissue probability map, and the multi-skin tissue probability map is visualized, thereby innovatively realizing the construction of a "grayscale-elasticity-super-resolution" three-modal fusion hierarchical evaluation network, which can break through the limitations of traditional evaluation methods such as high equipment cost, insufficient resolution, quantification and difficulty in multi-tissue collaborative analysis of the original elastic imaging network generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0073] Figure 1 A diagram showing an application environment of a skin tissue layering assessment method according to an embodiment;

[0074] Figure 2 Schematic diagram of a process for evaluating skin tissue stratification according to one embodiment;

[0075] Figure 3 Schematic diagram of a process for evaluating skin tissue stratification according to one embodiment;

[0076] Figure 4 This is a general diagram of the overall architecture of a skin tissue layering assessment method in another embodiment;

[0077] Figure 5 Schematic diagram of the structure of a multimodal enhancement and generation module of a skin tissue layer assessment method in another embodiment;

[0078] Figure 6 Schematic diagram of the structure of a tissue segmentation and stratification module of a skin tissue stratification assessment method in another embodiment;

[0079] Figure 7 is a structural block diagram of a skin tissue layer evaluation device according to one embodiment;

[0080] Figure 8 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0081] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0082] The skin tissue stratification evaluation method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The server 104 obtains the original grayscale ultrasound image of the skin tissue stratification and preprocesses the original grayscale ultrasound image to obtain a preprocessed ultrasound image; generates an elastic image for the preprocessed ultrasound image through the target elastic imaging generation network to obtain a target elastic modulus map; reconstructs the preprocessed ultrasound image through the target super-resolution reconstruction network to obtain a target high-resolution map; the target elastic imaging generation network and the target super-resolution reconstruction network are trained and generated based on a preset elastic super-resolution joint training strategy; feature extraction is performed on the preprocessed ultrasound image, the target elastic modulus map and the target high-resolution map to obtain a multi-modal hierarchical feature map; cross-modal attention fusion is performed on the multi-modal hierarchical features to obtain a fused feature map; the skin tissue stratification is hierarchically evaluated according to the fused feature map to obtain a multi-skin tissue probability map, and the multi-skin tissue probability map is visualized. Terminal 102 may include, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices may include smart speakers, smart TVs, smart air conditioners, smart car devices, and projectors. Portable wearable devices may include smart watches, smart bracelets, and head-mounted devices. Head-mounted devices may include virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, and the like. Server 104 may be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing cloud computing services.

[0083] In an exemplary embodiment, Figure 2 As shown, a skin tissue layering evaluation method is provided, which is applied to Figure 1 The server 104 in the example is used to illustrate the process, including the following steps 202 to 212.

[0084] Step 202 : obtaining an original grayscale ultrasound image of skin tissue layers, and preprocessing the original grayscale ultrasound image to obtain a preprocessed ultrasound image.

[0085] Among them, the skin tissue stratification is mainly composed of the following four layers: epidermis, dermis, subcutaneous fat layer and muscle. Among them, the epidermis is composed of keratinocytes and is a homogeneous strong echo band (thickness of about 0.05-0.1mm). The dermis contains collagen fiber bundles and elastic fibers, which appear as thin linear high echoes against a medium echo background (thickness 0.5-3mm). The subcutaneous fat layer (SMAS layer) is a low-echo reticular structure interspersed with high-echo fibrous septa (superficial fascia). Muscle is a common tissue in the human body. It should be noted that when ultrasound penetrates the skin, different tissue layers produce reflected echoes due to differences in acoustic impedance, forming an ultrasound image with layered characteristics.

[0086] In some embodiments, a portable ultrasound device (5-15 MHz probe) can be used to obtain original grayscale ultrasound images of skin tissue layers composed of skin, fascia, and muscles throughout the body, and the original grayscale ultrasound images can be preprocessed to obtain preprocessed ultrasound images.

[0087] In some embodiments, the original grayscale ultrasound image is preprocessed to obtain a preprocessed ultrasound image, including: determining a target area on the original grayscale ultrasound image to obtain a target ultrasound image; denoising the target ultrasound image to obtain a denoised ultrasound image; and scaling the denoised ultrasound image to obtain a preprocessed ultrasound image.

[0088] In some embodiments, a deep learning target detection model is used to automatically identify and determine the minimum circumscribed rectangular frame (fan-shaped rectangular frame) containing the ultrasound window in the original grayscale ultrasound image, and the skin ultrasound window range of the original grayscale ultrasound image is determined to obtain a target ultrasound image, thereby achieving desensitization processing of the image data and providing a target ultrasound image containing only skin ultrasound window information for subsequent calculations.

[0089] Among them, the selected deep learning target detection models may include but are not limited to YOLO series models, RCNN series models, DETR series models, etc.

[0090] In some embodiments, the fan-shaped rectangular frame is detected and calculated only for the first 10 frames of the ultrasound video stream in the original grayscale ultrasound image. Subsequently, the average or median coordinates of the vertex of the fan-shaped rectangular frame predicted in these 10 frames are calculated to determine the final cropping area. Subsequent ultrasound videos of the original grayscale ultrasound image are cropped based on this area to obtain the target ultrasound image, thereby ensuring computational efficiency and accuracy.

[0091] In some embodiments, various types of noise (such as speckle noise) are removed from the target ultrasound image to obtain a denoised ultrasound image, thereby preserving tissue clarity.

[0092] The noise removal methods used may include but are not limited to adaptive median filter, wavelet threshold denoising, non-local means and other methods.

[0093] In some embodiments, the denoised ultrasound image is resized using bilinear interpolation to a target size. The narrow edges of the scaled image are padded with a black background (three-channel pixel values ​​of (0, 0, 0)) to ensure that the image maintains its original aspect ratio and does not lose important information during the scaling process. Finally, the image pixel values ​​are normalized to 32-bit floating-point numbers between 0 and 1 by dividing them by the maximum pixel value of 255 to obtain a preprocessed ultrasound image for subsequent processing and analysis. The black background refers to three-channel pixel values ​​of (0, 0, 0). The target size can include, but is not limited to, 448×448, 256×256, and 224×224 pixels.

[0094] Step 204 : generating an elastic image of the preprocessed ultrasound image using a target elastic imaging generation network to obtain a target elastic modulus map.

[0095] Among them, the target elastic imaging generation network is a neural network model trained based on an improved framework of generative adversarial network, which is used to generate elastic modulus maps with a resolution consistent with the grayscale image of the preprocessed ultrasound image.

[0096] It should be noted that the improved framework of the generative adversarial network in the target elastic imaging generation network includes a first generator and a first discriminator. The first generator is a symmetric architecture similar to U-Net, including but not limited to a convolutional neural network with 5 or more layers (convolution kernel size is 3×3, 5×5, 7×7, etc.), a normalization layer (BatchNorm), and a nonlinear activation layer (such as LeakyReLU, ReLU). The output layer is a 1-channel elastic modulus map (with the same resolution as the input grayscale image), which is normalized to [-1, 1] using the Tanh activation function. By splicing the features of each layer of the encoder with the corresponding layer of the decoder, local anatomical structure information is retained. It should be further noted that the first discriminator is mainly used in the training stage. The first discriminator is based on the PatchGAN architecture and contains 3-5 convolution layers. The input layer is the real / generated elastic modulus map and the corresponding grayscale image (3 channels). The output layer is a 70×70 discriminant matrix, and each pixel represents the authenticity probability of the local area.

[0097] In some embodiments, elastic images are generated on preprocessed ultrasound images by the first target generator of the target elastic imaging generation network, and a target elastic modulus map consistent with the grayscale image of the preprocessed ultrasound image can be obtained, thereby facilitating the unification of subsequent cross-modal attention fusion steps.

[0098] In step 206, the preprocessed ultrasound image is super-resolved using a target super-resolution reconstruction network to generate a target high-resolution image. The target elastic imaging generation network and the target super-resolution reconstruction network are trained and generated based on a preset elastic super-resolution joint training strategy. The target super-resolution reconstruction network is a neural network model trained using an improved generative adversarial network framework and is used to generate a target high-resolution image with a higher resolution than the preprocessed grayscale ultrasound image.

[0099] It should be noted that the improved GAN architecture in the target super-resolution reconstruction network includes a second generator and a second discriminator. The second generator can adopt an ESR-GAN-based residual dense block structure consisting of 8-16 residual dense blocks. The data input is a single-channel low-resolution grayscale image. Each RRDB module includes dense connections and residual link layers. The resolution is quadrupled by using a multi-layer sub-pixel convolutional stacking network, making the output high-resolution grayscale image equivalent to that acquired by a 20MHz probe. Reinforced Luminance (ReLU) activation is used to ensure non-negativity. It should be further noted that the second discriminator is primarily used during the training phase. It uses a multi-scale discriminator consisting of three parallel sub-networks (with resolutions of the original image, 1 / 2, and 1 / 4, respectively) to constrain the generated image's overall structure (low-resolution sub-network) and local details (high-resolution sub-network) to be realistic. The residual dense block refers to the RRDB, the single-channel low-resolution grayscale image is equivalent to that acquired by a 5-15MHz probe, and the sub-pixel convolutional stacking network refers to the PixelShuffle.

[0100] In some embodiments, the preprocessed ultrasound image is super-resolution reconstructed by the second target generator of the target super-resolution reconstruction network to obtain a target high-resolution image equivalent to the image acquired by a 20MHz probe, for use in subsequent multimodal feature extraction steps and attention fusion steps.

[0101] Step 208: Extract features from the pre-processed ultrasound image, the target elastic modulus map, and the target high-resolution map to obtain a multi-modal hierarchical feature map, wherein the multi-modal hierarchical feature map includes ultrasound image hierarchical features under multiple modalities.

[0102] In some embodiments, for data from different modalities, different or the same preset lightweight networks can be used to extract the corresponding ultrasound image hierarchical features of the preprocessed ultrasound image, target elastic modulus map, and target high-resolution map in each modality, thereby obtaining a multimodal hierarchical feature map. Preset lightweight networks include, but are not limited to, MobileNet, Yolo, and Fast-SCNN models.

[0103] In step 210 , cross-modal attention fusion is performed on the multi-modal hierarchical features to obtain a fused feature map.

[0104] Among them, cross-modal attention fusion refers to dynamically weighting the hierarchical features of ultrasound images under each modality through the attention mechanism, thereby improving the segmentation accuracy of key areas.

[0105] In some embodiments, cross-modal attention fusion is performed on multimodal hierarchical feature maps to obtain a fused feature map, including: performing affine transformation on the multimodal hierarchical feature map to obtain an equal-sized multimodal feature map; performing channel alignment on the equal-sized multimodal feature map to obtain an equal-channel multimodal feature map; performing multimodal map splicing on the equal-channel multimodal feature map to obtain a spliced ​​multimodal feature map; performing global average pooling on the spliced ​​multimodal feature map to obtain channel weights; performing multimodal learning on the channel weights through a preset fully connected layer to obtain a weight vector; performing weighted summation on the spliced ​​multimodal feature map according to the weight vector to obtain a channel attention feature map; performing a convolution operation on the channel attention feature map to obtain a spatial weight matrix; activating the spatial weight matrix through a threshold function to obtain key anatomical boundaries; and constructing fused features based on the key anatomical boundaries, the spliced ​​multimodal feature map, and the channel weights to obtain a fused feature map.

[0106] In some embodiments, an affine transformation is used to unify the multimodal hierarchical feature maps to a 2R×2R resolution, thereby obtaining an equal-sized multimodal feature map. A 1×1 convolution is then used to unify the number of feature channels of each branch in the equal-sized multimodal feature map to R, thereby obtaining an equal-channel multimodal feature map. R can be 64, 128, 256, or other sizes.

[0107] In some embodiments, multimodal graph concatenation is performed on equal-channel multimodal feature maps to obtain a concatenated multimodal feature map (R×3=3R channels). Global average pooling is performed on the concatenated multimodal feature map to obtain channel weights. A preset fully connected layer is used to learn the importance of each modality in the concatenated multimodal feature map, thereby outputting a weight vector (3D). The concatenated multimodal feature map is weighted and summed according to the weight vector to obtain a channel attention feature map (R channels). A convolution operation is performed on the channel attention feature map to obtain a spatial weight matrix (R channels, 2R×2R). The spatial weight matrix is ​​activated using a sigmoid threshold function to obtain a key anatomical boundary. The key anatomical boundary may be the dermis-subcutaneous fat layer.

[0108] In some embodiments, a fusion feature is constructed based on key anatomical boundaries, spliced ​​multimodal feature maps and channel weights to obtain a fusion feature map. The specific construction process is shown in the following formula (1):

[0109] (1);

[0110] in, is the channel weight, is the spatial weight, is the characteristic diagram of each mode.

[0111] Step 212 , performing a hierarchical evaluation on the skin tissue layers according to the fused feature map, obtaining a multi-skin tissue probability map, and visually displaying the multi-skin tissue probability map.

[0112] The stratification assessment may be an assessment of the stratification of skin tissue, or may be a further quantitative assessment of collagen based on a multi-skin tissue probability map, but is not limited thereto.

[0113] In some embodiments, skin tissue stratification is evaluated based on the fused feature map, and the fused feature map is converted into a 4-channel multi-skin tissue probability map (corresponding to the four tissue layers of epidermis, dermis, fascia, and muscle) with the same size as the original grayscale ultrasound image by performing a multi-layer radial transformation or deconvolution operation on the fused feature map, and the final segmentation mask is generated by Softmax.

[0114] In some embodiments, the classification results in the multi-skin tissue probability map are spliced ​​according to the original block positions to generate a binary mask, and the edges are smoothed by Gaussian filtering and converted into a pseudo-color heat map for visual display, thereby improving the readability of the multi-skin tissue probability map.

[0115] In some embodiments, the method further includes: performing image segmentation on the preprocessed ultrasound image, the target elastic modulus map, and the target high-resolution map to obtain image block data; performing image feature extraction on the image block data to obtain an image feature vector; classifying the image feature vector according to a preset collagen classification model to obtain a binary classification result; calculating the collagen quantitative data of the skin tissue stratification based on the binary classification result; generating a structured report based on the collagen quantitative data and the multi-skin tissue probability map, and displaying the structured report.

[0116] In some embodiments, the input preprocessed ultrasound image, target elastic modulus map, and target high-resolution image are divided into non-overlapping image blocks of a fixed size to obtain image block data. Image features are extracted from each image block data to obtain an image feature vector. All image feature vectors of the divided blocks are input into a preset collagen classification model for classification, obtaining a binary classification result for each block. The image feature vectors include, but are not limited to, grayscale mean, grayscale variance, elastic modulus mean, and local texture features extracted using a gray-level co-occurrence matrix (GLCM) and Gabor filter. The fixed size can be 1×1, 3×3, or 7×7 pixels. The preset collagen classification model is constructed by Z-score normalization of continuous features such as grayscale value and elastic modulus, then using a fixed threshold method or machine learning methods such as logistic regression, random forest, and support vector machine to determine whether an image block is a collagen aggregation or non-aggregation classification model.

[0117] In some embodiments, collagen quantification data includes collagen area ratio, spatial heterogeneity index and elastic modulus gradient distribution; collagen quantification data of skin tissue stratification is calculated based on the binary classification results, including: morphologically smoothing the binary classification results through a spatial consistency correction strategy to obtain a morphologically smoothed result; calculating the collagen ratio based on the number of collagen blocks, single block area and total skin area data of the morphological smoothing results to obtain the collagen area ratio; performing heterogeneity calculation based on the collagen block distribution data of the morphological smoothing results to obtain a spatial heterogeneity index; obtaining the elastic modulus gradient distribution of the collagen area based on the binary classification results; generating a cumulative distribution model based on the elastic modulus gradient distribution; the cumulative distribution model is used to quantify the hardness change trend to obtain the elastic modulus gradient distribution.

[0118] In some embodiments, a spatial consistency correction strategy is adopted to perform morphological smoothing on the corresponding isolated misclassified blocks in the binary classification results to obtain a morphologically smoothed result.

[0119] In some embodiments, the collagen ratio is calculated based on the number of collagen blocks, the area of ​​a single block, and the total skin area data of the morphological smoothing result. The specific calculation process of the collagen area ratio is shown in the following formula (2):

[0120] (2).

[0121] In some embodiments, the collagen mass distribution data is calculated based on the calculation and mean, according to The specific process of calculating heterogeneity by using the mean and the spatial heterogeneity index is shown in the following formula (3):

[0122] (3).

[0123] In some embodiments, the elastic modulus gradient distribution of the collagen region is statistically analyzed based on the binary classification results to obtain the elastic modulus gradient distribution of the collagen region, and a cumulative distribution model (CDF) is generated according to the elastic modulus gradient distribution. The cumulative distribution model is used to quantify the hardness change trend to obtain the elastic modulus gradient distribution.

[0124] In some embodiments, based on the high-precision skin stratification results from multiple skin tissue probability maps, collagen ratio values ​​are quantified. Based on the quantitative relationship between collagen density, ultrasound grayscale values, and local elastic properties in the collagen quantification data, a regression model of collagen density, image grayscale values, and local tissue elastic properties is established, i.e., a preset collagen classification model. Through machine learning, the weight coefficients in the collagen classification model are dynamically optimized to achieve millimeter-level spatial resolution of collagen content under non-invasive conditions. This outputs a collagen spatial distribution heat map and a multidimensional quantitative structured report, providing an objective basis for accurate tissue collagen quantification and parameter selection for treatment options such as radiofrequency. The structured report includes core parameters such as the collagen ratio and heterogeneity of each skin layer, and the gradient change of collagen density with depth.

[0125] In the above-mentioned skin tissue stratification evaluation method, the original grayscale ultrasound image of the skin tissue stratification is obtained, and the original grayscale ultrasound image is preprocessed to obtain a preprocessed ultrasound image for subsequent processing and analysis. The elastic image of the preprocessed ultrasound image is generated by the target elastic imaging generation network, and the target elastic modulus map can be efficiently obtained. The preprocessed ultrasound image is super-resolution reconstructed by the target super-resolution reconstruction network to obtain a high-resolution target high-resolution map; the target elastic imaging generation network and the target super-resolution reconstruction network are trained and generated based on a preset elastic super-resolution joint training strategy, and the detailed features reconstructed by the elastic imaging generation network are embedded in the elastic imaging generation network , improve the recognition accuracy of tiny structures, extract features from preprocessed ultrasound images, target elastic modulus maps and target high-resolution maps, obtain multimodal hierarchical feature maps, perform cross-modal attention fusion on multimodal hierarchical features, obtain fusion feature maps, perform hierarchical evaluation of skin tissue stratification based on the fusion feature maps, obtain multi-skin tissue probability maps, and visualize the multi-skin tissue probability maps, thereby innovatively realizing the construction of a "grayscale-elasticity-super-resolution" three-modal fusion hierarchical evaluation network, which can break through the limitations of traditional evaluation methods in high equipment cost, insufficient resolution, quantification and difficulty in multi-tissue collaborative analysis, and realize integrated skin tissue stratification of whole body skin, fascia and muscle tissue.

[0126] In an exemplary embodiment, Figure 3 As shown, the skin tissue layering assessment method further includes steps 302 to 310. In which:

[0127] Step 302: Acquire a training ultrasound image of a low-resolution device and a training elastic modulus map of a high-resolution device; wherein the low-resolution device and the high-resolution device are operated synchronously.

[0128] In some embodiments, a training ultrasound image from a low-resolution device and a training elastic modulus map from a synchronously running high-resolution device can be collected, and the training ultrasound images and the training elastic modulus map can be used as paired datasets for neural network training. The low-resolution device refers to a 5-15 MHz probe device, and the high-resolution device refers to a 20 MHz probe device.

[0129] Step 304: Train a preset original elastic imaging generation network based on the training ultrasound images and the training elastic modulus map to obtain a candidate elastic imaging generation network. The preset original elastic imaging adopts an improved framework based on a generative adversarial network.

[0130] In some embodiments, a preset original elastic imaging generation network is trained based on the training ultrasound images and the training elastic modulus maps as input to obtain a candidate elastic imaging generation network. The candidate elastic imaging generation network can be, but is not limited to, an elastic imaging generation network from any round of training. However, the training rounds of the candidate elastic imaging generation network and the candidate super-resolution reconstruction network are performed alternately.

[0131] In step 306, the preset original super-resolution reconstruction network is trained based on the training ultrasound images to obtain a candidate super-resolution reconstruction network. A feature-level correlation exists between the candidate elastic imaging generation network and the candidate super-resolution reconstruction network. This feature-level correlation means that during the alternating training of the candidate elastic imaging generation network and the candidate super-resolution reconstruction network, the feature map of the candidate super-resolution reconstruction network is input into the connection layer of the elastic decoder in the candidate elastic imaging generation network and fused into the elastic modulus generation process via a 1×1 convolution, thereby enhancing the local details of the elastic modulus map.

[0132] In some embodiments, a training ultrasound image of a single-channel low-resolution grayscale image is used as input to train a preset original super-resolution reconstruction network to obtain a candidate super-resolution reconstruction network.

[0133] Step 308 : alternately jointly train the candidate elastic imaging generation network and the candidate super-resolution reconstruction network, and construct a joint optimization loss model for the candidate elastic imaging generation network and the candidate super-resolution reconstruction network during training.

[0134] In some embodiments, during the alternating joint training of the candidate elastic imaging generation network and the candidate super-resolution reconstruction network, after each round of training, the second generator is fixed, and then the first generator and the first discriminator are trained, and then the first generator is fixed, and the second generator and the second discriminator are trained, and a joint optimization loss model is designed based on the generation consistency loss of the candidate elastic imaging generation network and the candidate super-resolution reconstruction network, wherein the joint optimization loss model The formula is shown in formula (4):

[0135] (4);

[0136] in, represents the output of the candidate super-resolution reconstruction network, represents the output of the candidate elastography generation network, The training rounds are the same.

[0137] Step 310 : Jointly optimize the candidate elastic imaging generation network and the candidate super-resolution reconstruction network according to the joint optimization loss model to obtain a target elastic imaging generation network and a target super-resolution reconstruction network.

[0138] In some embodiments, based on the consistency loss output by the joint optimization loss model, the consistency loss in each round is optimized to a minimum or blunted by back propagation, but not limited to this, so that the candidate elastic imaging generation network and the candidate super-resolution reconstruction network in the current round are obtained as the target elastic imaging generation network and the target super-resolution reconstruction network respectively.

[0139] In order to better understand the solution of this application, Figure 4 、 Figure 5 and Figure 6 The details are as follows:

[0140] In some embodiments, as follows Figure 4 As shown, Figure 4 It shows the complete overall architecture of this application in practical application. Figure 4The system architecture presented in the paper is divided into four core modules, including data acquisition and preprocessing module, multimodal enhancement and generation module, tissue segmentation and stratification module, and collagen quantification module. Each module works together to form an end-to-end closed-loop processing flow. Specifically, in practical applications, first, the data acquisition and preprocessing module is used to obtain the original grayscale ultrasound image of the skin tissue stratification, and preprocess the original grayscale ultrasound image to obtain a preprocessed ultrasound image, and then send the preprocessed ultrasound image to the multimodal enhancement and generation module. Secondly, the multimodal enhancement and generation module is used to generate an elastic image of the preprocessed ultrasound image through the target elastic imaging generation network to obtain a target elastic modulus map, and perform super-resolution image reconstruction on the preprocessed ultrasound image through the target super-resolution reconstruction network to obtain a target high-resolution map, and send the preprocessed ultrasound image, target elastic modulus map and target high-resolution map to the tissue segmentation and stratification module. Then, the tissue segmentation and stratification module performs feature extraction on the preprocessed ultrasound image, target elastic modulus map and target high-resolution map to obtain a multimodal hierarchical feature map, performs cross-modal attention fusion on the multimodal hierarchical features to obtain a fused feature map, and performs hierarchical evaluation of the skin tissue stratification based on the fused feature map to obtain a multi-skin tissue probability map. Finally, the host device connected to the ultrasound device visualizes the multi-skin tissue probability map. In addition, in other embodiments, Figure 4 The tissue segmentation module will send the multi-skin tissue probability map, pre-processed ultrasound image, target elastic modulus map and target high-resolution map to the collagen quantification module for collagen quantitative analysis, obtain collagen quantitative data, and generate a structured report for display.

[0141] In some embodiments, as Figure 5 As shown, Figure 5 Shown Figure 4 The structural diagram of the multimodal enhancement and generation module in Figure 5 The low-resolution grayscale image in the pre-processed ultrasound image is Figure 5 The high-resolution grayscale image in the above is the target high-resolution image. Figure 5 The elastic modulus diagram in refers to the target elastic modulus diagram. Figure 5 The paper shows that the feature map of the super-resolution reconstruction network is input into the connection layer of the elastic decoder and fused into the elastic modulus generation process through 1×1 convolution, thereby improving the local details of the elastic modulus map. Figure 5 The super-resolution discriminator in

[15] uses a multi-scale discriminator, which contains three parallel sub-networks to constrain the authenticity of the generated image in terms of overall structure and local details.

[0142] In some embodiments, Figure 6 Shown Figure 4The structural diagram of the tissue segmentation and hierarchical module is used to achieve fine segmentation of four tissue layers: epidermis, dermis, fascia, and muscle. It can use two input modes: "single modality" and "multimodality". When performing tissue segmentation multimodal input, a multi-branch parallel-attention fusion architecture is adopted: the overall process is divided into three stages: feature extraction, cross-modal attention fusion, and hierarchical segmentation output. The specific architecture is as follows: Figure 6 shown.

[0143] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0144] Based on the same inventive concept, embodiments of the present application also provide a skin tissue layer assessment device for implementing the aforementioned skin tissue layer assessment method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more of the following embodiments of the skin tissue layer assessment device can be found in the aforementioned limitations of the skin tissue layer assessment method and will not be further elaborated here.

[0145] In an exemplary embodiment, Figure 7 As shown, a skin tissue layer evaluation device is provided, comprising: an acquisition module 701, an elastic image generation module 702, an image reconstruction module 703, a feature extraction module 704 and an attention fusion module 705, wherein:

[0146] An acquisition module 701 is used to acquire an original grayscale ultrasound image of skin tissue layers and preprocess the original grayscale ultrasound image to obtain a preprocessed ultrasound image;

[0147] an elastic image generation module 702 for generating an elastic image of the preprocessed ultrasound image using a target elastic imaging generation network to obtain a target elastic modulus map;

[0148] Image reconstruction module 703, configured to perform super-resolution image reconstruction on the pre-processed ultrasound image through a target super-resolution reconstruction network to obtain a target high-resolution image; the target elastic imaging generation network and the target super-resolution reconstruction network are trained and generated based on a preset elastic super-resolution joint training strategy;

[0149] A feature extraction module 704 is used to extract features from the pre-processed ultrasound image, the target elastic modulus map, and the target high-resolution map to obtain a multimodal hierarchical feature map;

[0150] Attention fusion module 705, used to perform cross-modal attention fusion on multi-modal hierarchical features to obtain a fused feature map;

[0151] In some embodiments, the acquisition module 701 is further used to determine the target area of ​​the original grayscale ultrasound image to obtain a target ultrasound image; denoise the target ultrasound image to obtain a denoised ultrasound image; and scale the denoised ultrasound image to obtain a preprocessed ultrasound image.

[0152] In some embodiments, the attention fusion module 705 is also used to perform affine transformation on the multimodal hierarchical feature map to obtain an equal-sized multimodal feature map; perform channel alignment on the equal-sized multimodal feature map to obtain an equal-channel multimodal feature map; perform multimodal map splicing on the equal-channel multimodal feature map to obtain a spliced ​​multimodal feature map; perform global average pooling on the spliced ​​multimodal feature map to obtain channel weights; perform multimodal learning on the channel weights through a preset fully connected layer to obtain a weight vector; perform weighted summation on the spliced ​​multimodal feature map according to the weight vector to obtain a channel attention feature map; perform convolution operation on the channel attention feature map to obtain a spatial weight matrix; activate the spatial weight matrix through a threshold function to obtain key anatomical boundaries; and construct fusion features based on the key anatomical boundaries, the spliced ​​multimodal feature map and the channel weights to obtain a fusion feature map.

[0153] In some embodiments, the device also includes: a collagen quantification module, which is used to perform image segmentation on the preprocessed ultrasound image, the target elastic modulus map and the target high-resolution map to obtain image block data; perform image feature extraction on the image block data to obtain an image feature vector; classify the image feature vector according to a preset collagen classification model to obtain a binary classification result; calculate the collagen quantification data of the skin tissue stratification based on the binary classification result; generate a structured report based on the collagen quantification data and the multi-skin tissue probability map, and display the structured report.

[0154] In some embodiments, collagen quantification data includes collagen area ratio, spatial heterogeneity index and elastic modulus gradient distribution; the collagen quantification module is also used to perform morphological smoothing on the binary classification results through a spatial consistency correction strategy to obtain a morphologically smoothed result; the collagen ratio is calculated based on the number of collagen blocks, single block area and total skin area data of the morphological smoothing result to obtain the collagen area ratio; the heterogeneity is calculated based on the collagen block distribution data of the morphological smoothing result to obtain the spatial heterogeneity index; the elastic modulus gradient distribution of the collagen area is obtained based on the binary classification results; a cumulative distribution model is generated based on the elastic modulus gradient distribution; the cumulative distribution model is used to quantify the hardness change trend to obtain the elastic modulus gradient distribution.

[0155] In some embodiments, the device also includes: a joint optimization training module for obtaining training ultrasound images of low-resolution devices and training elastic modulus maps of high-resolution devices; wherein the low-resolution device and the high-resolution device are operated synchronously; based on the training ultrasound images and the training elastic modulus maps, a preset original elastic imaging generation network is trained to obtain a candidate elastic imaging generation network; based on the training ultrasound images, a preset original super-resolution reconstruction network is trained to obtain a candidate super-resolution reconstruction network; wherein there is a feature layer association between the candidate elastic imaging generation network and the candidate super-resolution reconstruction network; the candidate elastic imaging generation network and the candidate super-resolution reconstruction network are alternately jointly trained, and a joint optimization loss model of the candidate elastic imaging generation network and the candidate super-resolution reconstruction network in training is constructed; the candidate elastic imaging generation network and the candidate super-resolution reconstruction network are jointly optimized according to the joint optimization loss model to obtain a target elastic imaging generation network and a target super-resolution reconstruction network.

[0156] In the above-mentioned skin tissue stratification evaluation device, by obtaining the original grayscale ultrasound image of the skin tissue stratification and preprocessing the original grayscale ultrasound image to obtain a preprocessed ultrasound image for subsequent processing and analysis, the target elastic imaging generation network is used to generate an elastic image of the preprocessed ultrasound image, and the target elastic modulus map can be efficiently obtained. The preprocessed ultrasound image is super-resolution reconstructed by the target super-resolution reconstruction network to obtain a high-resolution target high-resolution map; the target elastic imaging generation network and the target super-resolution reconstruction network are trained and generated based on a preset elastic super-resolution joint training strategy, and the detailed features reconstructed by the elastic imaging generation network are embedded in the target super-resolution reconstruction network. The elastic imaging generation network improves the recognition accuracy of tiny structures, extracts features from preprocessed ultrasound images, target elastic modulus maps, and target high-resolution maps to obtain multimodal hierarchical feature maps, performs cross-modal attention fusion on multimodal hierarchical features to obtain fused feature maps, performs hierarchical evaluation of skin tissue layers based on the fused feature maps, obtains multi-skin tissue probability maps, and visualizes the multi-skin tissue probability maps, thereby innovatively realizing the construction of a "grayscale-elasticity-super-resolution" three-modal fusion hierarchical evaluation network, which can break through the limitations of traditional evaluation methods such as high equipment cost, insufficient resolution, quantification, and difficulty in multi-tissue collaborative analysis.

[0157] Each module in the aforementioned skin tissue layer assessment device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor within a computer device in hardware form, or can be stored in a computer device memory in software form, allowing the processor to call and execute the corresponding operations of each module.

[0158] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 8As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store a target elastic imaging generation network and a target super-resolution reconstruction network. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a skin tissue stratification assessment method is implemented.

[0159] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0160] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the above-mentioned skin tissue layering assessment method when executing the computer program.

[0161] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned skin tissue layering assessment method are implemented.

[0162] In one embodiment, a computer program product is provided, comprising a computer program, which implements the steps of the above-mentioned skin tissue layering assessment method when executed by a processor.

[0163] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0164] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), quantum computing-based data processing logic devices, artificial intelligence (AI) processors, and the like.

[0165] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0166] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A skin tissue stratification evaluation method, characterized in that: The method comprises: Acquiring an original grayscale ultrasound image of skin tissue layers, and preprocessing the original grayscale ultrasound image to obtain a preprocessed ultrasound image; generating an elastic image of the preprocessed ultrasound image through a target elastic imaging generation network to obtain a target elastic modulus map; The preprocessed ultrasound image is super-resolution reconstructed by a target super-resolution reconstruction network to obtain a target high-resolution image; the target elastic imaging generation network and the target super-resolution reconstruction network are trained and generated based on a preset elastic super-resolution joint training strategy; performing feature extraction on the preprocessed ultrasound image, the target elastic modulus map, and the target high-resolution map to obtain a multimodal hierarchical feature map; Performing cross-modal attention fusion on the multimodal hierarchical features to obtain a fused feature map; The skin tissue layers are evaluated according to the fusion feature map to obtain a multi-skin tissue probability map, and the multi-skin tissue probability map is visualized.

2. The method according to claim 1, characterized in that The preprocessing of the original grayscale ultrasound image to obtain a preprocessed ultrasound image includes: Determining a target region on the original grayscale ultrasound image to obtain a target ultrasound image; Denoising the target ultrasound image to obtain a denoised ultrasound image; The denoised ultrasonic image is scaled to obtain the preprocessed ultrasonic image.

3. The method according to claim 1, characterized in that The cross-modal attention fusion of the multimodal hierarchical feature map to obtain a fused feature map includes: Performing an affine transformation on the multimodal hierarchical feature map to obtain a multimodal feature map of equal size; Performing channel alignment on the equal-sized multimodal feature maps to obtain equal-channel multimodal feature maps; Performing multimodal graph splicing on the equal-channel multimodal feature graph to obtain a spliced ​​multimodal feature graph; Performing global average pooling on the concatenated multimodal feature map to obtain channel weights; Performing multimodal learning on the channel weights through a preset fully connected layer to obtain a weight vector; Performing weighted summation on the concatenated multimodal feature map according to the weight vector to obtain a channel attention feature map; Performing a convolution operation on the channel attention feature map to obtain a spatial weight matrix; activating the spatial weight matrix through a threshold function to obtain key anatomical boundaries; A fusion feature is constructed based on the key anatomical boundary, the spliced ​​multimodal feature map and the channel weight to obtain the fusion feature map.

4. The method according to claim 1, wherein The method further comprises: Performing image segmentation on the preprocessed ultrasound image, the target elastic modulus map, and the target high-resolution map to obtain image block data; Performing image feature extraction on the image block data to obtain an image feature vector; Classifying the image feature vector according to a preset collagen classification model to obtain a binary classification result; Calculating collagen quantitative data of the skin tissue layer based on the binary classification results; A structured report is generated based on the collagen quantification data and the multiple skin tissue probability map, and the structured report is displayed.

5. The method according to claim 4, characterized in that The collagen quantitative data includes collagen area ratio, spatial heterogeneity index and elastic modulus gradient distribution; the collagen quantitative data of the skin tissue stratification calculated based on the binary classification results includes: Performing morphological smoothing on the binary classification results through a spatial consistency correction strategy to obtain a morphologically smoothed result; Calculating the collagen ratio based on the number of collagen blocks, the area of ​​a single block, and the total skin area of ​​the morphological smoothing result to obtain the collagen area ratio; performing heterogeneity calculation based on the collagen block distribution data of the morphological smoothing result to obtain the spatial heterogeneity index; Obtaining the elastic modulus gradient distribution of the collagen region based on the binary classification result; A cumulative distribution model is generated according to the elastic modulus gradient distribution; the cumulative distribution model is used to quantify the hardness variation trend to obtain the elastic modulus gradient distribution.

6. The method according to claim 1, characterized in that The method further comprises: Acquiring a training ultrasound image of a low-resolution device and a training elastic modulus map of a high-resolution device; wherein the low-resolution device and the high-resolution device are operated synchronously; Training a preset original elastic imaging generation network based on the training ultrasound image and the training elastic modulus map to obtain a candidate elastic imaging generation network; Training a preset original super-resolution reconstruction network based on the training ultrasound image to obtain a candidate super-resolution reconstruction network; wherein a feature layer association exists between the candidate elastic imaging generation network and the candidate super-resolution reconstruction network; Performing alternating joint training on the candidate elastic imaging generation network and the candidate super-resolution reconstruction network, and constructing a joint optimization loss model for the candidate elastic imaging generation network and the candidate super-resolution reconstruction network during training; The candidate elastic imaging generation network and the candidate super-resolution reconstruction network are jointly optimized according to the joint optimization loss model to obtain the target elastic imaging generation network and the target super-resolution reconstruction network.

7. A skin tissue layer evaluation device, characterized in that: The device comprises: an acquisition module, configured to acquire an original grayscale ultrasound image of skin tissue layers, and preprocess the original grayscale ultrasound image to obtain a preprocessed ultrasound image; an elastic image generation module, configured to generate an elastic image of the preprocessed ultrasound image through a target elastic imaging generation network to obtain a target elastic modulus map; An image reconstruction module is configured to perform super-resolution image reconstruction on the pre-processed ultrasound image through a target super-resolution reconstruction network to obtain a target high-resolution image; the target elastic imaging generation network and the target super-resolution reconstruction network are generated based on a preset elastic super-resolution joint training strategy; a feature extraction module, configured to extract features from the preprocessed ultrasound image, the target elastic modulus map, and the target high-resolution map to obtain a multimodal hierarchical feature map; An attention fusion module, configured to perform cross-modal attention fusion on the multimodal hierarchical features to obtain a fused feature map; A stratification evaluation module is used to perform a stratification evaluation on the skin tissue stratification according to the fusion feature map, obtain a multi-skin tissue probability map, and visualize the multi-skin tissue probability map.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images