Skin ct lesion recognition method based on boundary perception and hierarchical sequence supervision

Abstract

The application discloses a skin CT lesion identification method based on boundary perception and hierarchical sequence supervision. The method comprises the following steps: acquiring a skin CT continuous slice sequence to be diagnosed; inputting the continuous slice sequence into a pre-trained double-flow hierarchical network to extract semantic features and boundary features of each slice respectively; generating a gating weight based on the boundary features, performing mask purification on the semantic features to obtain denoised spatial features; outputting a local disease prediction probability based on each denoised spatial feature, and performing time sequence fusion on all denoised spatial features to output a global disease prediction probability. Through the above method, the application can utilize the boundary mask purification to force the noise to zero, significantly improve the noise robustness; construct hierarchical joint supervision to provide local gradient guidance, solve the contribution allocation problem and accelerate the convergence; synchronously output the local probability to break the black box limit, realize the quantitative evaluation and explainable diagnosis of tumor infiltration depth.

Description

Technical Field

[0001] This application relates to the field of medical image processing technology, and in particular to a method for identifying skin CT lesions based on boundary awareness and hierarchical sequence supervision. Background Technology

[0002] Skin reflection confocal microscopy (skin CT) enables non-invasive, layer-by-layer optical scanning along the Z-axis of the skin surface, generating high-resolution three-dimensional continuous slice sequences. In the clinical diagnosis of skin tumors (such as melanoma and basal cell carcinoma), the morphological evolution and invasiveness of tumor cells along the Z-axis depth are the core gold standard for assessing disease malignancy and formulating surgical resection plans. However, existing deep learning-based computer-aided diagnostic technologies face the following critical technical bottlenecks when processing continuous skin CT slice sequences: 1. Difficulty in allocating contributions in multi-instance learning: Existing sequence classification networks typically employ a weakly supervised multi-instance learning paradigm, using only the patient's final "sequence-level gold standard" for network parameter updates. Since only a few slices located at the dermal-epidermal junction often contain key pathological atypia in long sequences, single global supervision can prevent the network from accurately tracing global errors to local key pathological slices during backpropagation, leading to a high likelihood of overfitting and getting trapped in local optima.

[0003] 2. Strong coupling interference between blurred microscopic boundaries and high-frequency speckle noise: The principle of skin CT imaging determines that its image contrast is low and contains a large amount of inherent optical speckle noise. Existing sequence models usually directly feed unpurified two-dimensional slice features into temporal networks, which not only leads to the loss of cell microscopic topological boundaries, but also causes speckle noise to be nonlinearly amplified during deep sequence fusion, resulting in an extremely high false positive rate.

[0004] 3. "Black box" models lack clinical interpretability of vertical invasion depth: Existing end-to-end sequence diagnostic models only output the final global disease category, which cannot map and quantify the risk distribution of tumors on slices at different depths for clinicians, making it difficult to meet clinicians' actual needs for planning the physical boundaries of lesions. Summary of the Invention

[0005] This application provides a skin CT lesion identification method based on boundary awareness and hierarchical sequence supervision to solve the problems of long sequence contribution allocation failure, microscopic boundary blurring and strong coupling interference from high-frequency speckle noise, and lack of clinical interpretability due to vertical infiltration depth in the prior art.

[0006] To address the aforementioned technical problems, this application adopts the following technical solution: a method for identifying skin CT lesions based on boundary awareness and hierarchical sequence supervision, the method comprising: Obtain sequential CT slices of the skin to be diagnosed; The continuous slice sequence is input into a pre-trained two-stream hierarchical network to extract the semantic features and boundary features of each slice. Based on the boundary features, gating weights are generated, and the semantic features are purified by masking to obtain denoised spatial features; The local disease prediction probability is output based on each of the denoised spatial features, and the global disease prediction probability is output by temporal fusion of all the denoised spatial features.

[0007] In an optional embodiment of this application, the step of generating gating weights based on the boundary features and performing masking purification on the semantic features to obtain denoised spatial features includes: The boundary features are subjected to channel compression and nonlinear mapping to generate the gating weights; The gating weights are multiplied element-wise with the semantic features to reduce the background noise activation values ​​to zero, thus obtaining the denoised spatial features.

[0008] In an optional embodiment of this application, the step of outputting a local disease prediction probability based on each of the denoised spatial features and performing temporal fusion on all the denoised spatial features to output a global disease prediction probability includes: Each of the denoised spatial features is converted into a one-dimensional feature vector, and the one-dimensional feature vector is classified to obtain the local disease prediction probability. The one-dimensional feature vectors of each slice within the same sequence are concatenated in depth order and superimposed with positional codes to obtain a sequence feature matrix; Deep context features are extracted based on the sequence feature matrix, and the deep context features are classified to obtain the global disease prediction probability.

[0009] In an optional embodiment of this application, the training process of the two-stream hierarchical network includes the following: Obtain the true labels of local diseases and the true labels of global diseases; Slice-level cross-entropy loss is calculated based on the local disease prediction probability and the local disease true label, and sequence-level cross-entropy loss is calculated based on the global disease prediction probability and the global disease true label; The two-stream hierarchical network is optimized end-to-end based on the hierarchical joint loss function composed of the weighted sum of the slice-level cross-entropy loss and the sequence-level cross-entropy loss.

[0010] In one optional embodiment of this application, the end-to-end optimization of the two-stream hierarchical network based on the hierarchical joint loss function weighted by the slice-level cross-entropy loss and the sequence-level cross-entropy loss includes: At the start of training, the first weight corresponding to the slice-level cross-entropy loss is set to be greater than the second weight corresponding to the sequence-level cross-entropy loss. During the training iteration, a cosine annealing strategy is used to smoothly reduce the first weight and smoothly increase the second weight until the training ends and the second weight is greater than the first weight.

[0011] In an optional embodiment of this application, after outputting the local disease prediction probability, the method further includes: For each slice, the corresponding malignancy prediction probability is obtained from the local disease prediction probability; Based on the depth of each slice and the corresponding malignancy prediction probability, a disease risk change curve is plotted to assess the vertical infiltration depth of the lesion.

[0012] In an optional embodiment of this application, before inputting the continuous slice sequence into a pre-trained two-stream hierarchical network, the method further includes: The continuous slice sequence is rigidly registered based on the translation vector between adjacent slices. The registered continuous slice sequence is uniformly resampled to a fixed number of layers, and contrast-limited adaptive histogram equalization and uniform scaling are performed on each slice to obtain the standardized continuous slice sequence.

[0013] To address the aforementioned technical problems, another technical solution adopted in this application is: providing a skin CT lesion identification device based on boundary awareness and hierarchical sequence supervision, the device comprising: The acquisition module is used to acquire the continuous slice sequence of the skin CT scan to be diagnosed; The extraction module is used to input the continuous slice sequence into a pre-trained two-stream hierarchical network to extract the semantic features and boundary features of each slice respectively; The mask module is used to generate gating weights based on the boundary features and to perform mask purification on the semantic features to obtain denoised spatial features; The output module is used to output a local disease prediction probability based on each of the denoised spatial features, and to perform temporal fusion on all the denoised spatial features to output a global disease prediction probability.

[0014] To solve the above-mentioned technical problems, another technical solution adopted in this application is: to provide a computer device, including a memory, a processor and a computer program stored in the memory, characterized in that the processor executes the computer program to implement the steps of the above-mentioned skin CT lesion recognition method based on boundary perception and hierarchical sequence supervision.

[0015] To solve the above-mentioned technical problems, another technical solution adopted in this application is: to provide a storage medium storing a computer program thereon, characterized in that the computer program, when executed by a processor, implements the steps of the above-mentioned skin CT lesion recognition method based on boundary perception and hierarchical sequence supervision.

[0016] The beneficial effects of this application are as follows: Unlike existing technologies, this application discloses a skin CT lesion identification method based on boundary awareness and hierarchical sequence supervision. This method constructs a dual-stream hierarchical network and introduces boundary awareness and mask purification mechanisms. It utilizes extracted high-resolution boundary features to generate gating weights, performs spatial mask purification on deep semantic features, and mathematically forces the activation values ​​of background speckle noise without physical boundaries to zero, thus blocking the cascading amplification of noise in subsequent temporal networks and significantly improving the model's robustness against speckle noise. Simultaneously, by outputting local and global disease prediction probabilities based on denoised spatial features, a hierarchical joint supervision system with dual local and global constraints is constructed. Local slice-level supervision provides clear local gradient guidance for the network, completely solving the contribution allocation problem in long-sequence multi-instance learning and significantly accelerating convergence. Furthermore, the synchronously output local probabilities can intuitively reflect the risk distribution of lesions along the Z-axis depth, breaking the limitations of traditional black-box models and achieving quantitative assessment and clinically interpretable diagnosis of tumor vertical invasion depth. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein: Figure 1 This is a flowchart illustrating an embodiment of the skin CT lesion identification method based on boundary awareness and hierarchical sequence supervision provided in this application; Figure 2 This is a schematic diagram of an embodiment of the skin CT lesion identification device based on boundary perception and hierarchical sequence supervision provided in this application; Figure 3 This is a schematic diagram of the structure of an embodiment of the storage medium provided in this application; Figure 4 This is a schematic diagram of the structure of an embodiment of the computer device provided in this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0019] The terms "first," "second," and "third" used in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first," "second," or "third" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0020] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0021] This application provides a method for identifying skin CT lesions based on boundary awareness and hierarchical sequence supervision. This method relies on a deep learning framework, such as PyTorch. In terms of hardware, the model training process can be implemented using a high-performance graphics processor, such as an NVIDIA RTX 3090, to meet the rapid iteration requirements of the large-scale parameter matrix of the deep learning network. The diagnostic objective of this embodiment is exemplarily set as classifying skin CT sequences into three categories: benign, basal cell carcinoma, or melanoma. However, those skilled in the art will understand that the classification system of this method can be expanded according to actual clinical needs.

[0022] See Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the skin CT lesion identification method based on boundary awareness and hierarchical sequence supervision provided in this application. The skin CT lesion identification method based on boundary awareness and hierarchical sequence supervision includes: S10: Obtain the continuous CT slice sequence of the skin to be diagnosed.

[0023] Specifically, it involves acquiring a sequence of original skin reflective confocal microscopy (SEM) slices of the lesion to be diagnosed. SEM can perform layer-by-layer optical scanning along the Z-axis of the skin surface under non-invasive conditions, generating a high-resolution three-dimensional continuous slice sequence.

[0024] In the clinical diagnosis of skin tumors, the morphological evolution and invasiveness of tumor cells along the Z-axis depth direction are the core gold standard for assessing the malignancy of the disease. However, the raw, continuous slice sequences often suffer from problems such as interlayer displacement, inconsistent depth, and low contrast, which can severely affect the accuracy of feature extraction if directly input into the network. Therefore, before inputting the continuous slice sequences into a pre-trained two-stream hierarchical network, they need to undergo standardization preprocessing.

[0025] In an alternative embodiment, before inputting the consecutive slice sequences into a pre-trained two-stream hierarchical network, the method further includes: S11: Perform rigid registration on a continuous slice sequence based on the translation vector between adjacent slices.

[0026] After acquiring the original continuous slice sequence of skin CT, spatial translational misalignment can occur between adjacent slices due to the patient's breathing movements or minor shifts during the scan. To eliminate this inter-slice displacement and ensure the spatial coherence of the sequence in the Z-axis direction, this embodiment employs a phased cross-correlation algorithm to calculate the translation vector between adjacent slices for rigid registration.

[0027] The phased cross-correlation algorithm calculates the cross-correlation power spectrum of two adjacent slice images in the frequency domain and uses inverse Fourier transform to obtain the translation peak position in the spatial domain, thereby accurately estimating the translation vectors in the horizontal and vertical directions. Based on this translation vector, reverse translation compensation is applied to the slices, achieving rigid alignment between adjacent slices and effectively eliminating inter-layer spatial displacement artifacts.

[0028] S12: The registered continuous slice sequence is uniformly resampled to a fixed number of layers, and contrast-limited adaptive histogram equalization and uniform scaling are performed on each slice to obtain the standardized continuous slice sequence.

[0029] Because lesion depths vary among patients, the number of slices in the obtained serial slicing sequences often differs. To meet the requirement of a fixed input dimension for the subsequent depth-time fusion module, the registered sequences need to be uniformly resampled to a fixed number of slices, D. The specific value of the fixed number of slices, D, is usually determined in consultation with clinicians based on the depth coverage required for the pathological features; for example, in this embodiment, D=32. The resampling process uses an interpolation algorithm to uniformly select D slices along the Z-axis. Sequences with insufficient slices are copied and filled, while sequences with excessive slices are sampled at intervals.

[0030] Furthermore, the principles of skin CT imaging result in images with low contrast and a significant amount of inherent optical speckle noise. To enhance the visibility of subtle pathological structures in the slices, contrast-limited adaptive histogram equalization is performed on each slice. Unlike global histogram equalization, this algorithm divides the image into multiple local sub-blocks, calculates the local histogram within each sub-block, and limits the contrast, effectively preventing background noise from being excessively amplified while significantly improving the local contrast of the lesion area.

[0031] Finally, the equalized slices are scaled uniformly to a fixed pixel height H and width W, for example, H=256, W=256, thereby constructing a three-dimensional tensor data with dimensions [B,32,1,256,256], where B is the batch size.

[0032] The skin CT lesion identification method provided in this application based on boundary awareness and hierarchical sequence supervision eliminates inter-slice displacement by rigidly registering continuous slice sequences, meets the fixed-dimensional input requirements of the temporal fusion module by uniform resampling, and significantly improves the local contrast of the lesion area by limiting contrast adaptive histogram equalization and uniform scaling while suppressing excessive amplification of background noise. This provides high-quality and standardized input data for the subsequent two-stream hierarchical network, effectively ensuring the accuracy and stability of feature extraction and lesion identification.

[0033] S20: Input the continuous slice sequence into a pre-trained two-stream hierarchical network to extract the semantic features and boundary features of each slice.

[0034] A spatial-depth-based dual-stream hierarchical network is pre-constructed and trained. Logically, the dual-stream hierarchical network consists of a two-dimensional spatial feature extraction module and a slice-level classification branch for processing two-dimensional spatial streams, and a deep sequence fusion module and a sequence-level classification branch for processing one-dimensional depth streams.

[0035] The standardized 3D tensor data is fed into the 2D spatial feature extraction module layer by layer according to physical depth. For the i-th slice, the network extracts semantic features and boundary features through two parallel branches, as follows: The process of extracting semantic features is implemented by a semantic awareness branch. This branch uses a ResNet-34 network with fully connected layers removed as its backbone. ResNet-34 introduces residual connections, effectively mitigating the vanishing gradient problem during deep network training. The input slice undergoes a series of downsampling convolutions and residual block operations to progressively extract deep semantic features containing rich pathological semantic information, denoted as... Its tensor dimensions are [B, 512, 16, 16]. Here, 512 represents the number of channels in the feature map, and 16 and 16 represent the height and width of the feature map, respectively. Deep semantic features contain high-order abstract information such as cell morphology and tissue structure in slices; however, due to the downsampling characteristics of convolution operations, their spatial resolution is low, and some microscopic boundary information is lost.

[0036] It should be noted that the backbone network for extracting semantic features is not limited to ResNet-34 and can be flexibly replaced according to the computing power requirements of the actual medical terminal device. For example, on a server with sufficient computing power, it can be seamlessly replaced with a deeper ResNet-50 or DenseNet to obtain stronger feature representation capabilities, while on a mobile or portable device with limited computing power, it can be replaced with the lightweight MobileNet, which can also achieve the purpose of extracting pathological semantic features.

[0037] The process of extracting boundary features is implemented by a boundary-aware branch. To compensate for the shortcomings of semantic features in perceiving microscopic boundaries, this invention establishes an independent boundary-aware branch to extract high-resolution physical edges. First, the Canny operator is used to extract high-frequency physical edge maps of the original slices. The Canny operator includes steps such as Gaussian smoothing, gradient calculation, non-maximum suppression, and double threshold detection, which can accurately delineate physical contours with drastic grayscale changes in the image. Then, the extracted high-frequency physical edge maps are input into a shallow network consisting of three consecutive convolutional layers to extract high-resolution boundary features, denoted as . Because it uses a shallow network and avoids oversampling, the boundary features retain extremely high spatial resolution, enabling precise localization of cell topological boundaries and lesion contours. It should be noted that the boundary feature extraction operator is not limited to the Canny operator; it can also be equivalently replaced by other first- or second-order differential edge detection operators such as Sobel, Prewitt, and Laplacian, all of which can achieve the goal of high-frequency physical boundary extraction.

[0038] S30: Gating weights are generated based on boundary features, and semantic features are purified by masking to obtain denoised spatial features.

[0039] High-frequency speckle noise, prevalent in skin CT images, is strongly coupled with blurred microscopic boundaries. If unpurified semantic features are directly fed into a temporal network, the speckle noise will be nonlinearly amplified during deep sequence fusion, leading to a high false positive rate. Therefore, this application innovatively introduces a gated mask purification mechanism to spatially purify and denoise deep semantic features using high-resolution boundary features.

[0040] In an optional embodiment, step S30 above further includes: S31: Perform channel compression and nonlinear mapping on the boundary features to generate gated weights.

[0041] Specifically, the boundary features extracted by the boundary-aware branch The input is fed into the gated mask purification module. First, it undergoes channel compression through a 1×1 convolutional layer. The number of output channels of this 1×1 convolutional layer is set to 1. This parameter directly compresses the multi-channel boundary features into a single channel, resulting in a more efficient spatial gated weight matrix. The dimension is reduced to [B,1,H,W]. The purpose of compressing the number of channels to 1 is to meet the computational requirements of the tensor broadcasting mechanism, ensuring that when performing subsequent operations with the 512-channel semantic features, the mask weight of a single channel can be applied consistently and uniformly to all channels of the deep semantic features, thereby achieving uniform zeroing suppression of background speckle noise.

[0042] Subsequently, the result of the 1×1 convolution compression is input into the Sigmoid activation function for nonlinear mapping, mapping it to the [0,1] interval, thereby generating the spatial gated weight matrix. The closer the output value of the Sigmoid function is to 1, the higher the confidence that a high-frequency physical boundary exists at that spatial location; the closer it is to 0, the greater the likelihood that the location belongs to a smooth background or speckle noise.

[0043] S32: Multiply the gating weights element-wise with the semantic features to reduce the background noise activation values ​​to zero, thus obtaining the denoised spatial features.

[0044] Specifically, the generated spatial gating weight matrix Deep semantic features extracted from semantic perception branch Element-wise multiplication under the space broadcast mechanism is mathematically known as the Hadamard product, and its formula is as follows: , formula 1; in, The denoised spatial features are obtained after the i-th slice is purified by masking. The spatial gating weight matrix generated for the boundary features of the i-th slice; The deep semantic features extracted from the i-th slice through the semantic perception branch; This is an element-wise multiplication operation, also known as Hadamard product, used to multiply the elements at corresponding positions of two tensors with the same dimension.

[0045] The essence of this step is the soft mask or gating principle in spatial attention mechanisms. After processing by the Sigmoid activation function, each pixel value in the spatial gating weight matrix is ​​mapped to the [0,1] interval; among them, the weights of regions with high-frequency physical boundaries approach 1, while the weights of smooth background regions filled with speckle noise are compressed to approach 0. When performing element-wise multiplication, the activation values ​​of the deep semantic features corresponding to the background noise positions are forcibly multiplied by a very small value, approximately 0, and thus directly filtered out mathematically; while the effective features that truly contain lesion boundaries are completely preserved because they are multiplied by a weight of approximately 1.

[0046] Unlike existing technologies where blurred microscopic boundaries in skin CT images are strongly coupled with high-frequency speckle noise, leading to cascading amplification of speckle noise in temporal networks and resulting in high false positive rates, this application provides a skin CT lesion identification method based on boundary awareness and hierarchical sequence supervision. This method generates gate weights by channel compression and nonlinear mapping of boundary features, and then multiplies these gate weights element-wise with semantic features. Mathematically, this forces the activation values ​​of background speckle noise without high-frequency physical boundaries to zero, ensuring that the features fed into the subsequent temporal network are pure pathological features with high signal-to-noise ratio. This effectively blocks the cascading amplification of noise and significantly improves the model's robustness against speckle noise interference.

[0047] S40: Output the local disease prediction probability based on each denoised spatial feature, and perform temporal fusion on all denoised spatial features to output the global disease prediction probability.

[0048] The denoised spatial features, after masking and purification, not only contain rich local pathological semantics but also retain precise spatial boundary information. To achieve multi-granularity diagnosis, this step analyzes the features from both spatial and depth dimensions.

[0049] In an optional embodiment, step S40 above further includes: S41: Convert each denoised spatial feature into a one-dimensional feature vector, and perform classification processing on the one-dimensional feature vector to obtain the local disease prediction probability.

[0050] Specifically, for the denoising spatial features of each slice First, the feature map undergoes a spatial flattening operation using a global average pooling layer. The global average pooling layer calculates the average value of the feature map across the spatial dimensions, compressing the feature tensor of dimension [B, 512, 16, 16] into a one-dimensional feature vector of dimension [B, 512]. This operation not only significantly reduces the number of parameters but also preserves the global spatial statistics of each channel.

[0051] Subsequently, the one-dimensional feature vector is input into the slice-level classification branch for classification processing. This classification branch first passes through a fully connected hidden layer containing 256 neurons, followed by a ReLU activation function and a Dropout layer. The ReLU activation function introduces non-linear mapping capability, and the Dropout layer randomly discards some neurons during training to prevent overfitting at the slice level. Finally, it is connected to an output fully connected layer containing 3 neurons, and then processed by a Softmax activation function to output the local disease prediction probability distribution of a single slice. The Softmax function maps the output to a probability distribution, with the three neurons corresponding to the predicted probabilities of benign, basal cell carcinoma, and melanoma categories, respectively. The output of local probabilities provides the necessary local gradient signals for subsequent hierarchical joint supervision.

[0052] S42: Concatenate the one-dimensional feature vectors of each slice within the same sequence in depth order, and superimpose the positional encoding to obtain the sequence feature matrix.

[0053] The diagnosis of skin CT sequences depends not only on the pathological features of a single slice, but also on the evolutionary logic between slices along the Z-axis. Therefore, the flattened feature vectors of D slices within the same sequence are spliced ​​together in order of physical depth along the Z-axis to construct a sequence feature matrix of dimension [B,D,512].

[0054] Since self-attention mechanisms such as Transformer lack position awareness, this embodiment superimposes a sine and cosine position encoding matrix onto the sequence feature matrix to preserve the absolute spatial position information of the slices along the Z-axis. The position encoding is generated using sine and cosine functions of different frequencies, enabling the network to distinguish slice features at different depth levels, thus laying the foundation for subsequent extraction of contextual dependencies in the depth direction.

[0055] S43: Extract deep context features based on the sequence feature matrix, and classify the deep context features to obtain the global disease prediction probability.

[0056] The sequence feature matrix after overlay position encoding is input to the deep sequence fusion module. This sequence feature matrix is ​​formed by concatenating the feature vectors of each slice within the same sequence according to depth. Therefore, each feature vector in the matrix represents the slice feature at the corresponding depth position. In this embodiment, the deep sequence fusion module uses a Transformer encoder with a two-layer multi-head self-attention mechanism. The input feature dimension of the Transformer encoder is set to... To match the flattened feature dimension of the feature extraction module output; in the multi-head self-attention mechanism, the number of attention heads is set to 8, so that the feature subspace dimension processed by each attention head is equally divided into 64, which helps the model to fully capture the complex Z-axis dependencies between different slices from multiple parallel representation subspaces; in addition, the hidden layer extension dimension of the feedforward neural network inside the encoder is set to 2048.

[0057] Inside the Transformer encoder, a multi-head self-attention mechanism is used to calculate the dot product weights of the query (Q) and key (K) among the features of each slice, thereby determining the correlation between the current slice feature and other deep slice features in the sequence. Subsequently, based on this correlation weight, the values ​​(V) corresponding to each slice are weighted and summed, ensuring that each slice feature aggregates the contextual information of other slices in the entire sequence, thus adaptively extracting deep contextual features that incorporate the evolution logic along the Z-axis. After processing, the Transformer encoder outputs a deep contextual feature matrix with the same dimensions as the input sequence feature matrix.

[0058] To obtain a feature representation that characterizes the global properties of the entire sequence, the output deep context feature matrix is ​​aggregated, for example, by performing global average pooling along the depth dimension, thus compressing the matrix into a one-dimensional global sequence feature vector. Finally, the obtained global sequence feature vector is input into a sequence-level classification branch, which also consists of fully connected layers and a softmax activation function, to classify the global sequence feature vector, ultimately outputting the global disease prediction probability distribution for the entire continuous slice sequence. .

[0059] It should be noted that the time sequence model is not limited to the Transformer encoder, but can also be replaced by a bidirectional long short-term memory network (Bi-LSTM) or a gated recurrent unit (GRU), which can also achieve the fusion and extraction of the Z-axis physical depth context temporal information.

[0060] The skin CT lesion recognition method provided in this application, based on boundary awareness and hierarchical sequence supervision, obtains local prediction probabilities by converting denoised spatial features into one-dimensional feature vectors and performing classification processing. At the same time, it extracts depth context features by splicing them in depth order and superimposing position codes to obtain global prediction probabilities. This achieves dual-stream hierarchical parsing of local spatial features of slices and global depth evolution logic of sequences, providing accurate feature representation and probabilistic foundation for subsequent hierarchical joint supervision and multi-granularity diagnosis.

[0061] To enable the aforementioned two-stream hierarchical network to accurately identify lesions, this embodiment designs a hierarchical joint supervision training strategy, completely solving the contribution allocation problem in long-sequence multi-instance learning. The training process of this two-stream hierarchical network specifically includes the following sub-steps: S51: Obtain the true labels of local diseases and the true labels of global diseases.

[0062] During the model training phase, pre-labeled supervision labels need to be obtained. Specifically, for the i-th slice, its corresponding local disease true classification label is obtained. For the entire sequence, obtain the patient's final global gold standard label. Among them, local labels indicate the pathological state of a single slide, while global labels indicate the overall diagnostic result of the patient. Together, they form the labeling basis for hierarchical supervision.

[0063] S52: Calculate slice-level cross-entropy loss based on local disease prediction probability and local disease true label, and calculate sequence-level cross-entropy loss based on global disease prediction probability and global disease true label.

[0064] After obtaining the predicted probabilities through network forward propagation, the cross-entropy loss function is used to measure the difference between the predicted and actual values.

[0065] Slice-level cross-entropy loss The calculation formula is: , formula 2; in, The slice-level cross-entropy loss is used to measure the overall error between the local disease prediction probability and the local true label for all slices. Let be the local disease prediction probability distribution for the i-th slice; The true classification label for the local disease in the i-th slice; The total number of slices in a continuous slice sequence (i.e., a fixed number of layers), for example, D=32; This is the cross-entropy loss calculation function, used to measure the inconsistency between two probability distributions.

[0066] Sequence-level cross-entropy loss The calculation formula is: , formula 3; in, The sequence-level cross-entropy loss measures the error between the global disease prediction probability and the global true label for the entire sequence. The global disease prediction probability distribution for the entire continuous slice sequence; The ultimate global gold standard label for patients is the global true disease label for the entire sequence; This is the cross-entropy loss calculation function, used to measure the inconsistency between two probability distributions.

[0067] S53: End-to-end optimization of two-stream hierarchical networks is performed based on a hierarchical joint loss function composed of slice-level cross-entropy loss and sequence-level cross-entropy loss weighted together.

[0068] Hierarchical joint loss function It is composed of a weighted average of slice-level cross-entropy loss and sequence-level cross-entropy loss, and its formula is: , formula 4; in, is the hierarchical joint loss function, which is the overall objective function used for end-to-end optimization of a two-stream hierarchical network; The loss is the slice-level cross-entropy loss; For sequence-level cross-entropy loss; The first weight corresponding to the slice-level cross-entropy loss; This is the second weight corresponding to the sequence-level cross-entropy loss.

[0069] Unlike existing technologies where single global supervision leads to difficulties in allocating contributions for long-sequence multi-instance learning and the inability to trace global errors back to local key pathological slices, resulting in model overfitting, this application provides a skin CT lesion recognition method based on boundary awareness and hierarchical sequence supervision. This method performs end-to-end optimization by constructing a hierarchical joint loss function composed of a weighted average of slice-level cross-entropy loss and sequence-level cross-entropy loss. Local slice-level supervision forces the feature extractor to accurately capture cellular atypia, while global sequence-level supervision guides the network to understand the vertical evolution topological logic, thus providing clear local gradient guidance. This completely solves the problem of long-sequence contribution allocation and significantly accelerates the network's convergence speed.

[0070] To address the challenge of allocating contributions in long sequence learning, a dynamic decaying annealing strategy can be used to adjust the first and second weights.

[0071] In an optional embodiment, step S53 further includes: S531: At the start of training, set the first weight corresponding to the slice-level cross-entropy loss to be greater than the second weight corresponding to the sequence-level cross-entropy loss.

[0072] At the start of training, the first weight α corresponding to the slice-level cross-entropy loss is set to be greater than the second weight β corresponding to the sequence-level cross-entropy loss, for example, α=0.8 and β=0.2. Since only a few slices in long sequences contain key pathological atypia, single global supervision will cause the contribution allocation to fail. Therefore, in the early stage of training, a high-intensity local gradient is used to force the network to prioritize the extraction of cellular atypia features, forcing the feature extractor to accurately locate key slices.

[0073] S532: During the training iteration, a cosine annealing strategy is used to smoothly reduce the first weight and smoothly increase the second weight until the training ends and the second weight is greater than the first weight.

[0074] During training iterations, a cosine annealing strategy is used to smoothly decrease the first weight α and smoothly increase the second weight β. The mathematical expression of the cosine annealing strategy follows the decreasing and increasing trajectory of the cosine function, avoiding model oscillations caused by abrupt changes in weights.

[0075] By the end of training, the second weight β is greater than the first weight α. It's important to note that this doesn't mean the second weight β only just surpasses the first weight α at the end of training. Rather, throughout the training iterations, with the cosine annealing strategy, the second weight β gradually increases and smoothly surpasses the gradually decreasing first weight α, ensuring that the second weight β is greater than the first weight α by the end of training. For example, it can eventually smoothly transition to α=0.2 and β=0.8. In the mid-to-late stages when the model has achieved accurate local perception capabilities, increasing the weights of the global loss guides the network to converge to the complex vertical infiltration evolution logic at the sequence level.

[0076] The skin CT lesion recognition method provided in this application, based on boundary awareness and hierarchical sequence supervision, assigns greater weight to slice-level loss in the early stage of training, forces the network to prioritize the extraction of cellular atypia features with high-intensity local gradients, and smoothly transitions to assigning greater weight to sequence-level loss in the later stage of training, guiding the network to converge to the complex vertical infiltration evolution logic at the sequence level. This dynamic decaying annealing strategy takes into account both the accurate capture of local features and the understanding of global evolution logic, further improving the stability of model optimization and the final recognition performance.

[0077] Based on the hierarchical joint loss function with dynamic weights described above, the AdamW optimization algorithm is used to update the global parameters until convergence, thereby achieving end-to-end optimization of the two-stream hierarchical network. Specifically, the AdamW optimization algorithm decouples weight decay from the Adam algorithm, improving generalization ability.

[0078] In an optional embodiment, after outputting the local disease prediction probability in step S40, clinical inference and invasion depth visualization can be performed, specifically including the following sub-steps: S61: For each slice, obtain the corresponding malignancy prediction probability from the local disease prediction probability.

[0079] During the clinical inference phase of the model, the network trained on the sequence to be diagnosed is input, and the network front-end synchronously outputs the global final diagnostic probability of the lesion sequence. Simultaneously, for each slice in the sequence, the generated local disease prediction probability is analyzed. The distribution of the tumor is analyzed, and the probability value corresponding to the malignant tumor category is extracted as the malignancy prediction probability. For example, if the model output includes probabilities for benign, basal cell carcinoma, and melanoma, the probabilities corresponding to basal cell carcinoma and melanoma are added together or extracted separately as the malignancy risk indicator for that slice depth.

[0080] S62: Based on the depth of each slice and the corresponding malignancy prediction probability, plot the disease risk change curve to assess the vertical infiltration depth of the lesion.

[0081] Specifically, a disease risk variation curve is plotted with the physical depth along the Z-axis as the horizontal axis and the malignancy prediction probability corresponding to each slice as the vertical axis. This curve visually reflects the spatial evolution of tumor malignancy along the skin depth direction.

[0082] By identifying the mutation point in the localization curve that exceeds the safety threshold, clinical assessment of the vertical invasion depth of tumors can be aided. The safety threshold is a conventional, empirically-based dynamic threshold set by those skilled in the art based on prior clinical knowledge and the model's output distribution.

[0083] In practice, the local disease prediction probability output by the model is strictly mapped to the [0,1] interval. When the slice probability curve along the Z-axis crosses this safety threshold, it marks the appearance of the physical boundary of vertical tumor invasion in medical logic. This safety threshold can be adaptively adjusted based on the sensitivity requirements of actual medical equipment.

[0084] Unlike existing end-to-end sequence diagnostic models, which are essentially "black boxes" outputting only global disease categories and lacking clinical interpretability of vertical invasion depth, thus failing to meet the needs of surgical boundary planning, this application provides a skin CT lesion identification method based on boundary awareness and hierarchical sequence supervision. By obtaining the malignancy prediction probability from the local disease prediction probability and plotting a depth-based disease risk change curve, clinicians can intuitively locate the initiation and termination layers of tumor cell invasion, achieving a quantitative assessment of the vertical invasion depth of the tumor. This truly breaks the black box limitation and provides interpretable clinical auxiliary diagnostic evidence for surgical boundary planning.

[0085] This application provides a skin CT lesion identification device based on boundary awareness and hierarchical sequence supervision, see reference. Figure 2 , Figure 2 This is a schematic diagram of an embodiment of the skin CT lesion identification device based on boundary awareness and hierarchical sequence supervision provided in this application. The skin CT lesion identification device based on boundary awareness and hierarchical sequence supervision includes: Module 10 is used to acquire the continuous CT slice sequence of the skin to be diagnosed; Extraction module 20 is used to input the continuous slice sequence into a pre-trained two-stream hierarchical network to extract the semantic features and boundary features of each slice respectively; Mask module 30 is used to generate gate weights based on boundary features, perform mask purification on semantic features, and obtain denoised spatial features; The output module 40 is used to output a local disease prediction probability based on each of the denoised spatial features, and to perform temporal fusion on all the denoised spatial features to output a global disease prediction probability.

[0086] In an optional embodiment, the mask module 30 is specifically used for: The boundary features are subjected to channel compression and nonlinear mapping to generate the gating weights; The gating weights are multiplied element-wise with the semantic features to reduce the background noise activation values ​​to zero, thus obtaining the denoised spatial features.

[0087] In an optional embodiment, the output module 40 is specifically used for: Each of the denoised spatial features is converted into a one-dimensional feature vector, and the one-dimensional feature vector is classified to obtain the local disease prediction probability. The one-dimensional feature vectors of each slice within the same sequence are concatenated in depth order and superimposed with positional codes to obtain a sequence feature matrix; Deep context features are extracted based on the sequence feature matrix, and the deep context features are classified to obtain the global disease prediction probability.

[0088] In an optional embodiment, the apparatus further includes a training module for training the two-stream hierarchical network, the training module specifically being used for: Obtain the true labels of local diseases and the true labels of global diseases; Slice-level cross-entropy loss is calculated based on the local disease prediction probability and the local disease true label, and sequence-level cross-entropy loss is calculated based on the global disease prediction probability and the global disease true label; The two-stream hierarchical network is optimized end-to-end based on the hierarchical joint loss function composed of the weighted sum of the slice-level cross-entropy loss and the sequence-level cross-entropy loss.

[0089] In an optional embodiment, when the training module performs end-to-end optimization of the two-stream hierarchical network based on a hierarchical joint loss function weighted by the slice-level cross-entropy loss and the sequence-level cross-entropy loss, it is specifically used for: At the start of training, the first weight corresponding to the slice-level cross-entropy loss is set to be greater than the second weight corresponding to the sequence-level cross-entropy loss. During the training iteration, a cosine annealing strategy is used to smoothly reduce the first weight and smoothly increase the second weight until the training ends and the second weight is greater than the first weight.

[0090] In an optional embodiment, the output module 40 is further configured to: For each slice, the corresponding malignancy prediction probability is obtained from the local disease prediction probability; Based on the depth of each slice and the corresponding malignancy prediction probability, a disease risk change curve is plotted to assess the vertical infiltration depth of the lesion.

[0091] In an optional embodiment, the acquisition module 10 is further configured to: The continuous slice sequence is rigidly registered based on the translation vector between adjacent slices. The registered continuous slice sequence is uniformly resampled to a fixed number of layers, and contrast-limited adaptive histogram equalization and uniform scaling are performed on each slice to obtain the standardized continuous slice sequence.

[0092] Since the embodiments of the device part correspond to the embodiments of the above method, the description of the skin CT lesion recognition device based on boundary perception and hierarchical sequence supervision provided in this application is based on the above method embodiments. This application will not repeat the description here. It has the same beneficial effects as the skin CT lesion recognition method based on boundary perception and hierarchical sequence supervision.

[0093] See Figure 3 , Figure 3 This is a schematic diagram of an embodiment of the storage medium provided in this application.

[0094] The storage medium 300 stores program data 310, which, when executed by the processor, implements, as follows: Figure 1 The steps of the described method for identifying skin CT lesions based on boundary awareness and hierarchical sequence supervision.

[0095] The program data 310 is stored in a storage medium 300 and includes several instructions for causing a network device (which may be a router, personal computer, server, or other network device) or processor to execute all or part of the steps of the methods described in the various embodiments of this application.

[0096] Optionally, the storage medium 300 can be any medium capable of storing program data, such as a USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), disk, or optical disc.

[0097] See Figure 4 , Figure 4 This is a schematic diagram of the structure of an embodiment of the computer device provided in this application.

[0098] The device 400 includes a processor 420 and a memory 410 interconnected. The memory 410 stores a computer program, and when the processor 420 executes the computer program, it implements, for example, Figure 1 The steps of the described method for identifying skin CT lesions based on boundary awareness and hierarchical sequence supervision.

[0099] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, the storage medium embodiments and computer device embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0100] This application can be used in a wide range of general-purpose or special-purpose computing system environments or configurations. For example: personal computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, network PCs, minicomputers, distributed computing environments including any of the above systems or devices, etc.

[0101] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed.

[0102] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0103] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0104] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for identifying skin lesions on CT scans based on boundary awareness and hierarchical sequence supervision, characterized in that, include: Obtain sequential CT slices of the skin to be diagnosed; The continuous slice sequence is input into a pre-trained two-stream hierarchical network to extract the semantic features and boundary features of each slice. Based on the boundary features, gating weights are generated, and the semantic features are purified by masking to obtain denoised spatial features; The local disease prediction probability is output based on each of the denoised spatial features, and the global disease prediction probability is output by temporal fusion of all the denoised spatial features.

2. The skin CT lesion identification method based on boundary awareness and hierarchical sequence supervision according to claim 1, characterized in that, The step of generating gating weights based on the boundary features and performing masking purification on the semantic features to obtain denoised spatial features includes: The boundary features are subjected to channel compression and nonlinear mapping to generate the gating weights; The gating weights are multiplied element-wise with the semantic features to reduce the background noise activation values ​​to zero, thus obtaining the denoised spatial features.

3. The skin CT lesion identification method based on boundary awareness and hierarchical sequence supervision according to claim 1, characterized in that, The step of outputting a local disease prediction probability based on each of the denoised spatial features, and performing temporal fusion on all the denoised spatial features to output a global disease prediction probability, includes: Each of the denoised spatial features is converted into a one-dimensional feature vector, and the one-dimensional feature vector is classified to obtain the local disease prediction probability. The one-dimensional feature vectors of each slice within the same sequence are concatenated in depth order and superimposed with positional codes to obtain a sequence feature matrix; Deep context features are extracted based on the sequence feature matrix, and the deep context features are classified to obtain the global disease prediction probability.

4. The skin CT lesion identification method based on boundary awareness and hierarchical sequence supervision according to claim 1, characterized in that, The training process of the dual-stream hierarchical network is as follows: Obtain the true labels of local diseases and the true labels of global diseases; Slice-level cross-entropy loss is calculated based on the local disease prediction probability and the local disease true label, and sequence-level cross-entropy loss is calculated based on the global disease prediction probability and the global disease true label; The two-stream hierarchical network is optimized end-to-end based on the hierarchical joint loss function composed of the weighted sum of the slice-level cross-entropy loss and the sequence-level cross-entropy loss.

5. The skin CT lesion identification method based on boundary awareness and hierarchical sequence supervision according to claim 4, characterized in that, The end-to-end optimization of the two-stream hierarchical network based on the hierarchical joint loss function, which is a weighted sum of the slice-level cross-entropy loss and the sequence-level cross-entropy loss, includes: At the start of training, the first weight corresponding to the slice-level cross-entropy loss is set to be greater than the second weight corresponding to the sequence-level cross-entropy loss. During the training iteration, a cosine annealing strategy is used to smoothly reduce the first weight and smoothly increase the second weight until the training ends and the second weight is greater than the first weight.

6. The skin CT lesion identification method based on boundary awareness and hierarchical sequence supervision according to claim 1, characterized in that, After outputting the predicted probability of the local disease, the following is also included: For each slice, the corresponding malignancy prediction probability is obtained from the local disease prediction probability; Based on the depth of each slice and the corresponding malignancy prediction probability, a disease risk change curve is plotted to assess the vertical infiltration depth of the lesion.

7. The skin CT lesion identification method based on boundary awareness and hierarchical sequence supervision according to claim 1, characterized in that, Before inputting the continuous slice sequence into the pre-trained two-stream hierarchical network, the method further includes: The continuous slice sequence is rigidly registered based on the translation vector between adjacent slices. The registered continuous slice sequence is uniformly resampled to a fixed number of layers, and contrast-limited adaptive histogram equalization and uniform scaling are performed on each slice to obtain the standardized continuous slice sequence.

8. A skin CT lesion identification device based on boundary awareness and hierarchical sequence supervision, characterized in that, include: The acquisition module is used to acquire the continuous slice sequence of the skin CT scan to be diagnosed; The extraction module is used to input the continuous slice sequence into a pre-trained two-stream hierarchical network to extract the semantic features and boundary features of each slice respectively; The mask module is used to generate gating weights based on the boundary features and to perform mask purification on the semantic features to obtain denoised spatial features; The output module is used to output a local disease prediction probability based on each of the denoised spatial features, and to perform temporal fusion on all the denoised spatial features to output a global disease prediction probability.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the skin CT lesion identification method based on boundary awareness and hierarchical sequence supervision as described in any one of claims 1-7.

10. A storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the skin CT lesion identification method based on boundary awareness and hierarchical sequence supervision as described in any one of claims 1-7.

Images