Organ focus detection method and device and electronic equipment

Abstract

The invention relates to the technical field of medical images, and discloses an organ focus detection method and device, and electronic equipment, and the method comprises the steps: obtaining a scanning image sequence of an organ, the scanning image sequence comprises a plurality of two-dimensional slices; inputting each two-dimensional slice into a slice feature encoder of the focus classification model, and obtaining an initial feature vector of each two-dimensional slice by using the slice feature encoder; performing position coding on the initial feature vector of each two-dimensional slice, inputting the initial feature vector into a self-attention module of a focus classification model, and obtaining an enhanced feature vector, fused with global context information, of each two-dimensional slice by using the self-attention module; and obtaining a focus position detection result based on the enhanced feature vector of each two-dimensional slice. According to the method, through two-stage training, slice feature coding and a self-attention mechanism are combined, the global context is fused while the space sequence is reserved, the tiny focus is effectively detected, the calculation overhead is reduced, and performance, interpretability and practicability are considered.

Description

Technical Field

[0001] This disclosure relates to the technical field of medical imaging, such as a method, apparatus, or electronic device for detecting organ lesions. Background Technology

[0002] Medical imaging is the primary means of diagnosing various space-occupying lesions (such as tumors, cysts, and abscesses). For example, magnetic resonance imaging (MRI) is widely used for screening and evaluating lesions in multiple organs, including the brain, liver, breast, and prostate, due to its excellent soft tissue contrast and high sensitivity to small lesions. However, the medical imaging diagnosis of space-occupying lesions faces many challenges. First, lesions exhibit high heterogeneity in morphology, size, location, and signal characteristics, especially early small lesions, whose imaging manifestations are often atypical and easily overlooked in complex organ backgrounds. Second, the comprehensive and accurate analysis of three-dimensional medical imaging data is time-consuming and susceptible to subjective factors, carrying the risk of missed diagnoses and misdiagnoses.

[0003] In recent years, deep convolutional neural networks (DCNNs) have made significant progress in the field of medical image analysis, providing strong support for the development of computer-aided diagnostic systems. However, the effective application of artificial intelligence technology to the detection of organ space-occupying lesions still faces the following challenges: (1) Organ lesions have complex structures and diverse sizes and locations. In particular, small tumors account for a small proportion in the overall organ image and are difficult to detect effectively. High-resolution images undergo multiple convolution and pooling operations during DCNN training, which can easily lead to the loss of information on small lesions. (2) Most current target detection algorithms rely on supervised learning and require a large number of labeled ROI (Region of Interest) regions. However, ROI labeling relies on experienced radiologists, which is time-consuming and labor-intensive, resulting in a scarcity of samples with precise lesion delineation. Usually, there are only weak labels (such as image classification labels), making it difficult to achieve lesion localization. (3) Three-dimensional medical image data is large in volume and computationally complex, requiring high hardware resources and memory usage.

[0004] To address these challenges, Multi-Instance Learning (MIL) has been introduced as a weakly supervised learning paradigm into medical image analysis. MIL treats a patient's 3D image sequence as a "bag," where each 2D slice is an "instance." Under the standard assumption, a bag is labeled positive if it contains at least one positive instance with a lesion. This method aligns with clinical logic and simplifies the 3D problem into a 2D process, effectively reducing computational overhead. However, while instance-based methods offer interpretability, training can be insufficient due to unknown instance labels. Embedding-based methods, while performing well in classification, lack interpretability and struggle to locate key slices. More importantly, mainstream methods generally employ fixed pooling operators (such as Max or Mean Pooling), neglecting spatial relationships and contextual dependencies between slices. Clinically, lesions often span multiple consecutive slices; analyzing a single slice in isolation can lead to biased judgments and affect diagnostic accuracy.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or describe the scope of protection of these embodiments, but rather as a prelude to the detailed description that follows.

[0007] This disclosure provides an organ lesion detection method, device, and electronic device that can achieve efficient and interpretable lesion localization through two-stage weakly supervised training and self-attention mechanism.

[0008] According to a first aspect of this disclosure, a method for detecting organ lesions is provided, comprising: Obtain scanned image sequences of organs, wherein the scanned image sequences include multiple two-dimensional slices; Each two-dimensional slice is input into the slice feature encoder of the lesion classification model, and the initial feature vector of each two-dimensional slice is obtained by using the slice feature encoder. After the initial feature vector of each two-dimensional slice is positionally encoded, it is input into the self-attention module of the lesion classification model. The self-attention module is used to obtain the enhanced feature vector of each two-dimensional slice that incorporates global context information. Based on the enhanced feature vector of each two-dimensional slice, the lesion location detection results are obtained; Among them, the slice feature encoder is the backbone network of the slice-level classifier. The slice-level classifier is obtained by training a first preset model based on a first sample scan image sequence with slice-level labels. The lesion classification model is obtained by training a second preset model based on a second sample scan image sequence with sequence-level labels. The second preset model includes a slice feature encoder and a self-attention module.

[0009] In some embodiments, an enhanced feature vector incorporating global contextual information is obtained for each two-dimensional slice using a self-attention module, including: By utilizing a self-attention module to calculate the correlation weights between any two 2D slices, feature information from all other 2D slices is dynamically aggregated, generating an enhanced feature vector that incorporates global contextual information for each 2D slice.

[0010] In some embodiments, the slice-level classifier is trained through the following process: Obtain the first sample scan image sequence; A first preset model is constructed based on a preset convolutional neural network, a global average pooling layer, a convolutional layer, and a softmax layer; Based on the first sample scan image sequence, the first preset model is iteratively trained until the model converges, and a slice-level classifier is obtained. The training steps include: inputting each two-dimensional slice of the first sample scan image sequence into the first preset model, calculating the lesion existence prediction probability of each two-dimensional slice, calculating the first loss value based on the lesion existence prediction probability of each two-dimensional slice and the corresponding slice-level label, and updating the parameters of the first preset model according to the first loss value.

[0011] In some embodiments, the lesion classification model is trained through the following process: Obtain the second sample scan image sequence; The backbone network of the slice-level classifier is used as the slice feature encoder. Based on the slice feature encoder, the embedding module, the self-attention module, the fully connected layer, the softmax layer, and the multi-example voting module, a second preset model is constructed. Based on the second sample scan image sequence, the second preset model is iteratively trained until the model converges to obtain the lesion classification model. The training steps include: inputting each two-dimensional slice of the second sample scan image sequence into the second preset model, determining the predicted sequence-level label and its predicted probability corresponding to the second sample scan image sequence, calculating the second loss value based on the predicted sequence-level label and its predicted probability, and updating the parameters of the second preset model according to the second loss value.

[0012] In some embodiments, each two-dimensional slice of the second sample scan image sequence is input into a second preset model to determine the predicted sequence-level label and its predicted probability corresponding to the second sample scan image sequence, including: Each two-dimensional slice of the second sample scan image sequence is input into the second preset model, and the lesion presence prediction probability of each two-dimensional slice is output through the Softmax layer; Using a multi-instance voting module, based on the maximum probability voting strategy and the predicted probability of lesion presence in each two-dimensional slice, the predicted sequence-level label and its predicted probability corresponding to the second sample scan image sequence are determined.

[0013] In some embodiments, the lesion location detection result is obtained based on the enhanced feature vector of each two-dimensional slice, including: Based on the enhanced feature vector of each two-dimensional slice, the predicted probability of lesion presence in each two-dimensional slice is calculated; Based on the predicted probability of lesion presence in two-dimensional slices, a class activation heatmap for characterizing lesion-like regions is obtained through gradient backpropagation calculation. The location of lesions is obtained based on class activation heatmap.

[0014] In some embodiments, based on the predicted probability of lesion presence in a two-dimensional slice, a class activation heatmap for characterizing lesion-like regions is calculated via gradient backpropagation, including: The two-dimensional slice with the highest predicted probability of lesion presence is selected as the target two-dimensional slice, and the category prediction score corresponding to the target two-dimensional slice is used as the loss function. Backpropagation is performed on the lesion classification model on the loss function to calculate the gradient values ​​of each channel in the output feature map of the last convolutional layer of the classifier for the target two-dimensional slice. The average gradient value of each channel is calculated in the spatial dimension to obtain the weight value of the corresponding channel; The feature maps of the last convolutional layer are weighted and summed using weight values, and then processed by the ReLU activation function to generate a class activation heatmap for characterizing the lesion region.

[0015] In some embodiments, obtaining lesion location detection results based on class activation heatmaps includes: Binarize the class activation heatmap to obtain a binary image with the same size as the target two-dimensional slice; Perform connected component analysis on the binary image, retain the connected component with the largest area, and perform morphological dilation on the largest connected component. Calculate the minimum bounding rectangle containing the expanded connected components and obtain its boundary coordinates; Based on the boundary coordinates, a lesion localization box is generated on the target two-dimensional slice.

[0016] According to a second aspect of this disclosure, an organ lesion detection device is provided, comprising: The scan image acquisition module is configured to acquire a scan image sequence of an organ, wherein the scan image sequence includes multiple two-dimensional slices; The initial feature acquisition module is configured to: input each two-dimensional slice into the slice feature encoder of the lesion classification model, and use the slice feature encoder to obtain the initial feature vector of each two-dimensional slice; The enhanced feature acquisition module is configured to: encode the initial feature vector of each two-dimensional slice at its position and input it into the self-attention module of the lesion classification model, and use the self-attention module to obtain the enhanced feature vector of each two-dimensional slice that incorporates global context information; The lesion location detection module is configured to obtain lesion location detection results based on the enhanced feature vector of each two-dimensional slice; Among them, the slice feature encoder is the backbone network of the slice-level classifier. The slice-level classifier is obtained by training a first preset model based on a first sample scan image sequence with slice-level labels. The lesion classification model is obtained by training a second preset model based on a second sample scan image sequence with sequence-level labels. The second preset model includes a slice feature encoder and a self-attention module.

[0017] According to a third aspect of this disclosure, an electronic device is provided, including a processor and a memory storing program instructions, the processor being configured to execute the organ lesion detection method provided in the first aspect of this disclosure when the program instructions are executed.

[0018] The organ lesion detection method, apparatus, and electronic device provided in this disclosure can achieve the following technical effects: The organ lesion detection method provided in this disclosure first uses a slice feature encoder to extract initial features from each two-dimensional slice to preserve local details. Then, by introducing position encoding and a self-attention module, the initial feature vectors of all two-dimensional slices are globally modeled, so that the enhanced feature vector of each two-dimensional slice is fused with contextual information from neighboring and semantically related slices, thereby strengthening weak lesion signals and alleviating missed detections caused by insignificant local features. In addition, when annotation resources are limited, this method adopts a two-stage training strategy: first, a slice-level classifier is pre-trained on a small number of first sample scan image sequences with slice-level labels to obtain an effective slice feature encoder; then, the slice feature encoder is transferred to a second sample scan image sequence with only sequence-level labels, and end-to-end training is performed in conjunction with the self-attention module to obtain a lesion classification model that can implicitly identify key two-dimensional slices that contribute significantly to classification, thereby realizing lesion location detection. Meanwhile, the three-dimensional scan image sequence is decomposed into multiple two-dimensional slices for processing, which significantly reduces computational complexity and memory usage. The self-attention module replaces the traditional fixed pooling operation, which not only preserves the spatial order information between two-dimensional slices through position encoding, but also dynamically models the dependency relationship between any two two-dimensional slices, which is more in line with the actual situation of lesion distribution in continuous two-dimensional slices, and takes into account detection performance, interpretability and practicality.

[0019] The above general description and the description below are exemplary and illustrative only and are not intended to limit this disclosure. Attached Figure Description

[0020] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations and drawings do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are shown as similar elements. The drawings are not to be scaled. And wherein: Figure 1 This is a schematic flowchart of an organ lesion detection method provided in an embodiment of this disclosure; Figure 2 This is a schematic flowchart of another organ lesion detection method provided in this embodiment of the present disclosure; Figure 3 This is a schematic diagram of the structure of the slice-level classifier and lesion classification model provided in the embodiments of this disclosure during the model training process; Figure 4 This is a schematic diagram of the self-attention mechanism framework of the lesion classification model provided in this embodiment of the disclosure; Figure 5 This is a schematic diagram of an organ lesion detection device provided in an embodiment of this disclosure; Figure 6 This is a schematic diagram of another organ lesion detection device provided in an embodiment of this disclosure; Figure 7This is a schematic diagram of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0021] To provide a more detailed understanding of the features and technical content of the embodiments of this disclosure, the implementation of the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for illustrative purposes only and are not intended to limit the embodiments of this disclosure. In the following technical description, for ease of explanation, several details are used to provide a full understanding of the disclosed embodiments. However, one or more embodiments may still be implemented without these details. In other cases, well-known structures and devices may be simplified in their depiction to simplify the drawings.

[0022] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0023] Unless otherwise stated, the term "multiple" means two or more.

[0024] In this embodiment of the disclosure, the character " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B means: A or B.

[0025] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0026] The term "correspondence" can refer to an association or binding relationship. The correspondence between A and B means that there is an association or binding relationship between A and B.

[0027] This disclosure provides an electronic device, which can be a computer, terminal, server, or other device with computing capabilities. The electronic device is equipped with a lesion classification model and can implement the organ lesion detection method provided in this disclosure based on the lesion classification model. Specifically, the electronic device can input the scanned image sequence of an organ into the lesion classification model and use the lesion classification model to obtain the lesion location detection result.

[0028] The organ lesion detection method provided in this disclosure is applicable to various organs that may develop space-occupying lesions, including but not limited to the brain, liver, breast, prostate, lung, kidney, pancreas, and thyroid. Space-occupying lesions refer to pathological changes that abnormally occupy space within normal tissue structures, with typical types including tumors, cysts, abscesses, hematomas, hemangiomas, and other focal abnormal proliferations or accumulations.

[0029] Organ imaging sequences are three-dimensional data acquired through tomographic scanning of a target organ using medical scanning equipment, typically consisting of a series of consecutive two-dimensional slices. Medical scanning equipment that can be used to acquire these imaging sequences includes: Magnetic Resonance Imaging (MRI), Computed Tomography (CT), Ultrasound, Positron Emission Tomography (PET), and Single-Photon Emission Computed Tomography (SPECT). Different imaging modalities have their own advantages in terms of soft tissue contrast, spatial resolution, and functional information provision, and the appropriate scanning mode can be selected according to specific clinical needs and lesion types. This method is compatible with imaging sequences generated by the aforementioned equipment, exhibiting good versatility and clinical applicability.

[0030] In this embodiment, the lesion classification model is obtained by training a second preset model based on a second sample scan image sequence with sequence-level labels. The second preset model includes a slice feature encoder and a self-attention module. The slice feature encoder is the backbone network of the slice-level classifier, which is obtained by training a first preset model based on a first sample scan image sequence with slice-level labels.

[0031] In conjunction with the electronic device (hereinafter referred to as the device) provided in the embodiments of this disclosure, the embodiments of this disclosure provide a method for detecting organ lesions, such as... Figure 1 As shown, methods for detecting organ lesions include: S101, The device acquires scanned image sequences of organs.

[0032] In this embodiment of the disclosure, a three-dimensional tomographic scan of an organ can be performed using a medical scanning device to obtain a scanned image sequence containing multiple consecutive two-dimensional slices. The scanned image sequence reflects the spatial structural information of the organ, with each slice corresponding to a specific anatomical level.

[0033] S102, the device inputs each two-dimensional slice into the slice feature encoder of the lesion classification model, and uses the slice feature encoder to obtain the initial feature vector of each two-dimensional slice.

[0034] The slice feature encoder can automatically extract multi-level spatial semantic features from a single two-dimensional slice. Each two-dimensional slice is mapped to a high-dimensional initial feature vector after passing through the slice feature encoder. The initial feature vector retains key information related to the lesion, such as local texture, edges, and shape.

[0035] S103, the device encodes the initial feature vector of each two-dimensional slice at its position and inputs it into the self-attention module of the lesion classification model. The self-attention module is used to obtain the enhanced feature vector of each two-dimensional slice that incorporates global contextual information.

[0036] Positional encoding is used to explicitly incorporate the order or spatial location information of the slices in the original scan sequence, avoiding the loss of anatomical structural information due to the insensitivity of the self-attention mechanism to the input order. Subsequently, the two-dimensional slices are input into the self-attention module for processing, fusing information from different two-dimensional slices to obtain an enhanced feature vector for each two-dimensional slice that incorporates global contextual information.

[0037] S104, the device obtains the lesion location detection results based on the enhanced feature vector of each two-dimensional slice.

[0038] The organ lesion detection method provided in this disclosure first uses a slice feature encoder to extract initial features from each two-dimensional slice to preserve local details. Then, by introducing position encoding and a self-attention module, the initial feature vectors of all two-dimensional slices are globally modeled, so that the enhanced feature vector of each two-dimensional slice is fused with contextual information from neighboring and semantically related slices, thereby strengthening weak lesion signals and alleviating missed detections caused by insignificant local features. In addition, when annotation resources are limited, this method adopts a two-stage training strategy: first, a slice-level classifier is pre-trained on a small number of first sample scan image sequences with slice-level labels to obtain an effective slice feature encoder; then, the slice feature encoder is transferred to a second sample scan image sequence with only sequence-level labels, and end-to-end training is performed in conjunction with the self-attention module to obtain a lesion classification model that can implicitly identify key two-dimensional slices that contribute significantly to classification, thereby realizing lesion location detection. Meanwhile, the three-dimensional scan image sequence is decomposed into multiple two-dimensional slices for processing, which significantly reduces computational complexity and memory usage. The self-attention module replaces the traditional fixed pooling operation, which not only preserves the spatial order information between two-dimensional slices through position encoding, but also dynamically models the dependency relationship between any two two-dimensional slices, which is more in line with the actual situation of lesion distribution in continuous two-dimensional slices, and takes into account detection performance, interpretability and practicality.

[0039] In some embodiments, obtaining an enhanced feature vector that incorporates global context information for each two-dimensional slice using a self-attention module includes: dynamically aggregating feature information from all other two-dimensional slices by calculating the correlation weights between any two two-dimensional slices using the self-attention module, and generating an enhanced feature vector that incorporates global context information for each two-dimensional slice.

[0040] The initial feature vectors of each two-dimensional slice after position encoding are used as the input sequence and fed into the self-attention module. The self-attention module measures the contribution of each slice to the semantic or pathological association of other slices by calculating the correlation weight between any two two-dimensional slices. Subsequently, based on these dynamically generated weights, the features of all slices are weighted and aggregated to generate an enhanced feature vector that integrates full sequence context information for each two-dimensional slice.

[0041] By designing the above self-attention mechanism, even if the lesion signal in a two-dimensional slice is weak or its morphology is atypical, if it has a strong correlation with other lesion slices that are adjacent or semantically related, the self-attention mechanism can still enhance its features through high attention weights, effectively alleviating the problem of missed detection caused by insufficient local features. The self-attention mechanism can directly establish the dependency relationship between any two slices, and is especially suitable for complex situations where lesions span multiple non-contiguous or distant slices, which is more in line with clinical practice. In addition, under the condition of only sequence-level labels of the scanned image sequence (without slice-level annotation), the self-attention weight can serve as an indicator of the importance of lesion-related slices, implicitly guiding the model to focus on the area that truly contains the lesion, thereby achieving relatively accurate lesion localization without relying on fine annotation. In some embodiments, obtaining lesion location detection results based on the enhanced feature vector of each two-dimensional slice includes: calculating the predicted probability of lesion presence in each two-dimensional slice based on the enhanced feature vector of each two-dimensional slice; calculating a class activation heatmap to characterize the lesion-like region through gradient backpropagation based on the predicted probability of lesion presence in the two-dimensional slice; and obtaining lesion location detection results based on the class activation heatmap. Here, the class activation heatmap can reflect the approximate shape, extent, and intensity distribution of lesions within the slice, providing an intuitive basis for subsequent diagnosis.

[0042] Combination Figure 2 As shown in the embodiments of this disclosure, another method for detecting organ lesions is provided, which includes: S201, The device acquires scanned image sequences of organs.

[0043] S202, the device inputs each two-dimensional slice into the slice feature encoder of the lesion classification model, and uses the slice feature encoder to obtain the initial feature vector of each two-dimensional slice.

[0044] S203, the device encodes the initial feature vector of each two-dimensional slice at its position and inputs it into the self-attention module of the lesion classification model. The self-attention module is used to obtain the enhanced feature vector of each two-dimensional slice that incorporates global contextual information.

[0045] S204, the device calculates the predicted probability of lesion presence in each two-dimensional slice based on the enhanced feature vector of each slice.

[0046] S205, the device predicts the probability of lesion presence based on two-dimensional slices and calculates the class activation heatmap to characterize the lesion-like region through gradient backpropagation.

[0047] S206, the device obtains lesion location detection results based on class activation heatmap.

[0048] In some embodiments, a class activation heatmap for characterizing lesion regions is obtained by backpropagation of gradients based on the predicted probability of lesion presence in a two-dimensional slice. This includes: selecting the two-dimensional slice with the highest predicted probability of lesion presence as the target two-dimensional slice, and using the class prediction score corresponding to the target two-dimensional slice as the loss function; performing backpropagation on the lesion classification model on the loss function to calculate the gradient values ​​of each channel in the output feature map of the last convolutional layer of the classifier for the target two-dimensional slice; averaging the gradient values ​​of each channel in the spatial dimension to obtain the weight values ​​of the corresponding channels; using the weight values ​​to perform a weighted summation on the feature map of the last convolutional layer, and processing it through the ReLU activation function to generate a class activation heatmap for characterizing lesion regions.

[0049] In this embodiment of the disclosure, the class activation heatmap can be calculated using the following formulas 1 and 2: ---Formula 1; ---Formula 2.

[0050] Formula 1 is used to calculate the first... Each feature channel in the predicted category Importance weights are used to predict the category. Taking "having lesions" as an example, this refers to the degree of dependence of the lesion classification model on different feature channels when predicting "having lesions." Specifically, Indicates the first Each feature channel in category The weight (importance score) on the channel indicates the importance of the predicted category. The degree of contribution; This represents the output category prediction score (the raw score before activation by the Softmax layer). This indicates that the feature map output by the last convolutional layer is in the channel. Middle, position ( , The pixel value at () Represents the category prediction score The pixels in the feature map output by the last convolutional layer ( , ) pixel values The gradient represents the degree of influence of that pixel on the prediction result; This represents the total number of pixels in the feature map output by the last convolutional layer.

[0051] Formula 2 is used to generate class activation heatmaps. Specifically, Representing the class activation heatmap, Formula 2 assigns each feature channel Multiply by its corresponding weight Then, a weighted summation is performed, and finally, the ReLU function is used for activation to generate the final class activation heatmap.

[0052] In some embodiments, obtaining lesion location detection results based on class activation heatmaps includes: binarizing the class activation heatmap to obtain a binary image with the same size as the target two-dimensional slice; performing connected component analysis on the binary image, retaining the connected component with the largest area and performing morphological dilation on the largest connected component; calculating the minimum bounding rectangle containing the dilated connected component and obtaining its boundary coordinates; and generating a lesion location box on the target two-dimensional slice based on the boundary coordinates.

[0053] In the embodiments disclosed herein, such as Figure 3 As shown, the training process of the lesion classification model is divided into two stages. In stage 1, a first pre-defined model is constructed based on a pre-defined convolutional neural network and other related modules. This first pre-defined model can be considered as a slice-level classifier. The first pre-defined model is trained to enable the slice-level classifier to learn local features. In stage 2, the backbone network of the slice-level classifier is used as a slice feature encoder and, together with other related modules, a second pre-defined model is constructed. This second pre-defined model can be considered as a sequence-level classifier. The second pre-defined model is trained to enable it to learn global features, ultimately obtaining the lesion classification model.

[0054] In some embodiments, the slice-level classifier is trained through the following process: acquiring a first sample scan image sequence; constructing a first preset model based on a preset convolutional neural network, a global average pooling layer, a convolutional layer, and a softmax layer; iteratively performing training steps on the first preset model based on the first sample scan image sequence until the model converges, thereby obtaining the slice-level classifier. The training steps of the slice-level classifier include: inputting each two-dimensional slice of the first sample scan image sequence into the first preset model, calculating the lesion existence prediction probability of each two-dimensional slice, calculating the first loss value based on the lesion existence prediction probability of each two-dimensional slice and the corresponding slice-level label, and updating the parameters of the first preset model according to the first loss value.

[0055] In this embodiment, slice-level labels are used to indicate whether a corresponding two-dimensional slice has lesions. The types of slice-level labels include "with lesions" and "without lesions". Specifically, two-dimensional slices in the first sample scan image sequence with the slice-level label "with lesions" are positive samples, and two-dimensional slices in the first sample scan image sequence with the slice-level label "without lesions" are negative samples. Here, a cross-entropy loss function with a balance factor can be used to calculate the first loss value. Specifically, the first loss value can be calculated using the following formula 3: ---Formula 3.

[0056] In formula 3, This is the first loss value. It is the number of positive samples in the first sample scan image sequence. is the number of negative samples in the first sample scan image sequence, and N is the total number of samples in the first sample scan image sequence, i.e. . / , / ; This is the predicted probability of lesion presence in the two-dimensional slice output by the first preset model. The total loss sum for both positive and negative samples in the first sample scan image sequence is calculated separately. If the number of positive samples... Less than the number of negative samples , It will be greater than Therefore, the weight of positive samples in the total loss generated by a batch will be increased. Here, the smaller the positive-to-negative ratio J:K, the greater the weight assigned to positive samples, which to some extent balances the problem of the imbalance in attention given to positive and negative samples.

[0057] In this embodiment, the slice-level classifier uses a classic convolutional neural network as its backbone network, with optional structures including ResNet-50, VGGNet, and GoogLeNet. Combined with... Figure 3As shown, the backbone network consists of the following components: a convolutional layer (Conv7×7, s=2), a batch normalization layer, an activation function (usually ReLU), and a 3×3 max pooling layer (Maxpooling 3×3, s=2). These are followed by stacked blocks of two standard convolutional layers (Conv block ×2), three identity blocks (Identity block ×3), two standard convolutional layers (Conv block ×2), three identity blocks (Identity block ×3), five convolutional layers (Conv block ×5), and two identity blocks (Identity block ×2). These modules together form a deep residual structure used to extract discriminative features from a single 2D slice layer by layer, from low to high levels.

[0058] Following the backbone network are the top layers, including a global average pooling (GA) layer, a 1×1 convolutional layer (Conv 1×1), and a softmax layer. The GA layer compresses the spatial dimension into channel vectors, the 1×1 convolutional layer further adjusts the feature dimensions or performs lightweight feature fusion, and finally, the softmax layer outputs the probability distribution of whether the 2D slice belongs to "containing lesions" or "not containing lesions," i.e., the slice-predicted label. This overall architecture not only supports effective pre-training using slice-labeled data in the first stage but also provides a high-quality feature encoding foundation for subsequent weakly supervised sequence-level lesion detection.

[0059] In some embodiments, the lesion classification model is trained through the following process: acquiring a second sample scan image sequence; using the backbone network of the slice-level classifier as a slice feature encoder, which serves as a local feature encoder; constructing a second preset model based on the slice feature encoder, embedding module, self-attention module, fully connected layer (FC), softmax layer, and multi-instance voting module (MIL Vote Module); iteratively performing training steps on the second preset model based on the second sample scan image sequence until the model converges, thus obtaining the lesion classification model. Here, the self-attention module can serve as a global feature encoder.

[0060] The training steps of the second preset model include: inputting each two-dimensional slice of the second sample scan image sequence into the second preset model, determining the predicted sequence-level label and its predicted probability corresponding to the second sample scan image sequence, calculating the second loss value based on the predicted sequence-level label and its predicted probability, and updating the parameters of the second preset model according to the second loss value.

[0061] In some embodiments, inputting each two-dimensional slice of the second sample scan image sequence into a second preset model to determine the predicted sequence-level label and its predicted probability corresponding to the second sample scan image sequence includes: inputting each two-dimensional slice of the second sample scan image sequence into the second preset model, outputting the lesion presence prediction probability of each two-dimensional slice through a Softmax layer; and using a multi-instance voting module, determining the predicted sequence-level label and its predicted probability corresponding to the second sample scan image sequence based on the maximum probability voting strategy and the lesion presence prediction probability of each two-dimensional slice.

[0062] In this embodiment of the disclosure, each two-dimensional slice of the second sample scan image sequence is input to the slice feature encoder of the second preset model. The slice feature encoder obtains the initial feature vector of each two-dimensional slice. The embedding module performs position encoding on the initial feature vector of each two-dimensional slice to obtain the intermediate feature vector (e.g., ...). Figure 3 The algorithm iterates through slice-1 to slice-n vectors, then inputs the intermediate feature vectors into a self-attention module. This module generates an enhanced feature vector for each 2D slice, incorporating global contextual information. Each enhanced feature vector is then sequentially input into a fully connected layer and a softmax layer. The softmax layer outputs the predicted probability of lesion presence for each 2D slice. A multi-instance voting module, based on a maximum probability-based (MP-MIL) voting strategy and the predicted probability of lesion presence in each 2D slice, determines the predicted sequence-level label and its predicted probability for the second sample scan image sequence.

[0063] like Figure 4 As shown, the lesion classification model employs a self-attention mechanism, where the self-attention module includes a Transformer Encoder. The Transformer Encoder comprises a Multi-head Self-Attention (MSA) unit, a Multiple Linear Perception (MLP) unit, and a Normalization (Norm) unit.

[0064] Each 2D slice of the second sample scan image sequence is input to the Slice Encoder, which obtains an initial feature vector for each 2D slice. The Embedding module performs positional encoding on the initial feature vectors of each 2D slice to obtain intermediate feature vectors (Embedded Slices), which are then input to the Transformer Encoder. The Transformer Encoder learns the weighted relationships between slices through multi-head self-attention units and multi-layer perceptrons and outputs an enhanced feature vector for each 2D slice that incorporates global context information. The enhanced feature vectors of each 2D slice are then processed by a global average pooling (GA) layer and a fully connected (FC) layer before being input to the Softmax layer. The Softmax layer outputs the predicted probability of lesion presence for each 2D slice. The multi-instance voting module determines the predicted sequence-level label and its predicted probability corresponding to the second sample scan image sequence based on the maximum probability voting strategy and the predicted probability of lesion presence in each two-dimensional slice.

[0065] In this embodiment, the predicted sequence-level label types include "with lesions" and "without lesions," where "with lesions" is a positive label and "without lesions" is a negative label. The predicted sequence-level label is used to indicate whether the organ to which the second sample scan image sequence belongs has lesions. In the maximum probability voting strategy, if even one two-dimensional slice in the second sample scan image sequence is predicted to be a positive sample, the sequence-level label of the second sample scan image sequence is a positive label; conversely, if all two-dimensional slices in the second sample scan image sequence are predicted to be negative samples, the sequence-level label of the second sample scan image sequence is a negative label. That is, the probability of the sequence-level label is determined by the slice with the highest probability.

[0066] In this embodiment of the disclosure, the maximum probability voting strategy can be represented by the following formula 4: ---Formula 4.

[0067] In formula 4, where It is the probability that the sequence-level label of the second sample scan image sequence is a positive label. It is the predicted number The lesions in each two-dimensional slice have a predictive probability. It is the index value of the two-dimensional slice in the second sample scan image sequence. It is the number of two-dimensional slices contained in the second sample scan image sequence.

[0068] In this embodiment of the disclosure, a cross-entropy loss function with a balance factor can be used to calculate the second loss value. Specifically, the second loss value can be calculated using the following formula 5: ---Formula 5.

[0069] In Formula 5, It is the second loss value. It is the number of positive samples in the second sample scan image sequence. is the number of negative samples in the second sample scan image sequence, and N is the total number of samples in the second sample scan image sequence, i.e. . / , / . It is the predicted probability that the sequence-level label is a positive label. It is the predicted probability that the sequence-level label is a negative label.

[0070] Combination Figure 5 As shown, this embodiment of the present disclosure provides an organ lesion detection device 500, which includes a scan image acquisition module 501, an initial feature acquisition module 502, an enhanced feature acquisition module 503, and a lesion location detection module 504.

[0071] The scan image acquisition module 501 is configured to acquire a scan image sequence of an organ, wherein the scan image sequence includes multiple two-dimensional slices.

[0072] The initial feature acquisition module 502 is configured to input each two-dimensional slice into the slice feature encoder of the lesion classification model, and use the slice feature encoder to obtain the initial feature vector of each two-dimensional slice.

[0073] The enhanced feature acquisition module 503 is configured to: encode the initial feature vector of each two-dimensional slice at its position and input it into the self-attention module of the lesion classification model, and use the self-attention module to obtain the enhanced feature vector of each two-dimensional slice that incorporates global context information.

[0074] The lesion location detection module 504 is configured to obtain the lesion location detection result based on the enhanced feature vector of each two-dimensional slice.

[0075] The organ lesion detection device provided in this disclosure first uses a slice feature encoder to extract initial features from each two-dimensional slice to preserve local details. Then, by introducing position encoding and a self-attention module, it globally models the initial feature vectors of all two-dimensional slices, enabling the enhanced feature vectors of each two-dimensional slice to fuse contextual information from neighboring and semantically related slices, thereby strengthening weak lesion signals and mitigating missed detections caused by insignificant local features. Furthermore, when annotation resources are limited, this method employs a two-stage training strategy: first, a slice-level classifier is pre-trained on a small number of first-sample scan image sequences with slice-level labels to obtain an effective slice feature encoder; then, this slice feature encoder is transferred to a second-sample scan image sequence with only sequence-level labels, and end-to-end training is performed using a self-attention module to obtain a lesion classification model capable of implicitly identifying key two-dimensional slices that contribute significantly to classification, thus achieving lesion location detection. Meanwhile, the three-dimensional scan image sequence is decomposed into multiple two-dimensional slices for processing, which significantly reduces computational complexity and memory usage. The self-attention module replaces the traditional fixed pooling operation, which not only preserves the spatial order information between two-dimensional slices through position encoding, but also dynamically models the dependency relationship between any two two-dimensional slices, which is more in line with the actual situation of lesion distribution in continuous two-dimensional slices, and takes into account detection performance, interpretability and practicality.

[0076] The slice feature encoder is the backbone network of the slice-level classifier. The slice-level classifier is obtained by training a first preset model based on a first sample scan image sequence with slice-level labels. The lesion classification model is obtained by training a second preset model based on a second sample scan image sequence with sequence-level labels. The second preset model includes a slice feature encoder and a self-attention module.

[0077] In some embodiments, the enhanced feature acquisition module 503 is configured to: use a self-attention module to dynamically aggregate feature information from all other two-dimensional slices by calculating the correlation weight between any two two-dimensional slices, and generate an enhanced feature vector that incorporates global context information for each two-dimensional slice.

[0078] In some embodiments, the lesion location detection module 504 is configured to: Based on the enhanced feature vector of each two-dimensional slice, the predicted probability of lesion presence in each two-dimensional slice is calculated; Based on the predicted probability of lesion presence in two-dimensional slices, a class activation heatmap for characterizing lesion-like regions is obtained through gradient backpropagation calculation. The location of lesions is obtained based on class activation heatmap.

[0079] In some embodiments, the lesion location detection module 504 is configured to: The two-dimensional slice with the highest predicted probability of lesion presence is selected as the target two-dimensional slice, and the category prediction score corresponding to the target two-dimensional slice is used as the loss function. Backpropagation is performed on the lesion classification model on the loss function to calculate the gradient values ​​of each channel in the output feature map of the last convolutional layer of the classifier for the target two-dimensional slice. The average gradient value of each channel is calculated in the spatial dimension to obtain the weight value of the corresponding channel; The feature maps of the last convolutional layer are weighted and summed using weight values, and then processed by the ReLU activation function to generate a class activation heatmap for characterizing the lesion region.

[0080] In some embodiments, the lesion location detection module 504 is configured to: Binarize the class activation heatmap to obtain a binary image with the same size as the target two-dimensional slice; Perform connected component analysis on the binary image, retain the connected component with the largest area, and perform morphological dilation on the largest connected component. Calculate the minimum bounding rectangle containing the expanded connected components and obtain its boundary coordinates; Based on the boundary coordinates, a lesion localization box is generated on the target two-dimensional slice.

[0081] Combination Figure 6 As shown, this embodiment of the present disclosure provides another organ lesion detection device 500. The organ lesion detection device 500 includes a scan image acquisition module 501, an initial feature acquisition module 502, an enhanced feature acquisition module 503, and a lesion location detection module 504, and also includes a model training module 505.

[0082] In some embodiments, the model training module 505 is used to perform the training process of a slice-level classifier, wherein the slice-level classifier is trained through the following process: acquiring a first sample scan image sequence; constructing a first preset model based on a preset convolutional neural network, a global average pooling layer, a convolutional layer, and a softmax layer; iteratively performing training steps on the first preset model based on the first sample scan image sequence until the model converges, thereby obtaining the slice-level classifier. The above training steps include: inputting each two-dimensional slice of the first sample scan image sequence into the first preset model, calculating the predicted probability of lesion presence in each two-dimensional slice, calculating a first loss value based on the predicted probability of lesion presence in each two-dimensional slice and the corresponding slice-level label, and updating the parameters of the first preset model according to the first loss value.

[0083] In some embodiments, the model training module 505 is used to perform the training process of the lesion classification model, wherein the lesion classification model is trained through the following process: acquiring a second sample scan image sequence; using the backbone network of the slice-level classifier as a slice feature encoder, and constructing a second preset model based on the slice feature encoder, an embedding module, a self-attention module, a fully connected layer, a softmax layer, and a multi-example voting module; iteratively performing training steps on the second preset model based on the second sample scan image sequence until the model converges, thereby obtaining the lesion classification model. The above training steps include: inputting each two-dimensional slice of the second sample scan image sequence into the second preset model, determining the predicted sequence-level label and its predicted probability corresponding to the second sample scan image sequence, calculating a second loss value based on the predicted sequence-level label and its predicted probability, and updating the parameters of the second preset model according to the second loss value.

[0084] In some embodiments, each two-dimensional slice of the second sample scan image sequence is input into a second preset model to determine the predicted sequence-level label and its predicted probability corresponding to the second sample scan image sequence, including: Each two-dimensional slice of the second sample scan image sequence is input into the second preset model, and the lesion presence prediction probability of each two-dimensional slice is output through the Softmax layer; Using a multi-instance voting module, based on the maximum probability voting strategy and the predicted probability of lesion presence in each two-dimensional slice, the predicted sequence-level label and its predicted probability corresponding to the second sample scan image sequence are determined.

[0085] Combination Figure 7 As shown, this embodiment of the disclosure provides an electronic device 700, which includes a processor 701 and a memory 702. Optionally, the electronic device 700 may further include a communication interface 703 and a bus 704. The processor 701, communication interface 703, and memory 702 can communicate with each other via the bus 704. The communication interface 703 can be used for information transmission. The processor 701 can call logical instructions in the memory 702 to execute the organ lesion detection method of the above embodiment.

[0086] Furthermore, the logic instructions in the aforementioned memory 702 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0087] The memory 702, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this disclosure. The processor 701 executes functional applications and data processing by running the program instructions / modules stored in the memory 702, thereby implementing the organ lesion detection method in the above embodiments.

[0088] The memory 702 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 702 may include high-speed random access memory and may also include non-volatile memory.

[0089] This disclosure provides a computer-readable storage medium storing computer-executable instructions configured to perform the above-described organ lesion detection method.

[0090] The technical solutions of this disclosure can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in this disclosure. The aforementioned storage medium can be a non-transitory storage medium, such as a USB flash drive, external hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc., and other media capable of storing program code.

[0091] The foregoing description and accompanying drawings fully illustrate embodiments of this disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the terminology used in this application is for describing embodiments only and is not intended to limit the claims. As used in the description of embodiments and claims, the singular forms “a,” “an,” and “the” are intended to equally include the plural forms unless the context clearly indicates otherwise. Similarly, the term “and / or” as used in this application means including one or more of the associated listed items and all possible combinations thereof. Additionally, when used in this application, the term "comprise" and its variations "comprises" and / or "comprising" refer to the presence of stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Without further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, or apparatus that includes said element. In this document, each embodiment may focus on the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, the relevant parts can be referred to the description of the method section.

[0092] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this disclosure. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0093] The methods and products disclosed in the embodiments herein (including but not limited to devices and equipment) can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units may be merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to implement this embodiment according to actual needs. In addition, the functional units in the embodiments of this disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0094] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

Claims

1. An organ lesion detection method characterized by comprising: include: Obtain scanned image sequences of organs, wherein the scanned image sequences include multiple two-dimensional slices; Each two-dimensional slice is input into the slice feature encoder of the lesion classification model, and the initial feature vector of each two-dimensional slice is obtained by using the slice feature encoder. After the initial feature vector of each two-dimensional slice is positionally encoded, it is input into the self-attention module of the lesion classification model. The self-attention module is used to obtain the enhanced feature vector of each two-dimensional slice that incorporates global context information. Based on the enhanced feature vector of each two-dimensional slice, the lesion location detection results are obtained; Among them, the slice feature encoder is the backbone network of the slice-level classifier. The slice-level classifier is obtained by training a first preset model based on a first sample scan image sequence with slice-level labels. The lesion classification model is obtained by training a second preset model based on a second sample scan image sequence with sequence-level labels. The second preset model includes a slice feature encoder and a self-attention module.

2. The method of claim 1, wherein, The self-attention module is used to obtain an enhanced feature vector for each 2D slice, which incorporates global contextual information, including: By utilizing a self-attention module to calculate the correlation weights between any two 2D slices, feature information from all other 2D slices is dynamically aggregated, generating an enhanced feature vector that incorporates global contextual information for each 2D slice.

3. The method of claim 1, wherein, The slice-level classifier is trained through the following process: Obtain the first sample scan image sequence; A first preset model is constructed based on a preset convolutional neural network, a global average pooling layer, a convolutional layer, and a softmax layer; Based on the first sample scan image sequence, the first preset model is iteratively trained until the model converges, and a slice-level classifier is obtained. The training steps include: inputting each two-dimensional slice of the first sample scan image sequence into the first preset model, calculating the lesion existence prediction probability of each two-dimensional slice, calculating the first loss value based on the lesion existence prediction probability of each two-dimensional slice and the corresponding slice-level label, and updating the parameters of the first preset model according to the first loss value.

4. The method of claim 1, wherein, The lesion classification model is trained through the following process: Obtain the second sample scan image sequence; The backbone network of the slice-level classifier is used as the slice feature encoder. Based on the slice feature encoder, the embedding module, the self-attention module, the fully connected layer, the softmax layer, and the multi-example voting module, a second preset model is constructed. Based on the second sample scan image sequence, the second preset model is iteratively trained until the model converges to obtain the lesion classification model. The training steps include: inputting each two-dimensional slice of the second sample scan image sequence into the second preset model, determining the predicted sequence-level label and its predicted probability corresponding to the second sample scan image sequence, calculating the second loss value based on the predicted sequence-level label and its predicted probability, and updating the parameters of the second preset model according to the second loss value.

5. The method of claim 4, wherein, Each two-dimensional slice of the second sample scan image sequence is input into the second preset model to determine the predicted sequence-level label and its predicted probability corresponding to the second sample scan image sequence, including: Each two-dimensional slice of the second sample scan image sequence is input into the second preset model, and the lesion presence prediction probability of each two-dimensional slice is output through the Softmax layer; Using a multi-instance voting module, based on the maximum probability voting strategy and the predicted probability of lesion presence in each two-dimensional slice, the predicted sequence-level label and its predicted probability corresponding to the second sample scan image sequence are determined.

6. The method of claim 1, wherein, Based on the enhanced feature vector of each two-dimensional slice, the lesion location detection results are obtained, including: Based on the enhanced feature vector of each two-dimensional slice, the predicted probability of lesion presence in each two-dimensional slice is calculated; Based on the predicted probability of lesion presence in two-dimensional slices, a class activation heatmap for characterizing lesion-like regions is obtained through gradient backpropagation calculation. The location of lesions is obtained based on class activation heatmap.

7. The method of claim 6, wherein, Based on the predicted probability of lesion presence in two-dimensional slices, a class activation heatmap for characterizing lesion-like regions is obtained through gradient backpropagation, including: The two-dimensional slice with the highest predicted probability of lesion presence is selected as the target two-dimensional slice, and the category prediction score corresponding to the target two-dimensional slice is used as the loss function. Backpropagation is performed on the lesion classification model on the loss function to calculate the gradient values ​​of each channel in the output feature map of the last convolutional layer of the classifier for the target two-dimensional slice. The average gradient value of each channel is calculated in the spatial dimension to obtain the weight value of the corresponding channel; The feature maps of the last convolutional layer are weighted and summed using weight values, and then processed by the ReLU activation function to generate a class activation heatmap for characterizing the lesion region.

8. The method according to claim 6, characterized in that, Lesion location detection results are obtained based on class activation heatmaps, including: Binarize the class activation heatmap to obtain a binary image with the same size as the target two-dimensional slice; Perform connected component analysis on the binary image, retain the connected component with the largest area, and perform morphological dilation on the largest connected component. Calculate the minimum bounding rectangle containing the expanded connected components and obtain its boundary coordinates; Based on the boundary coordinates, a lesion localization box is generated on the target two-dimensional slice.

9. An organ lesion detection apparatus characterized by comprising: include: The scan image acquisition module is configured to acquire a scan image sequence of an organ, wherein the scan image sequence includes multiple two-dimensional slices; The initial feature acquisition module is configured to: input each two-dimensional slice into the slice feature encoder of the lesion classification model, and use the slice feature encoder to obtain the initial feature vector of each two-dimensional slice; The enhanced feature acquisition module is configured to: encode the initial feature vector of each two-dimensional slice at its position and input it into the self-attention module of the lesion classification model, and use the self-attention module to obtain the enhanced feature vector of each two-dimensional slice that incorporates global context information; The lesion location detection module is configured to obtain lesion location detection results based on the enhanced feature vector of each two-dimensional slice; Among them, the slice feature encoder is the backbone network of the slice-level classifier. The slice-level classifier is obtained by training a first preset model based on a first sample scan image sequence with slice-level labels. The lesion classification model is obtained by training a second preset model based on a second sample scan image sequence with sequence-level labels. The second preset model includes a slice feature encoder and a self-attention module.

10. An electronic device comprising a processor and a memory having stored thereon program instructions, wherein, The processor is configured to execute the organ lesion detection method as described in any one of claims 1 to 8 when running the program instructions.

Images