Prostate ultrasound image analysis method and device, electronic equipment, storage medium and program product

Abstract

The invention provides a prostate ultrasound image analysis method and device, electronic equipment, a storage medium and a program product, and relates to the technical field of computers. According to the method, the consistency of feature representation among different tasks is guaranteed through the unified multi-scale features extracted by the shared backbone network, and the model efficiency is improved; a prostate segmentation result is used as space priori knowledge through a trans-attention module, a focus detection process is dynamically guided, so that detection attention is focused in a prostate area, interference and false detection of adjacent tissues such as a bladder are remarkably inhibited, meanwhile, the detection sensitivity of small focuses and marginal focuses in glands is enhanced, and the detection accuracy of the small focuses and marginal focuses in the glands is improved. Therefore, the false detection rate and the omission rate of prostate focus detection can be effectively reduced.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a method, apparatus, electronic device, storage medium, and program product for prostate ultrasound image analysis. Background Technology

[0002] In current clinical practice, prostate ultrasound diagnosis mainly relies on physicians' experience for manual observation and subjective interpretation, which poses a serious challenge to its diagnostic efficiency and consistency. With the development of artificial intelligence technology, some studies have attempted to directly apply general target detection models (such as YOLO and Faster R-CNN) to the automatic detection of lesions in prostate ultrasound images.

[0003] However, this type of method has significant limitations: in ultrasound images, prostate lesions are extremely similar to background tissue, high and low echo regions, and even adjacent bladder structures in terms of texture and echo characteristics, making it difficult for the model to distinguish them accurately, resulting in a high rate of false positives and false negatives. Summary of the Invention

[0004] The purpose of this application is to provide a method, apparatus, electronic device, storage medium, and program product for prostate ultrasound image analysis, in order to improve the problem of high false detection and false negative rates in existing methods for detecting prostate lesions.

[0005] In a first aspect, embodiments of this application provide a method for analyzing prostate ultrasound images, the method comprising: Acquire prostate ultrasound images; Multi-scale image features of the prostate ultrasound images are extracted using a shared backbone network in a pre-trained multi-task deep learning model. Prostate segmentation information is predicted based on the multi-scale image features by the prostate segmentation head in the multi-task deep learning model. The lesion detection head in the multi-task deep learning model predicts prostate lesion detection information based on the multi-scale image features; The lesion detection head integrates a cross-attention module, which interacts with the prostate segmentation information output by the prostate segmentation head and the lesion feature information output by the feature extraction module in the lesion detection head to enhance the detection attention of the lesion area.

[0006] In the above implementation process, the consistency of feature representations among different tasks is ensured by using unified multi-scale features extracted by the shared backbone network, thereby improving model efficiency. By using the prostate segmentation results as spatial prior knowledge through the cross-attention module, the lesion detection process is dynamically guided, so that the detection attention is focused on the prostate region, which significantly suppresses the interference and false detection of adjacent tissues such as the bladder. At the same time, it enhances the detection sensitivity of small lesions inside the gland and marginal lesions, thereby effectively reducing the false detection rate and false negative rate of prostate lesion detection.

[0007] Optionally, the step of predicting prostate lesion detection information based on the multi-scale image features using the lesion detection head in the multi-task deep learning model includes: The lesion detection head in the multi-task deep learning model extracts lesion feature information based on the multi-scale image features.

[0008] The prostate segmentation information output by the prostate segmentation head is converted into key-value features, and the lesion feature information extracted by the lesion detection head is converted into query features; The enhanced prostate lesion detection information is obtained through the cross-attention module based on the key value features and query features; Based on the enhanced prostate lesion detection information, the lesion detection head predicts the prostate lesion detection information.

[0009] In the above implementation process, the segmentation results are encoded as key-value features representing anatomical structures, and lesion features are used as queries. An attention mechanism is employed to enable the detection process to dynamically and adaptively focus on the prostate region, thereby effectively suppressing interference from extraprostate tissues (such as the bladder) and significantly reducing the false positive detection rate. Simultaneously, this feature-level enhancement strengthens the response to lesions within the gland, especially those with weak edges or textures, improving detection sensitivity and localization accuracy.

[0010] Optionally, the key-value features include key features and value features. The enhanced prostate lesion detection information is obtained through the cross-attention module based on the key-value features and query features, including: The cross-attention module performs a dot product operation on the query features and the key features to obtain an attention score matrix. Each element in the attention score matrix represents the correlation strength between the lesion detection location and the prostate segmentation location. The attention weight matrix is ​​obtained by dividing the attention score matrix by the square root of the dimension of the key feature through the cross-attention module. The attention weight matrix represents the distribution of attention given to all prostate segmentation locations by the lesion detection location. The enhanced prostate lesion detection information is obtained through the cross-attention module based on the attention weight matrix and the value features.

[0011] In the above implementation process, by calculating the correlation strength between the lesion detection query and the segmentation key, and normalizing it into a probabilistic attention weight distribution, the model can accurately quantify the correlation between each detection location and various regions of the prostate. Finally, this weight is used to perform weighted fusion of segmentation value features, so that in the enhanced lesion detection features, the signal of the lesion region inside the prostate is selectively enhanced, while the noise response outside the gland is effectively suppressed. This process not only significantly improves the specificity and localization accuracy of lesion detection, but also embeds the clinical prior of "lesions are located inside the organ" in a data-driven manner, enhancing the model's discriminative ability and the interpretability of the results.

[0012] Optionally, the method further includes: The lesion classification head in the multi-task deep learning model predicts prostate lesion classification information based on the multi-scale image features.

[0013] In the above implementation process, the classification head directly utilizes the deep semantic features extracted from the shared backbone network, which enables fine-grained classification of lesions.

[0014] Optionally, the lesion classification head includes a benign / malignant lesion classification head and a lesion type classification head. The step of predicting prostate lesion classification information based on the multi-scale image features using the lesion classification head in the multi-task deep learning model includes: The benign or malignant classification information of prostate lesions is predicted based on the multi-scale image features using the lesion benign or malignant classification head. The prostate lesion type classification information is obtained by predicting the prostate lesion type based on the multi-scale image features using the lesion type classification head.

[0015] In the above implementation process, this dual-branch structure enables the model to make independent and collaborative judgments on the clinical risk level (benign or malignant) of lesions and specific pathological manifestations (such as calcification, cysts, nodules, etc.), providing doctors with a more comprehensive and hierarchical diagnostic reference; the two classification heads share the deep semantic features extracted by the backbone network, ensuring feature consistency and improving computational efficiency.

[0016] Optionally, the shared backbone network is a self-supervised pre-trained DINOv3 model based on the Vision Transformer architecture; And / or, the lesion detection head is implemented based on an anchor-free detection architecture; And / or, the prostate segmentation head is implemented based on the UNet-style Decoder architecture.

[0017] In the above implementation process, a self-supervised pre-trained DINOv3 model based on Vision Transformer is used as a shared backbone network, enabling it to learn powerful general visual representations from massive amounts of unlabeled data, significantly reducing the dependence on medical labeled data, and enhancing the model's ability to extract and generalize complex ultrasound image features. The lesion detection head combined with the anchor-free detection architecture simplifies the model design and improves the detection flexibility and localization accuracy of multi-scale and multi-shape lesions. At the same time, the prostate segmentation head based on UNet-style Decoder effectively fuses multi-scale features through skip connections, achieving fine segmentation of the prostate organ boundary.

[0018] Optionally, the multi-task deep learning model is trained in the following manner: Acquire multiple prostate ultrasound training images, including labeled and unlabeled images; The multiple prostate ultrasound training images are input into a multi-task deep learning model for training, and the joint loss of the multi-task deep learning model is obtained. The joint loss includes a weighted sum of segmentation loss and detection loss. When the joint loss satisfies the loss condition, a well-trained multi-task deep learning model is obtained.

[0019] In the above implementation process, this training method can make full use of massive and easily accessible unlabeled data to enhance the model's generalization ability and feature learning robustness, while combining limited labeled data to ensure the accuracy of key supervision signals. By designing a weighted joint optimization of segmentation and detection losses, feature sharing and knowledge complementarity between different tasks are promoted, and the synergistic improvement of organ segmentation and lesion detection is achieved. This not only reduces the risk of overfitting, but also significantly improves the overall analysis accuracy and stability of the model on complex ultrasound images.

[0020] Optionally, during the training of the multi-task deep learning model, a segmented unfreezing strategy is used to train the shared backbone network.

[0021] In the aforementioned implementation process, this strategy achieves stable and efficient transfer of general visual knowledge to the medical image domain by gradually unfreezing the parameters of the pre-trained backbone network in stages and from deep to shallow. Simultaneously, the progressive fine-tuning process allows the network to adapt hierarchically to the unique texture, structure, and semantic information of prostate ultrasound images. Ultimately, while fully preserving prior knowledge, it optimizes the model's performance on the target medical task, improving the stability of the training process and the generalization ability of the final model.

[0022] Secondly, embodiments of this application provide a prostate ultrasound image analysis device, the device comprising: The image acquisition module is used to acquire ultrasound images of the prostate. The feature extraction module is used to extract multi-scale image features of the prostate ultrasound image through a shared backbone network in a pre-trained multi-task deep learning model. The segmentation module is used to predict prostate segmentation information based on the multi-scale image features using the prostate segmentation head in the multi-task deep learning model; The lesion detection module is used to predict prostate lesion detection information based on the multi-scale image features through the lesion detection head in the multi-task deep learning model; The lesion detection head integrates a cross-attention module, which interacts with the prostate segmentation information output by the prostate segmentation head and the lesion feature information output by the feature extraction module in the lesion detection head to enhance the detection attention of the lesion area.

[0023] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the method provided in the first aspect above are performed.

[0024] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the method provided in the first aspect above.

[0025] Fifthly, embodiments of this application provide a computer program product, including computer program instructions, which, when read and executed by a processor, perform the steps of the method provided in the first aspect above.

[0026] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing embodiments of this application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0027] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 A flowchart of a prostate ultrasound image analysis method provided in this application embodiment; Figure 2 This is a schematic diagram of the system architecture corresponding to a prostate ultrasound image analysis method provided in an embodiment of this application; Figure 3 A structural block diagram of a prostate ultrasound image analysis device provided in this application embodiment; Figure 4 This is a schematic diagram of the structure of an electronic device for performing a prostate ultrasound image analysis method, provided as an embodiment of this application. Detailed Implementation

[0029] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0030] It should be noted that the terms "system" and "network" in the embodiments of this invention can be used interchangeably. "Multiple" refers to two or more; therefore, in the embodiments of this invention, "multiple" can also be understood as "at least two". "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / ", unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship.

[0031] It should also be noted that all actions involving the acquisition of signals, information, or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where the application is located, and with the authorization granted by the owner of the relevant device.

[0032] This application provides a method for prostate ultrasound image analysis. This method ensures the consistency of feature representations across different tasks by using unified multi-scale features extracted from a shared backbone network, thereby improving model efficiency. By using the prostate segmentation results as spatial prior knowledge through a cross-attention module, the method dynamically guides the lesion detection process, focusing detection attention on the prostate region. This significantly suppresses interference and false detections from adjacent tissues such as the bladder, while enhancing the detection sensitivity for small lesions within the gland and marginal lesions. As a result, it can effectively reduce the false detection rate and missed detection rate of prostate lesions.

[0033] Please refer to Figure 1 , Figure 1 A flowchart of a prostate ultrasound image analysis method provided in this application embodiment is included, the method comprising the following steps: Step S110: Obtain ultrasound images of the prostate.

[0034] The analysis system can receive real-time video streams or static ultrasound images from ultrasound equipment. To facilitate accurate subsequent analysis, the system can first perform standardized preprocessing on each frame of the input prostate ultrasound image, including resolution adjustment and grayscale normalization.

[0035] Resolution adjustment: Standardize the image size to a fixed value (e.g., 640*640 pixels). If the original image resolution is too high, edge-preserving downsampling algorithms (such as Lanczos interpolation) can be used to preserve key anatomical details.

[0036] Gray-level normalization: linearly maps the pixel value range of the image to [0,255] and performs contrast-limited adaptive histogram equalization to reduce imaging differences caused by different devices and different gain settings.

[0037] Finally, a standard-sized image can be generated as input to a multi-task deep learning model, for example, with dimensions [1,3,H,W], where 1 represents the batch size, 3 represents the RGB channels, H represents the image height, and W represents the image width.

[0038] Step S120: Extract multi-scale image features of prostate ultrasound images through a shared backbone network in a pre-trained multi-task deep learning model.

[0039] Multi-task deep learning models are a type of neural network model in which the shared backbone network can be a self-supervised learning network that has been pre-learned on massive amounts of natural images through self-supervised learning and has powerful general visual representation capabilities.

[0040] A shared backbone network can refer to a single feature extractor that is used by all subsequent tasks (such as segmentation, detection, etc.). This avoids training a separate feature extractor for each task, effectively improving efficiency and ensuring feature consistency.

[0041] The prostate ultrasound images obtained after the above preprocessing steps can be input into a shared backbone network to extract multi-scale image features. Multi-scale image features refer to a set of feature maps extracted from different depths of the shared backbone network, each with different receptive field kernel semantic layers.

[0042] Multi-scale image features can include feature maps at different levels, such as shallow features, mid-level features, and deep features. Shallow features have high spatial resolution and contain rich edge, texture, and detail information, making them suitable for accurate organ boundary segmentation. Mid-level features have moderate spatial resolution and contain structural and morphological information about organs and lesions, making them core features for object detection. Deep features have high semantic information and global context, but lower spatial resolution, making them suitable for classification tasks that require judgment of global information.

[0043] Step S130: Obtain prostate segmentation information by predicting multi-scale image features using the prostate segmentation head in the multi-task deep learning model.

[0044] Among them, the prostate segmentation head can be a decoder that receives multi-scale image features from a shared backbone network. It can gradually restore spatial details by fusing deep high-semantic features with shallow high-resolution features layer by layer through upsampling and skip connections.

[0045] The skip connection here can directly stitch the feature map of a certain layer in the shared backbone network onto the feature map of the corresponding layer in the decoder. This helps to pass the fine spatial information extracted in the early stage to the later stage and solves the problem of spatial information loss caused by downsampling.

[0046] The decoder then uses a convolutional layer and a sigmoid activation function to output a probability map that is identical to the input image, where the value of each pixel represents the probability that it belongs to the prostate region. By setting a threshold, a binarized prostate segmentation mask, i.e., prostate segmentation information, can be obtained.

[0047] A prostate segmentation mask is a matrix of the same size as the original image, where the foreground (prostate organ) pixel value is 1 and the background pixel value is 0, which can reflect the region where the prostate organ is located.

[0048] Step S140: Obtain prostate lesion detection information by predicting multi-scale image features using the lesion detection head in the multi-task deep learning model.

[0049] The lesion detection head integrates a cross-attention module, which interacts with the prostate segmentation information output by the prostate segmentation head and the lesion feature information output by the feature extraction module in the lesion detection head to enhance the detection attention of the lesion area.

[0050] Cross-attention modules are neural network modules that allow features of one task (prostate segmentation features in this scheme) to dynamically and adaptively influence and enhance features of another task (lesion detection features in this scheme). The core of these modules is the attention mechanism, which learns where to focus attention.

[0051] The lesion detection head can first extract lesion feature information for lesion detection from the multi-scale image features of the shared backbone network, where the feature extraction module has already extracted the lesion feature information, which includes preliminary lesion location and texture clues.

[0052] The cross-attention module receives two inputs: one is lesion feature information output from the feature extraction module of the lesion detection head, and the other is prostate segmentation information from the prostate segmentation head, which encodes a strong prior knowledge of the prostate spatial structure.

[0053] To enhance the detection focus on lesion regions, the cross-attention module can calculate the similarity between lesion feature information and prostate segmentation information through a scaled dot product attention mechanism, resulting in an attention weight map. This weight map essentially reflects "which part of the prostate region should receive how much attention at each location in the detection features." Then, this weight map is used to perform a weighted summation of the prostate segmentation information to generate a modulated enhanced feature.

[0054] By connecting this enhancement feature with the lesion feature information through residuals, the final enhanced feature can be obtained. This process strengthens the detection feature response located inside the prostate region while suppressing the noise response outside the gland.

[0055] This enhanced feature can be fed into subsequent sub-networks (such as classification and regression branches) in the lesion detection head, and finally, prostate lesion detection information can be obtained, which may include candidate lesion boxes. Each candidate lesion box may include the bounding box coordinates and the confidence score of the presence of a lesion within the box.

[0056] Let's illustrate the role of cross-attention modules with an example: Suppose that the lesion feature information from the lesion detection head also has a strong response to a hyperechoic point (non-lesion) in the bladder region. In the cross-attention module, the prostate segmentation features from the segmentation clearly indicate that this point is located outside the prostate contour. Therefore, the attention weights calculated by the cross-attention module reduce the importance of this location feature. Ultimately, in the enhanced features, the response of this hyperechoic point is weakened, thus avoiding misdetection as a prostate lesion. Conversely, a small lesion with weak features located at the internal boundary of the prostate will have its response enhanced through association with clear boundary features, thereby improving the detection rate.

[0057] In the above implementation process, the consistency of feature representations among different tasks is ensured by using unified multi-scale features extracted by the shared backbone network, thereby improving model efficiency. By using the prostate segmentation results as spatial prior knowledge through the cross-attention module, the lesion detection process is dynamically guided, so that the detection attention is focused on the prostate region, which significantly suppresses the interference and false detection of adjacent tissues such as the bladder. At the same time, it enhances the detection sensitivity of small lesions inside the gland and marginal lesions, thereby effectively reducing the false detection rate and false negative rate of prostate lesion detection.

[0058] Based on the above embodiments, during the processing of the cross-attention module, the lesion detection head extracts lesion feature information from multi-scale image features. This converts the prostate segmentation information output by the prostate segmentation head into key-value features, and the lesion feature information extracted by the lesion detection head into query features. Then, the cross-attention module obtains the enhanced prostate lesion detection information based on the key-value features and query features. When performing prediction using the lesion detection head, it can predict the final prostate lesion detection information based on the enhanced prostate lesion detection information.

[0059] The key-value features include key features K and value features V, both derived from the prostate segmentation information output by the prostate segmentation head, i.e., prostate segmentation features, denoted as F_seg. These features not only contain coarse spatial information about the prostate region but also retain richer semantic and contextual information than the final binary mask, serving as a carrier of spatial prior knowledge.

[0060] The query feature Q comes from the lesion feature information output by the lesion detection head in the early processing stage. It is usually a feature map, denoted as F_det. This feature map encodes preliminary retrieval clues such as regions and textures in the image that may contain lesions, but has not yet been filtered by anatomical structure information.

[0061] In the cross-attention module, input features are mapped to a unified feature space through three independent, learnable linear transformation layers (typically 1x1 convolutions), generating Q, K, and V. By using prostate segmentation information as keys and values, and lesion feature information as query features, feature interaction can be achieved. This ensures that noisy structures located outside the prostate are not noticed by the detection head, while suspected lesions inside the prostate are amplified and focused on. At the same time, it can improve the detection accuracy of small lesions and boundary lesions.

[0062] Query feature Q: Q = W_q * F_det, where W_q is a learnable weight matrix used to convert lesion feature information into a series of query vectors, each vector representing a specific location in the detected feature asking which part of the segmentation feature should be focused on more.

[0063] Key-valued features: K = W_k * F_seg, V = W_v * F_seg. W_k and W_v are also weight matrices that convert prostate segmentation features into index vectors. Each index vector summarizes the spatial context information of a certain location in the segmentation feature map. W_v converts the segmentation features into content vectors, which carry the actual anatomical structural information to be transmitted and aggregated.

[0064] The cross-attention module can perform correlation operations on key-value features and query features to obtain enhanced prostate lesion detection information.

[0065] In some implementations, in order to preserve the original detection clues and ensure the stability of the prediction, the obtained enhanced prostate lesion detection information can be residually connected with the lesion feature information extracted by the lesion detection head (such as addition or weighted summation) to obtain the final enhanced detection features, i.e., the enhanced prostate lesion detection information.

[0066] The enhanced prostate lesion detection information can then be used as the main input features and passed to the subsequent processing layers of the lesion detection head (e.g., convolutional layers for classifying "whether it is a lesion" and convolutional layers for regressing "lesion box location"). These subsequent layers of the lesion detection head decrypt the enhanced prostate lesion detection information and output the final prostate lesion detection information.

[0067] In the above implementation process, the segmentation results are encoded as key-value features representing anatomical structures, and the initial lesion features are used as queries. An attention mechanism is employed to enable the detection process to dynamically and adaptively focus on the prostate region, thereby effectively suppressing interference from extraprostate tissues (such as the bladder) and significantly reducing the false positive detection rate. Simultaneously, this feature-level enhancement strengthens the response to lesions within the gland, especially those with weak edges or textures, improving detection sensitivity and localization accuracy.

[0068] In the above implementation, the key-value features may include key features and value features. The cross-attention module can calculate the enhanced prostate lesion detection information based on the key-value features and query features. The calculation process includes: performing a dot product operation on the query features and key features to obtain an attention score matrix. Each element in the attention score matrix represents the correlation strength between the lesion detection location and the prostate segmentation location. Then, the mean attention score is divided by the square root of the dimension of the key features to obtain an attention weight matrix. The attention weight matrix represents the distribution of attention given to all prostate segmentation locations by the lesion detection location. Finally, the enhanced prostate lesion detection information is obtained based on the attention weight matrix and value features.

[0069] The calculation across attention modules is as follows:

[0070] in, This represents the enhanced prostate lesion detection information, where d represents the dimension of the key feature, Q represents the query feature, K represents the key feature, and V represents the value feature.

[0071] The calculation formula first calculates the similarity between the query feature Q and the key feature K. Specifically, it performs a dot product operation between the vector of each spatial location of Q and the vector of all spatial locations of K to obtain an attention score matrix, where the element (i,j) represents the association strength between the i-th detection location and the j-th segmentation location.

[0072] Then, the dot product result is divided by the square root of the dimension d of the key feature to prevent gradient vanishing. Next, the Softmax function is applied to each row (corresponding to a query position) for normalization, resulting in the final attention weight matrix. The sum of each row of the attention weight matrix is ​​1, representing the distribution of the model's attention to all positions in the segmentation feature map for a specific detection position.

[0073] The calculated attention weight matrix A is used to perform a weighted summation on the value features V, with the formula F'_det=A*V. For the i-th detection position, its enhanced feature F'_det_i is the weighted sum of all positional features in V, and the weights are the i-th row of A.

[0074] F'_det is the enhanced prostate lesion detection information after segmentation features, which contains the anatomical context most relevant to the current detection task.

[0075] For example, suppose a location Q_i in the detection features corresponds to a suspected lesion point in the image. Calculations show that it has the highest similarity to point K_j in the segmentation features, located in the central region of the prostate. Therefore, the attention weight matrix A_{i,j} obtained after softmax will be very large (close to 1). When aggregating V, the anatomical feature V_j from the central region of the prostate will dominate the content of F'_det_i, causing the model to refer to the prior information that "it is located in the center of the prostate" when determining whether the point is a lesion. Conversely, if a detection point Q_m has extremely low similarity to all K points in the prostate region, its attention weights will be evenly distributed, resulting in weak information in the aggregated F'_det_m, and the detection response of that point will be suppressed in subsequent processing.

[0076] In the above implementation process, by calculating the correlation strength between the lesion detection query and the segmentation key, and normalizing it into a probabilistic attention weight distribution, the model can accurately quantify the correlation between each detection location and various regions of the prostate. Finally, this weight is used to perform weighted fusion of segmentation value features, so that in the enhanced lesion detection features, the signal of the lesion region inside the prostate is selectively enhanced, while the noise response outside the gland is effectively suppressed. This process not only significantly improves the specificity and localization accuracy of lesion detection, but also embeds the clinical prior of "lesions are located inside the organ" in a data-driven manner, enhancing the model's discriminative ability and the interpretability of the results.

[0077] Based on the above embodiments, the multi-task deep learning model also includes a lesion classification head, which can predict prostate lesion classification information based on multi-scale image features.

[0078] Prostate lesion classification information may include lesion type classification information. The lesion classification head can generally be composed of multiple fully connected layers, and each fully connected layer can correspond to a classification task and a Softmax layer.

[0079] In some implementations, the input to the lesion classification head also includes the output of the lesion detection head. That is, the lesion detection head can input the coordinates and size information of the lesion detection box obtained by localization into the lesion classification head. In this way, the lesion classification head can locate the corresponding lesion position by ROI alignment, and then perform lesion classification detection based on multi-scale image features and lesion position.

[0080] In some implementations, the lesion classification head includes a benign / malignant lesion classification head and a lesion type classification head. The benign / malignant lesion classification information of the prostate lesion can be predicted based on multi-scale image features using the benign / malignant lesion classification head, and the prostate lesion type classification information can be predicted based on multi-scale image features using the lesion type classification head.

[0081] The lesion benign / malignant classification head and the lesion type classification head can share the same feature extraction process of the backbone network, but they have independent classification layers to output classification information in different dimensions.

[0082] In practice, if the input of the lesion classification head does not include the lesion detection head, then the structure of the lesion detection head can be integrated into the lesion benign / malignant classification head and the lesion type classification head for lesion detection. If its input is connected to the lesion detection head, the lesion area can be located directly based on the output of the aforementioned lesion detection head.

[0083] In this way, the lesion benign / malignant classification head and the lesion type classification head can accurately extract the corresponding feature regions from multi-scale image features based on the coordinates of the lesion detection box. For example, for an irregular lesion region, the classification head will accurately sample the corresponding points on the multi-scale image feature map, and calculate the value of each position on a fixed-size grid through interpolation, thereby obtaining accurate features that are not affected by the rounding of the bounding box coordinates.

[0084] The two classifiers can be independent, small fully connected networks, typically a single linear transformation. The weights of the two classifiers are not shared.

[0085] For the lesion benign / malignant classification head, it can reduce the dimensionality and abstract the multi-scale image features, extract the information most relevant to the classification, and then map all lesion feature vectors to the number of benign / malignant categories (e.g., 3 categories, including malignant, benign, and uncertain) through a fully connected layer. Then, it can be converted into a probability distribution through the softmax function, outputting a probability matrix, where each row represents the probability of a lesion belonging to each benign / malignant category.

[0086] For example, for a candidate lesion, the lesion benign or malignant classification head outputs a probability vector [0.02, 0.15, 0.83], indicating that the model judges it as "uncertain" with the highest probability (83%), "benign" with a probability of 15%, and "malignant" with a probability of 2%.

[0087] For the lesion type classification head, it can reduce the dimensionality and abstract multi-scale image features, extract the information most relevant to the classification, and then map all lesion feature vectors to the number of lesion categories (e.g., 4 categories, including calcifications, cysts, hypoechoic nodules, and others) through a fully connected layer. Similarly, it is converted into a probability distribution through a softmax function, outputting a probability matrix, where each row represents the probability of a lesion belonging to each lesion category.

[0088] For example, for a candidate lesion, the lesion type classification head outputs a probability vector [0.70, 0.25, 0.04, 0.01], indicating that the model judges it as "calcification" with the highest probability (70%), "cyst" with a probability of 25%, "hypoechoic nodule" with a probability of 4%, and "other" with a probability of 1%.

[0089] like Figure 2 The diagram shown illustrates one system architecture of this solution. The prostate ultrasound image is first adaptively adjusted for resolution, then input into a shared backbone network to extract multi-scale image features. These multi-scale features are then input into the prostate segmentation head, lesion detection head, and lesion classification head for detection, yielding the final output information. The final output information may include anterior lacrimal gland segmentation information, lesion detection box coordinates, detection confidence score, benign / malignant probability distribution for each lesion, and lesion type probability distribution for each lesion.

[0090] In the above implementation process, this dual-branch structure enables the model to make independent and collaborative judgments on the clinical risk level (benign or malignant) of lesions and specific pathological manifestations (such as calcification, cysts, nodules, etc.), providing doctors with a more comprehensive and hierarchical diagnostic reference; the two classification heads share the deep semantic features extracted by the backbone network, ensuring feature consistency and improving computational efficiency.

[0091] In some implementations, the shared backbone network described above can be a self-supervised pre-trained DINOv3 model based on the Vision Transformer architecture.

[0092] Specifically, the DINOv3 model series (such as ViT-L / 14) can be selected as the shared backbone network. DINOv3 is a model based on the Vision Transformer architecture and pre-trained on a very large-scale unlabeled dataset using the DINO (Self-Distillation Unlabeled Learning) method.

[0093] Through self-supervised learning, the model can understand general visual concepts of object parts, textures, and scenes in images, a capability that can be highly transferred to the field of medical imaging. Furthermore, leveraging its powerful pre-trained weights as a starting point, it can achieve excellent performance through fine-tuning on a relatively small number of labeled medical images, making it better suited for medical scenarios where annotation costs are high. The Self-Attention mechanism of the Transformer architecture enables it to model long-range dependencies between any two locations in an image, which is crucial for understanding the overall morphology of the prostate and the relationship between lesions and surrounding tissues.

[0094] When extracting multi-scale image features through this shared backbone network, a preprocessed prostate ultrasound image (e.g., 640*640*3) can be input into DINOv3. The network first segments the image into fixed-size (e.g., 14x14) non-overlapping image patches, then linearly projects each image patch into a feature vector (called Patch Embedding), adds positional encoding, and forms a sequence input.

[0095] The sequence is passed through a series of Transformer Encoder layers. Each layer contains a Multi-Head Self-Attention Network and a Feed-Forward Network. The network outputs feature maps at multiple layers.

[0096] To meet the needs of multiple tasks such as segmentation and detection, features of different depths can be extracted to construct a feature pyramid: High-resolution, low-semantic feature maps (e.g., 160x160 spatial size) rich in edge and texture details are extracted from shallower layers (e.g., layer 6) for fine segmentation.

[0097] Extracting medium-resolution, medium-semantic feature maps (e.g., 80x80) from middle layers (e.g., the 12th layer), which contain structural information of organs and lesions, is the core of object detection.

[0098] Extracting low-resolution, high-semantic feature maps (such as 40x40 or cls_token) from deep layers (e.g., the 18th or last layer) that contain global context is suitable for classification tasks.

[0099] In some implementations, the lesion detection head can be based on an anchor-free detection architecture.

[0100] Specifically, the lesion detection head can be improved using an anchor-free detection head, such as YOLOv10. It eliminates the need for predefined anchor boxes of various shapes and sizes found in traditional detectors. This eliminates the need for manually designing anchor sizes and proportions, simplifying the model and reducing hyperparameters. It can directly predict the target center point and boundaries, avoiding deviations caused by matching with preset anchors. It is more flexible in prostate ultrasound images with varying lesion shapes, and the calculations are more direct, typically resulting in faster inference speeds.

[0101] During prostate lesion detection using a lesion detection head, multi-scale feature maps, such as mid-level features, can be received from a shared backbone network. Firstly, features from adjacent scales can be fused using a lightweight feature pyramid network or SPPF module to enhance multi-scale perception capabilities.

[0102] The lesion detection head may include two parallel branches: Classification branch: A convolutional layer that outputs a feature map, where the value at each location represents the confidence that the location is the center of the lesion; Regression branch: A convolutional layer that outputs a feature map, predicting four values ​​(dx, dy, dw, dh) at each location, representing the offset from the center of the grid to the top left corner of the predicted box, and the width and height of the predicted box (normalized values ​​relative to the feature map size).

[0103] By combining grid coordinates, predicted offsets, and dimensions, the final lesion bounding box (x1, y1, x2, y2) and its confidence level are decoded.

[0104] In some implementations, the prostate segmentation head can be based on the UNet-style Decoder architecture.

[0105] Specifically, a UNet-style decoder is adopted. UNet is characterized by its encoder-decoder structure and smooth connections, making it suitable for medical image segmentation.

[0106] During prostate segmentation, multi-scale feature maps from a shared backbone network can be received, including both shallow and deep features. The deep semantic features F_deep (e.g., 40x40xC) are then upsampled (e.g., transposed convolution or bilinear interpolation) to a higher resolution (e.g., 80x80). The upsampled features are then skip-connected (usually a concatenation operation) with features from a middle layer of the backbone network that have had their channel count adjusted by a 1x1 convolution (e.g., F_mid adjusted to 80x80). Several convolutional layers are applied to the concatenated features to extract fused features. This process is repeated, upsampling again (to 160x160), skip-connected with the shallow features F_low, and convolved. The final convolutional layer is followed by a 1x1 convolution and a sigmoid activation function, outputting a single-channel probability heatmap (640x640x1) of the same size as the input image. Each pixel value represents the probability that it belongs to the prostate region. A binary segmentation mask can be obtained by setting a threshold (e.g., 0.5).

[0107] In this method, skip connections can directly transmit high-resolution, low-level features from the encoder (shared backbone network) to the decoder, where they are fused with deep, high-semantic features, resulting in precise segmentation boundaries that are suitable for outlining the irregular contours of the prostate.

[0108] In the above implementation process, a self-supervised pre-trained DINOv3 model based on Vision Transformer is used as a shared backbone network, enabling it to learn powerful general visual representations from massive amounts of unlabeled data, significantly reducing the dependence on medical labeled data, and enhancing the model's ability to extract and generalize complex ultrasound image features. The lesion detection head combined with the anchor-free detection architecture simplifies the model design and improves the detection flexibility and localization accuracy of multi-scale and multi-shape lesions. At the same time, the prostate segmentation head based on UNet-style Decoder effectively fuses multi-scale features through skip connections, achieving fine segmentation of the prostate organ boundary.

[0109] Based on the above embodiments, before using the multi-task deep learning model to analyze prostate ultrasound images, it is necessary to train the multi-task deep learning model. The training process is as follows: Multiple prostate ultrasound training images are acquired, including labeled and unlabeled images. These images are then input into a multi-task deep learning model for training. The joint loss of the multi-task deep learning model is obtained, which is a weighted sum of segmentation loss and detection loss. When the joint loss meets the loss condition, the trained multi-task deep learning model is obtained.

[0110] Regarding the collection of training images: A large amount of prostate ultrasound image data can be collected, covering different probe angles, patient body types, and lesion types. Some of these images can be precisely annotated by professional physicians. The annotation for each image should include at least: a prostate organ segmentation mask and a lesion detection box. If lesion classification is also required, classification labels should also be included. Other images do not need to be annotated; these images are used to assist training and enhance the model's generalization ability.

[0111] The labeled image set can be divided into training, validation, and test sets in approximately a 7:1:2 ratio. Unlabeled images are used only for specific stages of the training phase.

[0112] In some implementations, strong data augmentation can be applied to training images (including labeled and unlabeled images) to simulate the diversity of ultrasound imaging and improve model robustness. Augmentation operations include: Spatial transformations: random horizontal / vertical flip, random rotation (±15°), random scaling (0.8-1.2 times); Appearance transformation: Randomly adjust brightness and contrast, add Gaussian noise, and simulate ultrasonic speckle noise; Elastic deformation: The deformation of tissue caused by changes in probe pressure is simulated by applying random elastic mesh deformation. MixUp: Combines two images and their labels (for labeled images) at a certain ratio to create intermediate samples.

[0113] A batch of training images (including labeled and unlabeled images) is input into a multi-task deep learning model. Labeled images, after feature extraction through a shared backbone network, are activated across all task heads (segmentation, detection, and classification heads), producing corresponding predictions. Unlabeled images are typically processed only through the shared backbone network and the segmentation / detection heads, used to calculate consistency regularization loss or auxiliary self-supervised learning loss; their classification heads may be skipped. For example, performing two different random data augmentations on the same unlabeled image, inputting them into the shared backbone network and the segmentation head respectively, and calculating the consistency loss between the two output segmentation masks enhances the model's robustness to image transformations. Additionally, unlabeled images can also be used to calculate feature consistency regularization loss for detection tasks, encouraging the model to output similar feature representations for the same sample in different augmented versions at the feature level, thereby improving the model's generalization ability with limited labeled data.

[0114] Joint loss is the weighted sum of the losses from each task, and its general form is: ; in, These are learnable and adjustable weight parameters. The segmentation loss for the segmentation task. Focal Loss is the detection loss for the detection task. For the cross-entropy loss of the malicious classification task, The cross-entropy loss is used for the lesion type classification task (if there is no classification head, the joint loss may not include the cross-entropy loss). This joint training can achieve mutual regularization among the tasks, while improving the robustness and generalization ability of the shared backbone network and reducing the model's sensitivity to single-task noise.

[0115] For each training batch (which typically contains labeled and unlabeled images), the process includes the following: Forward propagation: Input the image to the model and calculate the loss for all activation tasks; Loss-weighted summation: Calculate the total loss based on the current weights (fixed or dynamic); Backpropagation: Calculates the gradient of the loss with respect to all trainable parameters of the model; Gradient clipping: To prevent gradient explosion, the gradient norm is clipped (e.g., a threshold of 1.0 is set). Parameter update: The optimizer updates the model parameters based on the gradient.

[0116] Training terminates when any of the following conditions are met: Early cessation: The core metrics monitored on the validation set (such as mAP for lesion detection) no longer improve within P consecutive rounds (e.g., 15 rounds).

[0117] Reach the maximum number of rounds: The training reaches the preset maximum number of rounds (e.g., 200 rounds).

[0118] Loss convergence: The total loss decreases to a stable plateau, and the change is less than the threshold δ (e.g., 1e-5) for several rounds.

[0119] After training, the model parameters that perform best on the validation set are selected as the final trained multi-task deep learning model for subsequent inference analysis.

[0120] In the above implementation process, this training method can make full use of massive and easily accessible unlabeled data to enhance the model's generalization ability and feature learning robustness, while combining limited labeled data to ensure the accuracy of key supervision signals. By designing a weighted joint optimization of segmentation and detection losses, feature sharing and knowledge complementarity between different tasks are promoted, and the synergistic improvement of organ segmentation and lesion detection is achieved. This not only reduces the risk of overfitting, but also significantly improves the overall analysis accuracy and stability of the model on complex ultrasound images.

[0121] Based on the above embodiments, a segmented unfreezing strategy is adopted to train the shared backbone network during the training process of the multi-task deep learning model.

[0122] Segmented unfreezing is a refined transfer learning technique designed to safely and efficiently adapt large-scale pre-trained general models (such as DINOv3) to specific medical imaging tasks. Its core principle is to progressively unlock (unfreeze) the parameters of the backbone network in stages, from deep to shallow, to prevent catastrophic forgetting and achieve smooth knowledge transfer.

[0123] The segmented unfreezing strategy can be divided into three stages, simulating the learning process of "first understanding high-level semantics, then adjusting intermediate structure, and finally fine-tuning basic features".

[0124] Phase 1 (Freeze the backbone, learn the task): Only the newly added task heads are trained, and the backbone network is used as a fixed feature extractor.

[0125] Phase Two (Gradual Unfreezing, Adaptation to the Domain): Starting from the deep layers of the network, gradually unfreeze parts of the backbone network upwards to adapt it to the medical image domain.

[0126] Phase 3 (Complete Fine-tuning, Collaborative Optimization): Unfreeze all network parameters and perform fine-tuning with a low learning rate to maximize overall performance.

[0127] Its goal is to maximize the use of the general knowledge of the pre-trained model with limited medical labeled data, while enabling the model to robustly learn the patterns and features specific to prostate ultrasound images.

[0128] (Taking DINOv3-ViT-L / 14 as an example), assume that the shared backbone network has 24 Transformer encoder blocks.

[0129] Phase 1: Freeze the backbone network and train the task leader.

[0130] The weights of all 24 Transformer blocks in the DINOv3 backbone network and the patch embedding layers are set to `requires_grad=False`, effectively freezing them. Gradients are not computed in these parts, and parameters are not updated.

[0131] The parameters of newly added modules such as segmentation heads, detection heads, and classification heads remain trainable. At this stage, training uses a high learning rate, the aim of which is to enable these new task heads to quickly learn how to interpret and utilize the general visual features provided by the backbone network, which are pre-trained from massive amounts of natural images, thereby initially establishing the basic ability to solve downstream medical tasks.

[0132] Once the task head training stabilizes, we move on to Phase Two.

[0133] Phase Two: Gradual, layered unfreezing and adaptation of the backbone network.

[0134] This phase begins at the deepest layer of the network (responsible for high-level semantic abstraction) and gradually unfreezes the Transformer blocks upwards according to a pre-defined plan. For example, it can be set to unfreeze two additional blocks every 10 training iterations, starting from the last 23rd and 24th layers and gradually unfreezing up to around the 10th layer in the middle of the network.

[0135] In this process, to balance the update speed of new and old knowledge, a hierarchical learning rate strategy was adopted: a smaller learning rate was applied to newly thawed backbone network layers to gently adjust their weights and gradually adapt them to the domain features of medical images; while a relatively higher learning rate was used to continue optimization for task heads that had been trained for some time. The goal of this stage is to enable the deep and intermediate layers of the model to learn to understand the high-level semantic and structural information of organs and lesions in prostate ultrasound images, achieving a smooth transition from general visual representations to medical professional concepts.

[0136] Phase 3: Full unfreezing and low-learning-rate fine-tuning.

[0137] After laying a solid foundation for the model in the first two stages, stage three initiates complete fine-tuning to pursue optimal performance. At this point, all remaining shallow parameters in the backbone network (including the first 10 Transformer blocks and patch embedding layers) are unfrozen, making all parameters of the entire model available for updating. The training configuration then adopts a globally uniform, extremely low learning rate, and typically enhances regularization techniques such as weight decay to prevent overfitting on small-scale medical data.

[0138] The purpose of this stage is to perform final fine calibration on the model's low-level feature extractor, enabling it to better capture low-level patterns such as texture and speckle noise unique to ultrasound images, and to achieve global collaborative optimization between the backbone network and all task heads, thereby fully releasing the model's potential and achieving the highest segmentation, detection, and classification accuracy.

[0139] In the aforementioned implementation process, this strategy achieves stable and efficient transfer of general visual knowledge to the medical image domain by gradually unfreezing the parameters of the pre-trained backbone network in stages and from deep to shallow. Simultaneously, the progressive fine-tuning process allows the network to adapt hierarchically to the unique texture, structure, and semantic information of prostate ultrasound images. Ultimately, while fully preserving prior knowledge, it optimizes the model's performance on the target medical task, improving the stability of the training process and the generalization ability of the final model.

[0140] Please refer to Figure 3 , Figure 3This is a structural block diagram of a prostate ultrasound image analysis device 200 provided in an embodiment of this application. The device 200 may be a module, program segment, or code on an electronic device. It should be understood that the device 200 corresponds to the above method embodiment and is capable of performing the various steps involved in the method embodiment. The specific functions of the device 200 can be found in the description above. To avoid repetition, detailed descriptions are appropriately omitted here.

[0141] Optionally, the device 200 includes: Image acquisition module 210 is used to acquire ultrasound images of the prostate. Feature extraction module 220 is used to extract multi-scale image features of the prostate ultrasound image through a shared backbone network in a pre-trained multi-task deep learning model; The segmentation module 230 is used to predict prostate segmentation information based on the multi-scale image features using the prostate segmentation head in the multi-task deep learning model; The lesion detection module 240 is used to predict prostate lesion detection information based on the multi-scale image features through the lesion detection head in the multi-task deep learning model; The lesion detection head integrates a cross-attention module, which interacts with the prostate segmentation information output by the prostate segmentation head and the lesion feature information output by the feature extraction module in the lesion detection head to enhance the detection attention of the lesion area.

[0142] Optionally, the lesion detection module 240 is configured to: extract lesion feature information based on the multi-scale image features using the lesion detection head in the multi-task deep learning model; convert the prostate segmentation information output by the prostate segmentation head into key-value features; and convert the lesion feature information extracted by the lesion detection head into query features; obtain enhanced prostate lesion detection information based on the key-value features and query features using the cross-attention module; and predict prostate lesion detection information based on the enhanced prostate lesion detection information using the lesion detection head.

[0143] Optionally, the key-value feature includes key features and value features. The lesion detection module 240 is used to perform a dot product operation on the query feature and the key feature through the cross-attention module to obtain an attention score matrix. Each element in the attention score matrix represents the correlation strength between the lesion detection location and the prostate segmentation location. The attention score matrix is ​​then divided by the square root of the dimension of the key feature through the cross-attention module to obtain an attention weight matrix. The attention weight matrix represents the distribution of attention given to all prostate segmentation locations by the lesion detection location. The enhanced prostate lesion detection information is obtained through the cross-attention module based on the attention weight matrix and the value features.

[0144] Optionally, the device 200 further includes: The classification module is used to predict prostate lesion classification information based on the multi-scale image features using the lesion classification head in the multi-task deep learning model.

[0145] Optionally, the lesion classification head includes a benign / malignant lesion classification head and a lesion type classification head. The classification module is used to predict benign / malignant prostate lesion classification information based on the multi-scale image features using the benign / malignant lesion classification head; and to predict prostate lesion type classification information based on the multi-scale image features using the lesion type classification head.

[0146] Optionally, the shared backbone network is a self-supervised pre-trained DINOv3 model based on the Vision Transformer architecture; And / or, the lesion detection head is implemented based on an anchor-free detection architecture; And / or, the prostate segmentation head is implemented based on the UNet-style Decoder architecture.

[0147] Optionally, the device 200 further includes: A training module is used to acquire multiple prostate ultrasound training images, including labeled and unlabeled images; input the multiple prostate ultrasound training images into a multi-task deep learning model for training, and obtain the joint loss of the multi-task deep learning model, which includes a weighted sum of segmentation loss and detection loss; when the joint loss satisfies the loss condition, a trained multi-task deep learning model is obtained.

[0148] Optionally, during the training of the multi-task deep learning model, a segmented unfreezing strategy is used to train the shared backbone network.

[0149] It should be noted that those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0150] Please refer to Figure 4 , Figure 4 This application provides a schematic diagram of an electronic device for performing a prostate ultrasound image analysis method. The electronic device may include: at least one processor 310, such as a CPU; at least one communication interface 320; at least one memory 330; and at least one communication bus 340. The communication bus 340 is used to establish communication between these components. In this embodiment, the communication interface 320 is used for signaling or data communication with other node devices. The memory 330 may be a high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, the memory 330 may also be at least one storage device located remotely from the processor. The memory 330 stores computer-readable instructions, which, when executed by the processor 310, enable the electronic device to perform the aforementioned method process.

[0151] Understandable. Figure 4 The structure shown is for illustrative purposes only; the electronic device may also include components that are more advanced than those shown. Figure 4 The more or fewer components shown, or having the same Figure 4 The different configurations shown. Figure 4 The components shown can be implemented using hardware, software, or a combination thereof.

[0152] This application provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it performs the method process executed by the electronic device in the above method embodiments.

[0153] This embodiment discloses a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer can perform the methods provided in the above-described method embodiments, such as including: Acquire prostate ultrasound images; Multi-scale image features of the prostate ultrasound images are extracted using a shared backbone network in a pre-trained multi-task deep learning model. Prostate segmentation information is predicted based on the multi-scale image features by the prostate segmentation head in the multi-task deep learning model. The lesion detection head in the multi-task deep learning model predicts prostate lesion detection information based on the multi-scale image features; The lesion detection head integrates a cross-attention module, which interacts with the prostate segmentation information output by the prostate segmentation head and the lesion feature information output by the feature extraction module in the lesion detection head to enhance the detection attention of the lesion area.

[0154] In summary, the embodiments of this application provide a method, device, electronic device, storage medium, and program product for prostate ultrasound image analysis. This method ensures the consistency of feature representations across different tasks by sharing unified multi-scale features extracted from a backbone network, thereby improving model efficiency. By using prostate segmentation results as spatial prior knowledge through a cross-attention module, the detection process is dynamically guided, focusing detection attention on the prostate region. This significantly suppresses interference and false detections from adjacent tissues such as the bladder, while enhancing the detection sensitivity for small lesions within the gland and marginal lesions. Consequently, it can effectively reduce the false detection rate and missed detection rate of prostate lesions.

[0155] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0156] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0157] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0158] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0159] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for analyzing prostate ultrasound images, characterized in that, The method includes: Acquire prostate ultrasound images; Multi-scale image features of the prostate ultrasound images are extracted using a shared backbone network in a pre-trained multi-task deep learning model. Prostate segmentation information is predicted based on the multi-scale image features by the prostate segmentation head in the multi-task deep learning model. The lesion detection head in the multi-task deep learning model predicts prostate lesion detection information based on the multi-scale image features; The lesion detection head integrates a cross-attention module, which interacts with the prostate segmentation information output by the prostate segmentation head and the lesion feature information output by the feature extraction module in the lesion detection head to enhance the detection attention of the lesion area.

2. The method according to claim 1, characterized in that, The process of obtaining prostate lesion detection information based on the multi-scale image features using the lesion detection head in the multi-task deep learning model includes: The lesion detection head in the multi-task deep learning model extracts lesion feature information based on the multi-scale image features; The prostate segmentation information output by the prostate segmentation head is converted into key-value features, and the lesion feature information extracted by the lesion detection head is converted into query features; The enhanced prostate lesion detection information is obtained through the cross-attention module based on the key value features and query features; Based on the enhanced prostate lesion detection information, the lesion detection head predicts the prostate lesion detection information.

3. The method according to claim 2, characterized in that, The key-value features include key features and value features. The enhanced prostate lesion detection information is obtained through the cross-attention module based on the key-value features and query features, including: The cross-attention module performs a dot product operation on the query features and the key features to obtain an attention score matrix. Each element in the attention score matrix represents the correlation strength between the lesion detection location and the prostate segmentation location. The attention weight matrix is ​​obtained by dividing the attention score matrix by the square root of the dimension of the key feature through the cross-attention module. The attention weight matrix represents the distribution of attention given to all prostate segmentation locations by the lesion detection location. The enhanced prostate lesion detection information is obtained through the cross-attention module based on the attention weight matrix and the value features.

4. The method according to claim 1, characterized in that, The method further includes: The lesion classification head in the multi-task deep learning model predicts prostate lesion classification information based on the multi-scale image features.

5. The method according to claim 4, characterized in that, The lesion classification head includes a benign / malignant lesion classification head and a lesion type classification head. The process of predicting prostate lesion classification information based on the multi-scale image features using the lesion classification head in the multi-task deep learning model includes: The benign or malignant classification information of prostate lesions is predicted based on the multi-scale image features using the lesion benign or malignant classification head. The prostate lesion type classification information is obtained by predicting the prostate lesion type based on the multi-scale image features using the lesion type classification head.

6. The method according to claim 1, characterized in that, The shared backbone network is a self-supervised pre-trained DINOv3 model based on the VisionTransformer architecture. And / or, the lesion detection head is implemented based on an anchor-free detection architecture; And / or, the prostate segmentation head is implemented based on the UNet-style Decoder architecture.

7. The method according to any one of claims 1-6, characterized in that, The multi-task deep learning model is trained in the following way: Acquire multiple prostate ultrasound training images, including labeled and unlabeled images; The multiple prostate ultrasound training images are input into a multi-task deep learning model for training, and the joint loss of the multi-task deep learning model is obtained. The joint loss includes a weighted sum of segmentation loss and detection loss. When the joint loss satisfies the loss condition, a well-trained multi-task deep learning model is obtained.

8. The method according to claim 7, characterized in that, During the training of the multi-task deep learning model, a segmented unfreezing strategy is used to train the shared backbone network.

9. A prostate ultrasound image analysis device, characterized in that, The device includes: The image acquisition module is used to acquire ultrasound images of the prostate. The feature extraction module is used to extract multi-scale image features of the prostate ultrasound image through a shared backbone network in a pre-trained multi-task deep learning model. The segmentation module is used to predict prostate segmentation information based on the multi-scale image features using the prostate segmentation head in the multi-task deep learning model; The lesion detection module is used to predict prostate lesion detection information based on the multi-scale image features through the lesion detection head in the multi-task deep learning model; The lesion detection head integrates a cross-attention module, which interacts with the prostate segmentation information output by the prostate segmentation head and the lesion feature information output by the feature extraction module in the lesion detection head to enhance the detection attention of the lesion area.

10. An electronic device, characterized in that, It includes a processor and a memory, the memory storing computer-readable instructions that, when executed by the processor, perform the method as described in any one of claims 1-8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it performs the method as described in any one of claims 1-8.

12. A computer program product, characterized in that, It includes computer program instructions, which, when read and executed by a processor, perform the method as described in any one of claims 1-8.

Images