Tiny focus identification method for prostate puncture and application

Abstract

The invention discloses a tiny focus recognition method for prostate puncture and application, and the method comprises the steps: S1, collecting and preprocessing an MRI image and an ultrasonic image of a patient, and obtaining a preprocessed MRI image and a preprocessed ultrasonic image; s2, constructing a texture attention residual network, and extracting MRI texture enhancement features; constructing a multi-scale edge response gating network, and extracting ultrasonic edge enhancement features; s3, performing spatial transformation and resampling on the ultrasonic edge enhancement features to obtain aligned ultrasonic enhancement features; s4, calculating a focus area weight feature matrix, an MRI dynamic weight matrix, an ultrasonic dynamic weight matrix, redundancy-removed MRI features, redundancy-removed ultrasonic features and tiny protection features; and S5, identifying the tiny focus area, and controlling the puncture robot to puncture. The problems that a traditional method is poor in tiny focus recognition capacity, missed diagnosis of the focus is prone to occurring, and puncture accuracy is insufficient can be solved.

Description

Technical Field

[0001] This invention relates to the field of prostate medical technology, and in particular to a method and application for identifying tiny lesions in prostate biopsy. Background Technology

[0002] Prostate biopsy is a crucial method for diagnosing prostate cancer, and accurate identification of millimeter-scale lesions is a core prerequisite for improving the accuracy of biopsy sampling and reducing the rate of missed diagnoses. Currently, prostate lesion identification in clinical practice mainly relies on single imaging technologies. While magnetic resonance imaging (MRI) offers high soft tissue resolution and can initially display lesions through multi-parameter signal differences, its imaging method limits its ability to achieve real-time matching of tissue location during surgery. Ultrasound imaging can provide real-time intraoperative imaging, but its insufficient soft tissue resolution and susceptibility to artifacts limit its ability to identify small lesions. With the application of deep learning technology in medical imaging, existing solutions mostly build image fusion models based on this technology. However, their core focus is primarily on the "positional alignment" between MRI and ultrasound images, without specific optimization for the characteristics of millimeter-scale lesions. This results in fused images still struggling to clearly distinguish between small lesions and normal tissue, affecting the accuracy of biopsy navigation and increasing the risk of missed diagnoses of small lesions. Summary of the Invention

[0003] In view of this, the present invention aims to provide a method and application for identifying small lesions in prostate biopsy, so as to solve the problem that traditional methods have weak ability to identify small lesions, resulting in easy missed diagnosis of lesions and insufficient puncture accuracy.

[0004] A method for identifying minute lesions in the prostate via biopsy, comprising: S1: Acquire the patient's MRI and ultrasound images and perform preprocessing to obtain preprocessed MRI and ultrasound images; S2: Based on the preprocessed MRI images, a texture attention residual network is constructed to extract MRI texture enhancement features; based on the preprocessed ultrasound images, a multi-scale edge response gating network is constructed to extract ultrasound edge enhancement features. The texture attention residual network extracts basic MRI features through residual network layers, calculates texture feature matrix using gray-level co-occurrence matrix, and combines the features extracted by global covariance pooling with dimension expansion, convolution and splicing operations. Then, it generates texture attention weights through Softmax function, and strengthens the texture information of small lesions through Hadamard product and residual connection, thereby extracting MRI texture enhancement features. The multi-scale edge response gating network calculates multi-scale fused basic ultrasound features through multi-scale convolution, Laplacian operator and Sigmoid function, and then calculates edge enhancement gating coefficients by combining max pooling and average pooling. The edge information of small lesions is enhanced by Hadamard product and residual connection, and then ultrasound edge enhancement features are extracted. S3: Based on the preprocessed MRI image, the ultrasound edge enhancement features are spatially transformed and resampled to obtain aligned ultrasound enhancement features; S4: Calculate the lesion region weight feature matrix based on MRI texture enhancement features and aligned ultrasound enhancement features; then calculate the MRI dynamic weight matrix, ultrasound dynamic weight matrix, redundancy-free MRI features, and redundancy-free ultrasound features; finally, extract the micro-protective features. S5: Identify small lesion areas based on micro-protective features and control the puncture robot to perform puncture.

[0005] Furthermore, step S1 also includes: S11: Acquire MRI images of the patient before surgery using magnetic resonance imaging equipment, including T2WI and DWI sequences; acquire ultrasound images of the patient during surgery using ultrasound equipment; apply non-local mean filtering to denoise the MRI images and adaptive median filtering to denoise the ultrasound images to obtain denoised MRI images and denoised ultrasound images. S12: Based on the denoised MRI image and the denoised ultrasound image, the gray values ​​of the images are adjusted using the Z-score normalization method to obtain the gray-scale normalized MRI image and the gray-scale normalized ultrasound image. S13: Based on the grayscale-normalized MRI image and the grayscale-normalized ultrasound image, bicubic interpolation is used to unify the spatial resolution of the two types of images, resulting in preprocessed MRI images and preprocessed ultrasound images.

[0006] Furthermore, step S2 also includes: S21: Based on the preprocessed MRI images, a texture attention residual network is constructed to enhance the texture information of small lesions and extract features to obtain MRI texture enhancement features. The calculation method is as follows: in, As a basic feature of MRI, For residual network layers, The images are preprocessed MRI images. For texture feature matrix, For the calculation of the gray-level co-occurrence matrix, As texture attention weight, For the Softmax function, For splicing operations, It is a convolutional layer. For dimensional expansion operations, For global covariance pooling, For MRI texture enhancement features, For Hadamah accumulation; S22: Based on the preprocessed ultrasound image, a multi-scale edge response gating network is constructed to enhance the edge information of small lesions and extract features to obtain ultrasound edge enhancement features. The calculation method is as follows: in, Here, represents the basic ultrasound features at the s-th scale, where s is the scale index. The image is a preprocessed ultrasound image. For edge features at scale s, For the Laplace operator, Let be the scale weight coefficient for the s-th scale. For the Sigmoid function, To integrate basic ultrasound features across multiple scales, To enhance the gating coefficient at the edge, For max pooling, For average pooling, To take the absolute value, This is an ultrasound edge enhancement feature.

[0007] It should be further explained that although MRI images can provide high-resolution tissue images, the texture information of small prostate lesions is extremely subtle and the texture difference between them and the surrounding normal tissue is very subtle. They are easily submerged by the texture information of normal tissue. In the traditional feature extraction process, if the network depth is insufficient or there is a lack of targeted texture enhancement mechanism, the subtle texture features of small lesions are easily lost in the feature transmission process, making it difficult to effectively distinguish between lesions and normal tissue, thus affecting the accuracy of subsequent lesion identification. To address the aforementioned challenges, this invention extracts basic MRI features through a residual network layer, laying the foundation for subsequent texture enhancement. Secondly, it utilizes a gray-level co-occurrence matrix to calculate a texture feature matrix, specifically quantifying the texture distribution patterns in local areas to accurately capture subtle differences in local texture between small lesions and normal tissue, compensating for the insufficient prominence of texture information in the basic features. Thirdly, it extracts global statistical features through global covariance pooling, then maps these features to a spatial dimension matching the texture feature matrix through dimensionality expansion. After processing by convolutional layers, these features are concatenated with the texture feature matrix, ensuring that the features simultaneously contain local texture details and global texture statistics. Subsequently, a Softmax function generates texture attention weights, dynamically allocating attention based on texture complexity, allowing the network to automatically focus on small lesion areas with higher texture complexity. Finally, a Hadamard product combines the texture attention weights with the basic MRI features, and superimposes the basic MRI features to form residual connections. This enhances the texture features of the lesion area while avoiding the loss of basic features due to over-enhancement, ensuring that the texture information of small lesions is clearly preserved and highlighted. Existing technologies for extracting lesion features from MRI images often employ conventional convolutional neural networks to directly extract features, lacking targeted attention mechanisms and residual protection mechanisms. This makes it difficult to focus on the subtle textures of small lesions and easily leads to the confusion of texture features between normal tissue and lesions. Furthermore, existing technologies often separate local texture extraction from global feature extraction or simply superimpose different features without forming an organic fusion, resulting in low feature utilization and a high risk of information conflicts. In contrast, this invention fuses the processed global statistical features with the texture feature matrix through a splicing operation, enabling a single feature to simultaneously contain both local and global information. The fused features are then dynamically filtered using texture attention weights, and finally, the enhanced features are fused with the basic features through residual connections. This achieves deep integration of multiple types of features while protecting the original feature information through residual connections. This invention does not simply pile up various technical modules but specifically designs the interaction logic of each module for enhancing the texture of small lesions, ensuring close connection between each link and effectively overcoming the technical bottleneck of multi-module collaborative enhancement.

[0008] On the other hand, ultrasound images are susceptible to artifacts and noise interference, and the edge contours of small prostate lesions are inherently blurred, with a smooth transition between the boundary and the surrounding normal tissue. At the same time, the edges of small lesions of different sizes and locations are difficult to capture completely at a single scale—the edges of small lesions are easily ignored at a large scale, and the edges of large lesions are prone to losing their overall contour at a small scale. This makes it difficult for traditional feature extraction methods to accurately obtain lesion edge information and effectively define the lesion range, which in turn affects the subsequent alignment with MRI features and the accuracy of lesion identification. This invention extracts basic ultrasound features at different scales through multi-scale convolution. For the edge characteristics of small lesions of different sizes, different scales of convolution are used to capture the fine edges of small lesions (small-scale convolution) and the overall contour of large lesions (large-scale convolution), avoiding the omission of edge information at a single scale. Secondly, the Laplacian operator is used to calculate the edge features of the basic ultrasound features at each scale, specifically enhancing the edge details at each scale. Then, the Sigmoid function generates scale weight coefficients for each scale—scales with more significant edge responses (i.e., scales that better match the current lesion edge characteristics) have higher weight coefficients, achieved through Hadamard product and summation operations. Dynamic weighted fusion of multi-scale edge features preserves the dominant edge information at each scale while avoiding interference from ineffective scale features, resulting in multi-scale fused basic ultrasound features. Finally, average pooling is used to calculate the regional mean of the multi-scale fused basic ultrasound features, and the difference between the absolute value of the feature and the mean is used to highlight the distinction between the edge and the background. Max pooling is then used to further enhance significant edge regions, generating edge enhancement gating coefficients. These coefficients are then combined with the multi-scale fused basic ultrasound features through Hadamard product and residual connection, amplifying the lesion edge information a second time while preserving the integrity of the basic features. This forms a complete logical closed loop from edge extraction to enhancement, ensuring that the edges of small lesions are clearly distinguishable. In multi-scale feature fusion, the challenge lies in coordinating the extraction and fusion of multi-scale features, avoiding information conflicts between them, and achieving targeted enhancement of edge features rather than indiscriminate enhancement (such as enhancing edges while amplifying noise or artifacts). Existing technologies often suffer from blurred edge information after multi-scale feature superposition due to a lack of dynamic coordination mechanisms, or poor enhancement results due to the failure to distinguish between effective edges and noise. This invention achieves multi-scale feature pre-screening and fusion by strengthening edges at each scale using the Laplacian operator and generating scale weight coefficients using the Sigmoid function, thus avoiding interference from invalid scales. Simultaneously, by using edge enhancement gating coefficients, it specifically focuses on the difference between edges and background, ensuring that only effective edges are enhanced, not noise or artifacts. This effectively overcomes the technical bottleneck of collaborative enhancement and noise suppression of multi-scale edge features, ensuring that ultrasound edge enhancement features are both complete and accurate.

[0009] Furthermore, step S3 also includes: S31: Using the preprocessed MRI image as the reference template, a multimodal image registration algorithm based on mutual information is used to map the preprocessed ultrasound image to the MRI coordinate system and calculate the spatial transformation matrix. S32: Based on the spatial transformation matrix, the ultrasound edge enhancement features are mapped to the MRI coordinate system to obtain the registered ultrasound enhancement features; S33: Keep the spatial position of the MRI texture enhancement features in the MRI coordinate system unchanged, and resample the registered ultrasound enhancement features according to the spatial resolution of the MRI texture enhancement features to make the spatial resolution of the two consistent, so as to obtain the aligned ultrasound enhancement features.

[0010] Furthermore, step S4 also includes: Based on the redundancy-removed MRI features, the redundancy-removed ultrasound features, and the MRI dynamic weight matrix and ultrasound dynamic weight matrix, the differential features and interaction features are first calculated. Then, the differential features and interaction features are concatenated and processed by convolution. The convolution result is then combined with the sum of the redundancy-removed MRI features and the redundancy-removed ultrasound features by Hadamard product operation. Finally, the differential features are added to extract the micro-protective features.

[0011] Furthermore, step S4 also includes: S41: Calculate the lesion region weighted feature matrix based on MRI texture enhancement features and aligned ultrasound enhancement features. The calculation method is as follows: in, This is the weighted feature matrix of the lesion region. For scaling dot product attention operations, The aligned ultrasound enhancement features; S42: Calculate the MRI dynamic weight matrix, ultrasound dynamic weight matrix, redundancy-free MRI features, and redundancy-free ultrasound features based on MRI texture enhancement features, aligned ultrasound enhancement features, and lesion region weight feature matrix. S43: Based on the redundancy-removed MRI features, redundancy-removed ultrasound features, and the MRI dynamic weight matrix and ultrasound dynamic weight matrix, extract micro-protective features. The calculation method is as follows: in, For difference features, For interactive features, It is a minor protective feature.

[0012] It should be further explained that although the redundant information in the bimodal features (redundant MRI features and redundant ultrasound features) after redundancy removal is removed, the effective features of small lesions are inherently weak. If a simple fusion method is used, such as direct weighted summation, the features of small lesions are easily averaged by the common information in the bimodal features, resulting in the dilution of the features of small lesions. At the same time, there is complementarity between MRI texture details and ultrasound edge information. If this complementary difference cannot be accurately captured and preserved during fusion, the modal advantage cannot be brought into play. Instead, the key features of small lesions are obscured by the fusion operation, making it difficult to form fusion features with high discriminative power, which affects the accuracy of subsequent lesion identification. To address the aforementioned challenges, this invention first uses the dynamic weight matrix of MRI and the dynamic weight matrix of ultrasound to perform Hadamard products with the redundancy-removed MRI features and the redundancy-removed ultrasound features, respectively. This allows the bimodal features to be dynamically weighted and filtered to highlight the effective features of the lesion area. Then, by taking the absolute value, the difference features are obtained to capture the complementary differences between the bimodal features in the lesion area, such as the subtle differences between MRI texture details and ultrasound edge information. This avoids the common information of the modalities from obscuring the specific features of small lesions, allowing the unique features of small lesions to be highlighted. Secondly, interactive features are obtained through Hadamard product to enhance the effective co-occurrence characteristics of bimodal features in the lesion area, thereby further improving the intensity of key features of the lesion and compensating for the inherent weakness of features in small lesions. Next, the difference features and interaction features are concatenated, and then processed by a convolutional layer to integrate the difference features and co-occurrence features of the two modalities into more discriminative high-order information, avoiding the limitations of a single feature type and allowing the fused features to simultaneously contain modal complementarity and feature co-occurrence. Finally, the convolutional layer processing results are combined with the sum of the redundancy-removed MRI features and the redundancy-removed ultrasound features to perform a Hadamard product, thereby enhancing the basic fusion features with higher-order information. At the same time, differential features are superimposed to form a residual-like protection mechanism, ensuring that the complementary differences of small lesions captured in the differential features are not lost by the higher-order fusion operation. This not only leverages the complementary advantages of bimodal features but also firmly protects the weak features of small lesions from being diluted.

[0013] Furthermore, step S42 also includes: S421: Based on the MRI texture enhancement features and the aligned ultrasound enhancement features, calculate the confidence scores of the MRI texture enhancement features, the confidence scores of the aligned ultrasound enhancement features, the MRI information entropy, and the ultrasound information entropy. The calculation method is as follows: in, Confidence of MRI texture enhancement features The confidence level of the aligned ultrasound enhancement features. For local covariance pooling operation, For MRI information entropy, This is the function for calculating information entropy. The entropy of ultrasonic information; S422: Based on the lesion region weight feature matrix, the confidence level of MRI texture enhancement features, and the confidence level of aligned ultrasound enhancement features, calculate the MRI dynamic weight matrix and the ultrasound dynamic weight matrix. The calculation method is as follows: in, This is the MRI dynamic weight matrix. This is the ultrasonic dynamic weight matrix; S423: Based on the MRI information entropy and ultrasound information entropy, calculate the redundancy-free MRI features and ultrasound features. The calculation method is as follows: in, The redundancy suppression gate coefficient, The MRI features after redundancy removal. These are the ultrasound features after redundancy removal.

[0014] It should be further explained that, regarding MRI texture enhancement features and aligned ultrasound enhancement features, modal differences and regional heterogeneity still exist after spatial alignment. The MRI and ultrasound features of the lesion area each have their advantages (MRI texture is delicate, ultrasound edge real-time performance is good), but the features of the normal area have high redundancy, and the reliability of the bimodal features of different areas is different (MRI features are more reliable in some areas, and ultrasound features are more reliable in some areas). Traditional methods are difficult to dynamically distinguish between lesions and normal areas, and are difficult to assign weights according to feature reliability. At the same time, they cannot effectively suppress redundant information in the normal area, which can easily lead to the features of small lesions being overwhelmed by redundant information or weakened by unreasonable weight allocation. Step S42 of this invention first calculates the confidence scores of MRI texture enhancement features and ultrasound enhancement features through local covariance pooling, measuring the stability of features within each local region (the smaller the covariance, the more stable the feature, and the higher the confidence score), and locating the reliable regions in each of the bimodal features; simultaneously, it calculates the MRI information entropy and ultrasound information entropy, quantifying the redundancy of features through information entropy (the higher the entropy value, the more chaotic and redundant the information), providing a basis for subsequent redundancy removal; Secondly, potential lesion areas are marked based on the weight feature matrix of the lesion area. Combining the confidence of the two types of features, a dynamic weight calculation logic is constructed: in the lesion area (high weight matrix value), features with high confidence are given higher weights first (e.g., if the MRI confidence is high, the MRI dynamic weight is high); in the normal area (low weight matrix value), complementary weights are given to the low-confidence area first (e.g., if the ultrasound confidence is low, the MRI dynamic weight is supplemented). Through this mechanism of "enhancing reliable features in the lesion area and complementing weak features in the normal area", the weight allocation is ensured to be accurately matched with the regional characteristics and feature reliability. Finally, the redundancy suppression gating coefficient is calculated based on information entropy. The gating coefficient of redundant regions with high entropy values ​​is close to 1. By weakening the features of this region through the Hadamard product with the features, the redundant information of normal tissue is suppressed in a targeted manner, while the effective features of the lesion region are preserved. Existing technologies often employ fixed weight allocation (such as equal weights or empirical weights) or adjust weights using only a single indicator (such as feature strength) in the processing steps before dual-modal feature fusion. Furthermore, redundancy removal operations frequently utilize global threshold filtering, lacking specificity: fixed weights cannot adapt to the differences in feature reliability across different regions, resulting in the weakening of effective features in lesion areas and the retention of redundant features in normal areas; adjusting weights using a single indicator struggles to balance lesion localization and feature quality, and global redundancy removal easily leads to the accidental deletion of weak features from small lesions. In contrast, step S42 of this invention combines the lesion region weight matrix to distinguish between lesion and normal regions, utilizing local covariance pooling for evaluation. The system assesses feature reliability and locates redundant regions based on information entropy, ultimately achieving an organic combination of dynamic weight allocation and targeted redundancy removal, rather than simple weight setting or filtering. Its advantage lies in the regional specificity and feature reliability sensitivity of the weight allocation, which can retain the effective features of small lesions while weakening the redundant interference of normal regions. The three dimensions of information—lesion region location (lesion region weight feature matrix), feature reliability assessment (local covariance pooling), and redundant information quantification (information entropy)—are closely related rather than independent. Through the connection and sharing of data flows, the weight allocation avoids redundant regions, enabling the synergistic utilization of multi-dimensional information.

[0015] Furthermore, step S5 also includes: S51: Based on the micro-protective characteristics, the YOLOv8 network is used to identify and select the micro-lesion regions in the preprocessed ultrasound image, and the coordinates of the two-dimensional bounding box of the lesion are output. S52: Convert the two-dimensional bounding box coordinates of the lesion into three-dimensional spatial coordinates, and generate a feasible puncture path through a path planning algorithm; S53: Based on the feasible puncture path, generate motion control instructions for the puncture needle. The motion control instructions include the feed direction, feed speed and feed depth, and send them to the puncture robot actuator.

[0016] Furthermore, the aforementioned method for identifying minute lesions used in prostate biopsy can be applied to assist in precise localization during prostate biopsy surgery, including: Step 1: Acquire the patient's MRI and ultrasound images and perform preprocessing to obtain preprocessed MRI and ultrasound images; Step 2: Based on the preprocessed MRI images, construct a texture attention residual network to extract MRI texture enhancement features; based on the preprocessed ultrasound images, construct a multi-scale edge response gating network to extract ultrasound edge enhancement features. Step 3: Based on the preprocessed MRI image, perform spatial transformation and resampling on the ultrasound edge enhancement features to obtain aligned ultrasound enhancement features; Step 4: Calculate the lesion region weight feature matrix based on MRI texture enhancement features and aligned ultrasound enhancement features; then calculate the MRI dynamic weight matrix, ultrasound dynamic weight matrix, redundancy-removed MRI features, and redundancy-removed ultrasound features; finally, extract the micro-protective features. Step 5: Identify the micro-lesion area based on the micro-protective features and control the puncture robot to perform puncture.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) To address the problems of weak ability to identify small lesions, easy to miss diagnoses, and insufficient puncture accuracy of traditional methods, this invention specifically enhances, precisely aligns, and integrates the features of the two modalities with a small feature protection mode; a texture attention residual network is designed to enhance the texture details of small lesions in MRI images and avoid the loss of texture information caused by traditional single feature extraction; a multi-scale edge response gating network is designed to highlight the edge contour of lesions in ultrasound images and solve the problem of edge blurring under ultrasound artifact interference; then, through spatial transformation and resampling, the dual-modal feature alignment is achieved. Combined with the lesion area weight matrix, dynamic weight allocation, and redundancy suppression, the extracted small protective features can effectively retain the weak features of lesions and weaken the interference of normal tissue, ultimately greatly improving the accuracy of small lesion identification, reducing missed diagnoses, providing a precise positioning basis for the puncture robot, and significantly improving the puncture accuracy.

[0018] (2) To address the problem that the subtle texture differences of small lesions in prostate MRI images are easily lost and difficult to distinguish using traditional methods, this invention designs a texture attention residual network to extract MRI texture enhancement features. First, the residual network layer extracts basic features, then the gray-level co-occurrence matrix is ​​used to quantify local texture differences, and global statistical features extracted by global covariance pooling are combined with splicing, convolution and Softmax to generate texture attention weights, focusing on the lesion area. Finally, the lesion texture is enhanced and the basic features are protected by Hadamard product and residual connection. Compared with the problem of traditional ordinary convolution lacking attention and residual mechanism and easy separation of local and global features, this invention realizes deep integration of multiple features and module collaborative enhancement, effectively preserving the texture information of small lesions and improving the recognition accuracy.

[0019] (3) To address the issues of ultrasound images being susceptible to artifact noise interference, blurred edges of small prostate lesions, and difficulty in capturing edges completely at a single scale, this invention designs a multi-scale edge response gating network to extract ultrasound edge enhancement features. First, multi-scale convolution is used to capture the fine edges of small lesions and the overall contour of large lesions, respectively. Then, the Laplacian operator is used to enhance the edges at each scale. Dynamic scale weight coefficients are generated by Sigmoid and weighted fusion to obtain the multi-scale fused basic ultrasound features. Finally, edge enhancement gating coefficients are generated by mean difference calculation and max pooling, and the lesion edges are amplified a second time by combining the Hadamard product and residual connection. Compared with the problems of lack of dynamic coordination and easy mixing of edges and noise in traditional multi-scale processing, this invention achieves multi-scale feature screening and fusion, accurately enhances effective edges, suppresses interference, breaks through the bottleneck of collaborative enhancement and noise suppression, and ensures complete and accurate edge features.

[0020] (4) To address the modal differences and regional heterogeneity issues that exist after spatial alignment of MRI texture enhancement features and aligned ultrasound enhancement features, this invention constructs a collaborative processing mechanism. It calculates the confidence of dual-modal features through local covariance pooling, locates the reliability of features in each region, and calculates information entropy to quantify the degree of redundancy. The lesion area is marked with a weight feature matrix of the lesion area, and a dynamic weight logic is constructed in combination with the confidence to achieve precise matching of weight with regional characteristics and reliability. Based on information entropy, a gating coefficient is generated to target and suppress redundancy in the normal area. Compared with the limitations of traditional fixed weights and global redundancy removal, this invention shares multi-dimensional information, forms a closed-loop collaboration, ensures the specificity and sensitivity of weights, and efficiently preserves the features of small lesions, laying the foundation for fusion.

[0021] (5) Although the bimodal features after redundancy removal have eliminated some redundancy, the effective feature strength of small lesions is weak. Simple fusion is easily diluted by the averaging of common information. Moreover, if the complementarity between MRI texture details and ultrasound edge information is not accurately preserved, it will cover up the key features of the lesion and affect the distinguishability of the fused features. To address this, the present invention first performs Hadamard product with the corresponding redundancy removal features using a dynamic weight matrix to screen and highlight the effective features of the lesion area. Then, it takes the absolute value of the difference features to capture the complementary differences between the two modes to highlight the uniqueness of the lesion. The interaction features are obtained through Hadamard product to compensate for the insufficient feature strength of the lesion. After splicing the difference and interaction features, they are integrated into a high-order information with both complementarity and co-occurrence through a convolutional layer. Finally, the convolution result is combined with the bimodal redundancy removal features to perform Hadamard product. Combined with the difference features, a residual protection mechanism is constructed to give full play to the advantages of the bimodality and avoid the dilution of the weak features of small lesions. Attached Figure Description

[0022] Figure 1 A flowchart illustrating a method for identifying minute lesions in prostate biopsy provided by the present invention; Figure 2 Registered ultrasound images provided for this invention; Figure 3 The image shows the result of identifying the small lesion area provided by the present invention. Detailed Implementation

[0023] The present invention will be further described below with reference to the accompanying drawings, but this is not intended to limit the present invention in any way. Any modifications or substitutions made based on the teachings of the present invention shall fall within the protection scope of the present invention.

[0024] Example 1: A method for identifying minute lesions in prostate biopsy, such as... Figure 1 As shown, it includes the following steps: S1: Acquire the patient's MRI and ultrasound images and preprocess them to obtain preprocessed MRI and ultrasound images, including: S11: Acquire MRI images of the patient before surgery using magnetic resonance imaging equipment, including T2WI and DWI sequences; acquire ultrasound images of the patient during surgery using ultrasound equipment; apply non-local mean filtering to denoise the MRI images and adaptive median filtering to denoise the ultrasound images to obtain denoised MRI images and denoised ultrasound images. S12: Based on the denoised MRI image and the denoised ultrasound image, the gray values ​​of the images are adjusted using the Z-score normalization method to obtain the gray-scale normalized MRI image and the gray-scale normalized ultrasound image. S13: Based on the grayscale-normalized MRI image and the grayscale-normalized ultrasound image, bicubic interpolation is used to unify the spatial resolution of the two types of images, resulting in preprocessed MRI images and preprocessed ultrasound images.

[0025] S2: Based on the preprocessed MRI images, a texture attention residual network is constructed to extract MRI texture enhancement features; based on the preprocessed ultrasound images, a multi-scale edge response gating network is constructed to extract ultrasound edge enhancement features, including: S21: Based on the preprocessed MRI images, a texture attention residual network is constructed to enhance the texture information of small lesions and extract features to obtain MRI texture enhancement features. The calculation method is as follows: in, As a basic feature of MRI, For residual network layers, The images are preprocessed MRI images. For texture feature matrix, For the calculation of the gray-level co-occurrence matrix, As texture attention weight, For the Softmax function, For splicing operations, It is a convolutional layer. For dimensional expansion operations, For global covariance pooling, For MRI texture enhancement features, For Hadamah accumulation; S22: Based on the preprocessed ultrasound image, a multi-scale edge response gating network is constructed to enhance the edge information of small lesions and extract features to obtain ultrasound edge enhancement features. The calculation method is as follows: in, Here, represents the basic ultrasound features at the s-th scale, where s is the scale index. The image is a preprocessed ultrasound image. For edge features at scale s, For the Laplace operator, Let be the scale weight coefficient for the s-th scale. For the Sigmoid function, To integrate basic ultrasound features across multiple scales, To enhance the gating coefficient at the edge, For max pooling, For average pooling, To take the absolute value, This is an ultrasound edge enhancement feature.

[0026] For example, the residual network layer in step S21 adopts the ResNet-18 structure and contains 4 residual blocks. The parameters of each block are as follows: Input layer: Receives a preprocessed MRI image of size 256×256×1, with a kernel size of 3×3, stride of 1, padding=1, output channel number of 64, and activation function of ReLU; Residual Block 1: Contains 2 convolutional layers, each with a kernel size of 3×3, stride of 1, padding=1, and 64 output channels. Each convolutional layer is followed by a BatchNorm2d block and a ReLU activation function. The residual connections are identity mappings. Residual Block 2: Contains 2 convolutional layers with a kernel size of 3×3, stride of 2, padding=1, and 128 output channels. The residual connections use 1×1 convolutions (stride of 2, 128 output channels) to match the dimension. The other parameters are the same as those of Residual Block 1. Residual Block 3: 256 output channels, 1×1 residual connection, convolution stride of 2, other parameters are the same as Residual Block 2; Residual block 4: 512 output channels, 1×1 residual connection, convolution stride of 2, other parameters are the same as residual block 2; The residual network layer outputs 16×16×512 MRI baseline features.

[0027] The gray-level co-occurrence matrix based on MRI fundamental features has the following parameter settings: Gray-level quantization: quantizes the feature values ​​of basic MRI features into 16 levels of gray; Step size: set to 1 in all four directions: horizontal, vertical, 45°, and 135°. Window size: 5×5, meaning that for each pixel on the basic MRI feature, GLCM is calculated in a 5×5 neighborhood; Output: Each channel outputs a 16×16 GLCM matrix, resulting in a 16×16×512 texture feature matrix T.

[0028] S3: Based on the preprocessed MRI image, spatial transformation and resampling are performed on the ultrasound edge enhancement features to obtain aligned ultrasound enhancement features, including: S31: Using the preprocessed MRI image as a reference template, a multimodal image registration algorithm based on mutual information is employed to map the preprocessed ultrasound image to the MRI coordinate system, such as... Figure 2 As shown, the spatial transformation matrix is ​​then calculated. S32: Based on the spatial transformation matrix, the ultrasound edge enhancement features are mapped to the MRI coordinate system to obtain the registered ultrasound enhancement features; S33: Keep the spatial position of the MRI texture enhancement features in the MRI coordinate system unchanged, and resample the registered ultrasound enhancement features according to the spatial resolution of the MRI texture enhancement features to make the spatial resolution of the two consistent, so as to obtain the aligned ultrasound enhancement features.

[0029] S4: Based on the MRI texture enhancement features and the aligned ultrasound enhancement features, calculate the lesion region weight feature matrix; then calculate the MRI dynamic weight matrix, the ultrasound dynamic weight matrix, the redundancy-removed MRI features, and the redundancy-removed ultrasound features; finally, extract the micro-protective features, including: S41: Calculate the lesion region weighted feature matrix based on MRI texture enhancement features and aligned ultrasound enhancement features. The calculation method is as follows: in, This is the weighted feature matrix of the lesion region. For scaling dot product attention operations, The aligned ultrasound enhancement features; S42: Based on the MRI texture enhancement features, aligned ultrasound enhancement features, and lesion region weight feature matrix, calculate the MRI dynamic weight matrix, ultrasound dynamic weight matrix, redundancy-free MRI features, and redundancy-free ultrasound features, including: S421: Based on the MRI texture enhancement features and the aligned ultrasound enhancement features, calculate the confidence scores of the MRI texture enhancement features, the confidence scores of the aligned ultrasound enhancement features, the MRI information entropy, and the ultrasound information entropy. The calculation method is as follows: in, Confidence of MRI texture enhancement features The confidence level of the aligned ultrasound enhancement features. For local covariance pooling operation, For MRI information entropy, This is the function for calculating information entropy. The entropy of ultrasonic information; S422: Based on the lesion region weight feature matrix, the confidence level of MRI texture enhancement features, and the confidence level of aligned ultrasound enhancement features, calculate the MRI dynamic weight matrix and the ultrasound dynamic weight matrix. The calculation method is as follows: in, This is the MRI dynamic weight matrix. This is the ultrasonic dynamic weight matrix; S423: Based on the MRI information entropy and ultrasound information entropy, calculate the redundancy-free MRI features and ultrasound features. The calculation method is as follows: in, The redundancy suppression gate coefficient, The MRI features after redundancy removal. These are the ultrasound features after redundancy removal.

[0030] S43: Based on the redundancy-removed MRI features, redundancy-removed ultrasound features, and the MRI dynamic weight matrix and ultrasound dynamic weight matrix, extract micro-protective features. The calculation method is as follows: in, For difference features, For interactive features, It is a minor protective feature.

[0031] Specifically, for scenarios where local noise interference exists in MRI texture enhancement features and aligned ultrasound enhancement features, leading to misjudgment of region correlation during scaling dot product attention operations, this invention also provides a method for calculating the lesion region weight feature matrix based on feature differences and Gaussian smoothing, to replace step S41. The calculation method is as follows: in, This is an element-wise absolute difference matrix, used to intuitively quantify the degree of difference in bimodal features within local regions. This is a Gaussian smoothing operation.

[0032] S5: Identify small lesion areas based on micro-protective features and control the puncture robot to perform puncture, including: S51: Based on the characteristics of small protective features, the YOLOv8 network is used to identify and select small lesion regions in the preprocessed ultrasound images, such as... Figure 3 As shown, the coordinates of the two-dimensional bounding box of the lesion are then output; S52: Convert the two-dimensional bounding box coordinates of the lesion into three-dimensional spatial coordinates, and generate a feasible puncture path through a path planning algorithm; S53: Based on the feasible puncture path, generate motion control instructions for the puncture needle. The motion control instructions include the feed direction, feed speed and feed depth, and send them to the puncture robot actuator.

[0033] Example 2: The above-mentioned method for identifying minute lesions in prostate biopsy is applied to assist in precise localization during prostate biopsy surgery, including: Step 1: Acquire the patient's MRI and ultrasound images and perform preprocessing to obtain preprocessed MRI and ultrasound images; Step 2: Based on the preprocessed MRI images, construct a texture attention residual network to extract MRI texture enhancement features; based on the preprocessed ultrasound images, construct a multi-scale edge response gating network to extract ultrasound edge enhancement features. Step 3: Based on the preprocessed MRI image, perform spatial transformation and resampling on the ultrasound edge enhancement features to obtain aligned ultrasound enhancement features; Step 4: Calculate the lesion region weight feature matrix based on MRI texture enhancement features and aligned ultrasound enhancement features; then calculate the MRI dynamic weight matrix, ultrasound dynamic weight matrix, redundancy-removed MRI features, and redundancy-removed ultrasound features; finally, extract the micro-protective features. Step 5: Identify the micro-lesion area based on the micro-protective features and control the puncture robot to perform puncture.

[0034] It should be noted that the sequence numbers of the above embodiments of the present invention are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or method. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0035] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0036] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for identifying minute lesions in prostate biopsy, characterized in that, Includes the following steps: S1: Acquire the patient's MRI and ultrasound images and perform preprocessing to obtain preprocessed MRI and ultrasound images; S2: Based on the preprocessed MRI images, construct a texture attention residual network to extract MRI texture enhancement features; Based on the preprocessed ultrasound images, a multi-scale edge response gating network is constructed to extract ultrasound edge enhancement features; The texture attention residual network extracts basic MRI features through residual network layers, calculates texture feature matrix using gray-level co-occurrence matrix, and combines the features extracted by global covariance pooling with dimension expansion, convolution and splicing operations. Then, it generates texture attention weights through Softmax function, and strengthens the texture information of small lesions through Hadamard product and residual connection, thereby extracting MRI texture enhancement features. The multi-scale edge response gating network calculates multi-scale fused basic ultrasound features through multi-scale convolution, Laplacian operator and Sigmoid function, and then calculates edge enhancement gating coefficients by combining max pooling and average pooling. The edge information of small lesions is enhanced by Hadamard product and residual connection, and then ultrasound edge enhancement features are extracted. S3: Based on the preprocessed MRI image, the ultrasound edge enhancement features are spatially transformed and resampled to obtain aligned ultrasound enhancement features; S4: Calculate the lesion region weight feature matrix based on MRI texture enhancement features and aligned ultrasound enhancement features; then calculate the MRI dynamic weight matrix, ultrasound dynamic weight matrix, redundancy-free MRI features, and redundancy-free ultrasound features; finally, extract the micro-protective features. S5: Identify small lesion areas based on micro-protective features and control the puncture robot to perform puncture.

2. The method for identifying minute lesions in prostate biopsy according to claim 1, characterized in that, Step S1 includes: S11: Acquire MRI images of the patient before surgery using magnetic resonance imaging equipment, including T2WI and DWI sequences; acquire ultrasound images of the patient during surgery using ultrasound equipment; apply non-local mean filtering to denoise the MRI images and adaptive median filtering to denoise the ultrasound images to obtain denoised MRI images and denoised ultrasound images. S12: Based on the denoised MRI image and the denoised ultrasound image, the gray values ​​of the images are adjusted using the Z-score normalization method to obtain the gray-scale normalized MRI image and the gray-scale normalized ultrasound image. S13: Based on the grayscale-normalized MRI image and the grayscale-normalized ultrasound image, bicubic interpolation is used to unify the spatial resolution of the two types of images, resulting in preprocessed MRI images and preprocessed ultrasound images.

3. The method for identifying minute lesions in prostate biopsy according to claim 1, characterized in that, Step S2 includes: S21: Based on the preprocessed MRI images, a texture attention residual network is constructed to enhance the texture information of small lesions and extract features to obtain MRI texture enhancement features. The calculation method is as follows: in, As a basic feature of MRI, For residual network layers, The images are preprocessed MRI images. For texture feature matrix, For the calculation of the gray-level co-occurrence matrix, As texture attention weight, For the Softmax function, For splicing operations, It is a convolutional layer. For dimensional expansion operations, For global covariance pooling, For MRI texture enhancement features, For Hadamah accumulation; S22: Based on the preprocessed ultrasound image, a multi-scale edge response gating network is constructed to enhance the edge information of small lesions and extract features to obtain ultrasound edge enhancement features. The calculation method is as follows: in, Here, represents the basic ultrasound features at the s-th scale, where s is the scale index. The image is a preprocessed ultrasound image. For edge features at scale s, For the Laplace operator, Let be the scale weight coefficient for the s-th scale. For the Sigmoid function, To integrate basic ultrasound features across multiple scales, To enhance the gating coefficient at the edge, For max pooling, For average pooling, To take the absolute value, This is an ultrasound edge enhancement feature.

4. The method for identifying minute lesions in prostate biopsy according to claim 3, characterized in that, Step S3 includes: S31: Using the preprocessed MRI image as the reference template, a multimodal image registration algorithm based on mutual information is used to map the preprocessed ultrasound image to the MRI coordinate system and calculate the spatial transformation matrix. S32: Based on the spatial transformation matrix, the ultrasound edge enhancement features are mapped to the MRI coordinate system to obtain the registered ultrasound enhancement features; S33: Keep the spatial position of the MRI texture enhancement features in the MRI coordinate system unchanged, and resample the registered ultrasound enhancement features according to the spatial resolution of the MRI texture enhancement features to make the spatial resolution of the two consistent, so as to obtain the aligned ultrasound enhancement features.

5. The method for identifying minute lesions in prostate biopsy according to claim 3, characterized in that, Step S4 includes: Based on the redundancy-removed MRI features, the redundancy-removed ultrasound features, and the MRI dynamic weight matrix and ultrasound dynamic weight matrix, the differential features and interaction features are first calculated. Then, the differential features and interaction features are concatenated and processed by convolution. The convolution result is then combined with the sum of the redundancy-removed MRI features and the redundancy-removed ultrasound features by Hadamard product operation. Finally, the differential features are added to extract the micro-protective features.

6. The method for identifying minute lesions in prostate biopsy according to claim 5, characterized in that, Step S4 includes: S41: Calculate the lesion region weighted feature matrix based on MRI texture enhancement features and aligned ultrasound enhancement features. The calculation method is as follows: in, This is the weighted feature matrix of the lesion region. For scaling dot product attention operations, The aligned ultrasound enhancement features; S42: Calculate the MRI dynamic weight matrix, ultrasound dynamic weight matrix, redundancy-free MRI features, and redundancy-free ultrasound features based on MRI texture enhancement features, aligned ultrasound enhancement features, and lesion region weight feature matrix. S43: Based on the redundancy-removed MRI features, redundancy-removed ultrasound features, and the MRI dynamic weight matrix and ultrasound dynamic weight matrix, extract micro-protective features. The calculation method is as follows: in, For difference features, For interactive features, It is a minor protective feature.

7. The method for identifying minute lesions in prostate biopsy according to claim 6, characterized in that, Step S42 includes: S421: Based on the MRI texture enhancement features and the aligned ultrasound enhancement features, calculate the confidence scores of the MRI texture enhancement features, the confidence scores of the aligned ultrasound enhancement features, as well as the MRI information entropy and the ultrasound information entropy. The calculation method is as follows: in, Confidence of MRI texture enhancement features The confidence level of the aligned ultrasound enhancement features. For local covariance pooling operation, For MRI information entropy, This is the function for calculating information entropy. The entropy of ultrasonic information; S422: Based on the lesion region weight feature matrix, the confidence level of MRI texture enhancement features, and the confidence level of aligned ultrasound enhancement features, calculate the MRI dynamic weight matrix and the ultrasound dynamic weight matrix. The calculation method is as follows: in, This is the MRI dynamic weight matrix. This is the ultrasonic dynamic weight matrix; S423: Based on the MRI information entropy and ultrasound information entropy, calculate the redundancy-free MRI features and ultrasound features. The calculation method is as follows: in, The redundancy suppression gate coefficient, The MRI features after redundancy removal. These are the ultrasound features after redundancy removal.

8. The method for identifying minute lesions in prostate biopsy according to claim 6, characterized in that, Step S5 includes: S51: Based on the micro-protective characteristics, the YOLOv8 network is used to identify and select the micro-lesion regions in the preprocessed ultrasound image, and the coordinates of the two-dimensional bounding box of the lesion are output. S52: Convert the two-dimensional bounding box coordinates of the lesion into three-dimensional spatial coordinates, and generate a feasible puncture path through a path planning algorithm; S53: Based on the feasible puncture path, generate motion control instructions for the puncture needle, the motion control instructions including the feed direction, feed speed and feed depth, and send them to the puncture robot actuator.

9. The application of the method according to any one of claims 1-8 in precise localization during prostate biopsy, characterized in that, include: Step 1: Acquire the patient's MRI and ultrasound images and perform preprocessing to obtain preprocessed MRI and ultrasound images; Step 2: Based on the preprocessed MRI images, construct a texture attention residual network to extract MRI texture enhancement features; Based on the preprocessed ultrasound images, a multi-scale edge response gating network is constructed to extract ultrasound edge enhancement features; Step 3: Based on the preprocessed MRI image, perform spatial transformation and resampling on the ultrasound edge enhancement features to obtain aligned ultrasound enhancement features; Step 4: Calculate the lesion region weight feature matrix based on MRI texture enhancement features and aligned ultrasound enhancement features; then calculate the MRI dynamic weight matrix, ultrasound dynamic weight matrix, redundancy-removed MRI features, and redundancy-removed ultrasound features; finally, extract the micro-protective features. Step 5: Identify the micro-lesion area based on the micro-protective features and control the puncture robot to perform the puncture; The application, by executing steps 1 to 5, utilizes the generated two-dimensional bounding box coordinates of the lesion and the motion control commands of the puncture needle to provide precise puncture positioning assistance for prostate puncture surgery.

Images