A method and system for planning a breast tumor puncture path

Abstract

The application discloses a breast tumor puncture path planning method and system. First, a target image is collected, and region segmentation is performed on the target image to obtain a breast contour image; then, three-dimensional reconstruction is performed on the breast contour image to obtain a breast three-dimensional image; the breast three-dimensional image is input into a pre-constructed hybrid adaptive attention model to extract a breast tumor lesion area; then, the breast area is modeled as a weighted graph, a tumor lesion target node and a starting node are taken as two endpoints, and a puncture path is planned by comparing distance values between adjacent nodes, so that accurate positioning of the breast tumor and accurate planning of the puncture path are realized.

Description

Technical Field

[0001] This invention relates to the field of breast tumor biopsy technology, and more particularly to a method and system for planning breast tumor biopsy pathways. Background Technology

[0002] Breast cancer is the most common malignant tumor worldwide. In 2018, there were approximately 2.1 million cases of breast cancer globally, accounting for 11.6% of all cancers; and approximately 630,000 deaths from breast cancer, accounting for 6.6% of all cancers. In 2014, breast cancer accounted for 16.51% of all female malignant tumor cases, ranking first among female malignant tumors. By 2021, the number of newly diagnosed breast cancer cases in China had reached 420,000. In recent years, the incidence of breast cancer among women in my country has become increasingly serious, showing a trend of earlier peak age of onset. Therefore, it is essential to utilize new technologies to detect breast tumors as early as possible.

[0003] With the continuous improvement of living standards, people expect to detect breast tumors earlier and have higher requirements for the effectiveness of breast tumor surgery. In recent years, with the deepening research in tumor detection and treatment, percutaneous breast biopsy has become an extremely important method for treating breast tumors. Image-guided biopsy plays an increasingly important role in tumor surgical treatment. The accuracy of the puncture path is a key factor affecting the outcome of breast tumor surgical treatment.

[0004] Therefore, there is a need to develop a method that can accurately locate breast tumors and plan puncture routes. Summary of the Invention

[0005] This invention provides a method and system for planning puncture paths for breast tumors, which enables precise localization of breast tumors and planning of puncture paths.

[0006] This invention provides a method for planning a puncture path for breast tumors, comprising:

[0007] Acquire target images;

[0008] The target image is segmented to obtain a breast contour image;

[0009] A three-dimensional image of the breast is obtained by reconstructing the breast contour image.

[0010] The three-dimensional image of the breast is input into a pre-built hybrid adaptive attention model to extract the breast tumor lesion area;

[0011] The breast region is modeled as a weighted graph, where each region is a node, the connection path between regions is an edge, and each edge has a weight representing the difficulty or distance of puncture from one region to another. The tumor lesion region is the target node.

[0012] Using the target node and the starting node as the two endpoints, the puncture path is planned by comparing the distance values ​​between adjacent nodes.

[0013] Specifically, after acquiring the target image, the process also includes:

[0014] Histogram equalization enhancement and homomorphic filtering enhancement are performed on the target image respectively to obtain two enhanced breast ultrasound images that are spatially and temporally correlated and complementary in information.

[0015] Gaussian pyramid decomposition was performed on the enhanced breast ultrasound images that possessed spatiotemporal correlation and information complementarity to obtain images of different resolutions;

[0016] Subtracting the images of different resolutions yields a series of difference images;

[0017] The difference images are fused using a weighted average to obtain a series of new pyramid images;

[0018] The series of new pyramid images are superimposed layer by layer to obtain the final merged image.

[0019] Specifically, the step of performing region segmentation on the target image to obtain a breast contour image includes:

[0020] Create an empty image with the same size as the target image;

[0021] Perform pixel traversal on the empty image based on the target image;

[0022] Using the formula grad B (X)=δ B (X)-β B (X) Calculate the morphological gradient grad of the corresponding region in the empty image. B (X); where δ B (X) is the image after dilation of the target image, β B (X) is the image after the target image has undergone an erosion operation;

[0023] The morphological gradient grad B (X) is compared with a preset threshold;

[0024] If the morphological gradient grad BIf (X) is greater than the preset threshold, corresponding pixel values ​​are generated in the corresponding area of ​​the empty image until the entire breast contour is generated.

[0025] Specifically, the step of performing three-dimensional reconstruction from the breast contour image to obtain a three-dimensional breast image includes:

[0026] A series of breast contour images are superimposed on a pre-created image stack to form a three-dimensional volume to be explored;

[0027] The three-dimensional volume is meshed to obtain multiple cubic units;

[0028] The attribute values ​​of the vertices of each cube unit are compared with preset attribute thresholds to determine the state of each vertex;

[0029] The corresponding triangle structure is obtained by querying a predefined triangle lookup table based on the combination of the states of each vertex.

[0030] The triangles generated inside each cube unit are interpolated according to the state positions of their vertices to obtain accurate triangles;

[0031] Connect all the precise triangles to form the final three-dimensional image of the breast.

[0032] Specifically, the step of inputting the three-dimensional image of the breast into a pre-constructed hybrid adaptive attention model to extract the breast tumor lesion region includes:

[0033] The breast 3D image is processed in parallel using three convolutional layers with different kernel sizes in the hybrid adaptive attention model to obtain three feature maps with different receptive fields:

[0034] F 3 =W 3×3 ×F input

[0035] F 5 =W 5×5 ·F input

[0036]

[0037] Among them, F input It is the three-dimensional image of the breast, W 3×3 W 5×5 , There are three convolution kernels, F 3 F 5 F D These are three feature maps with different receptive fields;

[0038] The feature map and A new feature map is obtained by using global average pooling.

[0039] For the feature map F G Perform Sigmoid activation to obtain F D Channel attention map α and F 5 Channel attention map α ′ =1-α;

[0040] pass F were obtained respectively D and F 5 Calibrated feature map and

[0041] pass For feature map F respectively 3 , and Perform a convolution operation to obtain the convolutional feature map F. S1 and

[0042] pass For the feature map F S1 and Perform convolution operations and activation using the sigmoid function to obtain the spatial attention map β; where σ is the sigmoid activation function. r It is the ReLU activation function;

[0043] pass Obtained respectively and F S1 Calibrated feature map and F S1′ Where, β′=1-β;

[0044] pass The output F of the entire breast tumor lesion region of the hybrid adaptive attention model is obtained. out .

[0045] The present invention also provides a system for planning the puncture path for breast tumors, comprising:

[0046] The image acquisition module is used to acquire target images;

[0047] The region segmentation module is used to segment the target image into regions to obtain a breast contour image;

[0048] A 3D reconstruction module is used to perform 3D reconstruction based on the breast contour image to obtain a 3D image of the breast.

[0049] The lesion area extraction module is used to input the three-dimensional image of the breast into a pre-built hybrid adaptive attention model to extract the lesion area of ​​the breast tumor;

[0050] The weighted graph construction module is used to model the breast region as a weighted graph, where each region is a node, the connection path between regions is an edge, and each edge has a weight, representing the difficulty or distance of puncture from one region to another. The tumor lesion region is the target node.

[0051] The puncture path planning module is used to plan the puncture path by comparing the distance values ​​between adjacent nodes, with the target node and the starting node as the two endpoints.

[0052] Specifically, it also includes:

[0053] The image enhancement module is used to perform histogram equalization enhancement and homomorphic filtering enhancement on the target image respectively, to obtain two enhanced breast ultrasound images that are spatially and temporally correlated and complementary in information.

[0054] The Gaussian pyramid decomposition module is used to perform Gaussian pyramid decomposition on the enhanced breast ultrasound images that have spatiotemporal correlation and information complementarity to obtain images of different resolutions.

[0055] The image subtraction module is used to subtract the images of different resolutions to obtain a series of difference images;

[0056] An image fusion module is used to fuse the difference images using a weighted average to obtain a series of new pyramid images;

[0057] The image overlay module is used to overlay the series of new pyramid images layer by layer to obtain the final fused image.

[0058] Specifically, the region segmentation module includes:

[0059] An empty image creation unit is used to create an empty image with the same size as the target image;

[0060] A pixel traversal unit is used to perform pixel traversal on the empty image based on the target image;

[0061] Morphological gradient calculation unit, used to calculate gradient using the formula grad B (X)=δ B (X)-β B (X) Calculate the morphological gradient grad of the corresponding region in the empty image. B (X); where δ B(X) is the image after dilation of the target image, β B (X) is the image after the target image has undergone an erosion operation;

[0062] A morphological gradient comparison unit is used to compare the morphological gradient grad B (X) is compared with a preset threshold;

[0063] The breast contour image creation unit is used to create a breast contour image if the morphological gradient grad B If (X) is greater than the preset threshold, corresponding pixel values ​​are generated in the corresponding area of ​​the empty image until the entire breast contour is generated.

[0064] Specifically, the three-dimensional reconstruction module includes:

[0065] An image stacking unit is used to superimpose a series of breast contour images on a pre-created image stack to form a three-dimensional volume to be explored;

[0066] Mesh division unit, used to divide the three-dimensional volume into meshes to obtain multiple cubic units;

[0067] A vertex state determination unit is used to compare the attribute values ​​of the vertices of each cube unit with a preset attribute threshold to determine the state of each vertex.

[0068] The lookup unit is used to query a predefined triangle lookup table based on the combination of the states of each vertex to obtain the corresponding triangle structure.

[0069] An interpolation unit is used to interpolate the triangular structure generated inside each cube unit according to the state position of its vertices to obtain an accurate triangle;

[0070] A 3D image construction unit is used to connect all the precise triangles to form the final 3D image of the breast.

[0071] Specifically, the lesion area extraction module includes:

[0072] The parallel processing unit is used to perform parallel processing on the three-dimensional breast image through three convolutional layers with different kernel sizes in the hybrid adaptive attention model, resulting in three feature maps with different receptive fields:

[0073] F 3 =W 3×3 ×F input

[0074] F 5 =W 5×5 ·F input

[0075]

[0076] Among them, F input It is the three-dimensional image of the breast, W 3×3 W 5×5 , There are three convolution kernels, F 3 F 5 F D These are three feature maps with different receptive fields;

[0077] A global average pooling unit is used to process the feature map. and A new feature map is obtained by using global average pooling.

[0078] Feature map activation unit, used for activating the feature map F G Perform Sigmoid activation to obtain F D Channel attention map α and F 5 Channel attention map α ′ =1-α;

[0079] The first feature map calibration unit is used to... F were obtained respectively D and F 5 Calibrated feature map and

[0080] Feature map convolutional units are used to pass through For feature map F respectively 3 , and Perform a convolution operation to obtain the convolutional feature map F. S1 and

[0081] Spatial attention map acquisition unit, used to obtain spatial attention maps through For the feature map F S1 and Perform convolution operations and activation using the sigmoid function to obtain the spatial attention map β; where σ is the sigmoid activation function. r It is the ReLU activation function;

[0082] The second feature map calibration unit is used to... Obtained respectively and F S1 Calibrated feature map and F S1′ Where, β′=1-β;

[0083] Lesion area acquisition unit, used to obtain lesion area The output F of the entire breast tumor lesion region of the hybrid adaptive attention model is obtained. out .

[0084] One or more technical solutions provided in this invention have at least the following technical effects or advantages:

[0085] First, the target image is acquired and segmented to obtain the breast contour image. Then, the breast contour image is reconstructed in three dimensions to obtain a three-dimensional breast image. The three-dimensional breast image is input into a pre-built hybrid adaptive attention model to extract the breast tumor lesion area. Next, the breast region is modeled as a weighted graph, with the tumor lesion target node and the starting node as the two endpoints. The puncture path is planned by comparing the distance values ​​between adjacent nodes, thus achieving precise localization of the breast tumor and precise planning of the puncture path.

[0086] Furthermore, the present invention also has the following advantages:

[0087] 1. Homomorphic filtering algorithm is used to weaken speckle noise in breast ultrasound images. Histogram equalization algorithm is used to compensate for the information entropy loss during the denoising process of homomorphic filtering algorithm. Then, the two enhanced breast ultrasound images with spatiotemporal correlation and information complementarity are fused in layers. This preserves the details in the images that are useful for subsequent analysis and diagnosis, resulting in better images and improving the accuracy of image construction, which in turn improves the accuracy of puncture path planning.

[0088] 2. Compared to traditional convolutional operations, the Hybrid Adaptive Attention (HAAM) module in this invention can capture more features across different receptive fields. Furthermore, the HAAM module can guide the network to adaptively select more robust representations in both channel and spatial dimensions.

[0089] 3. Based on data-driven and nonlinear autoregressive dynamic neural network algorithms, precise and automated excision of breast tumor lesions is achieved, enabling standardized screening and treatment of breast cancer. Attached Figure Description

[0090] Figure 1 A flowchart illustrating a method for planning a breast tumor biopsy path according to an embodiment of the present invention;

[0091] Figure 2 This diagram illustrates the working principle of the hybrid adaptive attention model in the breast tumor puncture path planning method provided in this embodiment of the invention.

[0092] Figure 3 This is a block diagram of a breast tumor puncture path planning system provided in an embodiment of the present invention. Detailed Implementation

[0093] This invention provides a method and system for planning puncture paths for breast tumors, which can accurately locate breast tumors and plan puncture paths.

[0094] The technical solutions in the embodiments of the present invention are designed to achieve the above-mentioned technical effects, and the overall concept is as follows:

[0095] The target image is acquired; then, morphological gradients are extracted from the image to emphasize the image contour and eliminate redundant pixels remaining after the thresholding stage. An iterator is then applied to the resulting image to select morphological gradients that meet the threshold, extracting the desired breast contour. Based on the extracted breast contour segmentation, a 3D image is constructed. An image stack is created, consisting of a series of ultrasound images. These ultrasound images are naturally formed in the order they were acquired. These ultrasound images are superimposed on each other in the stack to form a 3D volume to be explored. Each ultrasound image in the stack is 1 mm thick. The region of interest is extracted from the 3D volume. These isosurfaces can be viewed as dividing the volume into several small cubes (voxels), forming a 3D mesh. For each voxel in the 3D mesh, it is determined whether its vertices are within the extracted geometry. This determines which voxels belong to the breast contour region. Using the vertex information of these voxels, 3D reconstruction can be performed, i.e., a 3D model with the breast contour can be generated. Based on the constructed 3D ultrasound images, a novel Hybrid Adaptive Attention Model (HAAM) is used to enhance the deep learning model's attention to breast lesion regions in complex environments. An Adaptive Attention U-Net (AAU-net) is constructed using HAAM for lesion segmentation in breast ultrasound images. The Hybrid Adaptive Attention Model (HAAM) module mainly consists of convolutional layers with different kernel sizes, channel attention blocks, and spatial attention blocks. Channel attention blocks guide the segmentation network to select more representative breast lesion features. The channel attention mechanism focuses on the category of breast image features, while the spatial attention mechanism focuses on the location of breast lesion features. Then, by constructing a weight map, the tumor lesion region is used as the target node; the target node and the starting node are used as endpoints, and the puncture path is planned by comparing the distance values ​​between adjacent nodes.

[0096] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0097] See Figure 1 The method for planning a breast tumor biopsy path provided in this embodiment of the invention includes:

[0098] Step S110: Acquire the target image;

[0099] Specifically, a SIEMENS ACUSON S2000 ultrasound machine with a probe frequency of 4–9 MHz was used. The patient lay supine with both arms raised and abducted to fully expose the breast and axilla. Multi-sectional scanning of the breast was performed, and multiple images of the lesion site were captured.

[0100] To preserve details in the image that are useful for later analysis and diagnosis, and to obtain better images, the following steps are also taken after acquiring the target image:

[0101] Histogram equalization and homomorphic filtering enhancement were performed on the target image respectively to obtain two enhanced breast ultrasound images that are spatially and temporally correlated and complementary in information.

[0102] Specifically, histogram equalization is performed on breast ultrasound images to enhance them. This involves non-linear stretching of the images to redistribute the number of pixels across different gray levels, ensuring that the number of pixels within a given gray range is approximately equal. This reduces the number of pixels in areas with a high number of pixels and increases the number of pixels in areas with a low number of pixels; that is, the peaks in the gray-level histogram decrease, while the valleys on either side increase. The histogram of the equalized breast ultrasound image is flatter, enhancing image contrast and increasing information entropy. The functional expression for histogram equalization of breast ultrasound images is:

[0103]

[0104] Among them, S i It is the cumulative pixel percentage of grayscale i, k is the grayscale level, and n is the number of grayscale levels. i is the number of pixels in the breast ultrasound image that reproduce grayscale i, and n is the total number of pixels in the breast ultrasound image.

[0105] Furthermore, homomorphic filtering in the frequency domain is used to enhance the image. This enhances details in dark areas without sacrificing details in bright areas. The breast ultrasound image f(x,y) is modeled as the product of illumination intensity i(x,y) and reflection intensity r(x,y), therefore:

[0106] f(x,y)=i(x,y)r(x,y)

[0107] Gaussian pyramid decomposition was performed on the enhanced breast ultrasound images that possessed spatiotemporal correlation and information complementarity to obtain images at different resolutions;

[0108] Subtracting images of different resolutions yields a series of difference images;

[0109] By using a weighted average to fuse the difference images, a series of new pyramid images are obtained;

[0110] A series of new pyramid images are superimposed layer by layer to obtain the final merged image.

[0111] Step S120: Perform region segmentation on the target image to obtain a breast contour image;

[0112] This step is explained in detail: the target image is segmented to obtain a breast contour image, including:

[0113] Create an empty image with a black background and the same dimensions as the target image, where all pixels have a value of 0.

[0114] The empty image is traversed pixel by pixel based on the target image; specifically, the pixels traverse the image in five directions: from right to left, from left to right, from bottom to top, along the left diagonal, and along the right diagonal.

[0115] Using the formula grad B (X)=δ B (X)-β B (X) Calculate the morphological gradient grad of the corresponding region in the empty image. B (X); where δ B (X) is the image after dilation of the target image, β B (X) is the image after the target image has undergone erosion;

[0116] morphological gradient grad B (X) is compared with a preset threshold;

[0117] If the morphological gradient grad B If (X) is greater than the preset threshold, it means that pixel X belongs to the breast contour region. The corresponding pixel value is generated in the corresponding region in the empty image until the entire breast contour is generated.

[0118] Step S130: Perform three-dimensional reconstruction using the breast contour image to obtain a three-dimensional image of the breast;

[0119] This step is explained in detail. A three-dimensional reconstruction is performed using the breast contour image to obtain a three-dimensional breast image, including:

[0120] A series of breast contour images are superimposed on a pre-created image stack to form a three-dimensional volume to be explored;

[0121] The three-dimensional volume is meshed to obtain multiple cubic elements;

[0122] The attribute values ​​of each vertex in the cube are compared with preset attribute thresholds to determine the state of each vertex.

[0123] Specifically, if the attribute value of a vertex is greater than a preset attribute threshold, it indicates that the vertex is outside the breast surface; otherwise, it indicates that the vertex is inside the breast surface. In this embodiment, an 8-bit binary number (0 or 1) is used to represent the state of each vertex, where 0 indicates that it is outside the breast surface and 1 indicates that it is inside the breast surface.

[0124] The corresponding triangle structure is obtained by querying a predefined triangle lookup table based on the combination of the states of each vertex.

[0125] The triangles generated inside each cube unit are interpolated according to the state positions of their vertices to obtain accurate triangles;

[0126] Connect all the precise triangles to form the final three-dimensional image of the breast.

[0127] Step S140: Input the 3D image of the breast into a pre-built hybrid adaptive attention model to extract the breast tumor lesion area; specifically, the working principle of the hybrid adaptive attention model is as follows: Figure 2 As shown.

[0128] This step involves inputting a 3D image of the breast into a pre-built hybrid adaptive attention model to extract the breast tumor lesion region, including:

[0129] By performing parallel processing on the 3D breast image using three convolutional layers with different kernel sizes in a hybrid adaptive attention model, three feature maps with different receptive fields are obtained:

[0130] F 3 =W 3×3 ×F input

[0131] F 5 =W 5×5 ·F input

[0132]

[0133] Among them, F input It is a 3D image of the breast, W 3×3 W 5×5 , There are three convolution kernels, F 3 F 5 F D These are three feature maps with different receptive fields;

[0134] feature map and A new feature map is obtained by using global average pooling. Right now

[0135] For feature map F G Perform Sigmoid activation to obtain F D Channel attention map α and F 5 Channel attention map α ′ =1-α;

[0136] pass F were obtained respectively D and F 5 Calibrated feature map and in, It is F D The calibrated feature map It is F 5 The calibrated feature map.

[0137] To further improve the robustness of network representation features, a novel spatial self-attention block is used. (By F...) 3 , and The channel self-attention block serves as the input to the spatial self-attention block. Specifically, through...

[0138] For feature map F respectively 3 , and Perform a convolution operation to obtain the convolutional feature map F. S1 and

[0139] It should be noted that, in order to refine the target's location information, a 1×1 convolution operation is first performed on the input feature map, i.e.

[0140] F S1 =W 1×1 ·F 3

[0141]

[0142] Among them, F S1 It is for F 3 The feature map after 1x1 convolution. Yes and The feature map is obtained by performing a 1x1 convolution after summing.

[0143] pass For feature map F S1 and Perform convolution operations and activation using the sigmoid function to obtain the spatial attention map β; where σ is the sigmoid activation function. rIt is the ReLU activation function;

[0144] pass Obtained respectively and F S1 Calibrated feature map and F S1′ Where, β′=1-β;

[0145] pass The output F of the entire hybrid adaptive attention model for the breast tumor lesion region is obtained. out .

[0146] Step S150: Model the breast region as a weighted graph, where each region is a node, the connection path between regions is an edge, and each edge has a weight, representing the difficulty or distance of puncture from one region to another. The tumor lesion region is the target node.

[0147] Step S160: Using the target node and the starting node as the two endpoints, the puncture path is planned by comparing the distance values ​​between adjacent nodes.

[0148] This step is explained in detail. Using the target node and the starting node as endpoints, the puncture path is planned by comparing the distances between adjacent nodes, including:

[0149] Starting from the starting node, for each node, calculate the distance from that node to all its neighboring nodes and update the distance values ​​and predecessor nodes. Select the unvisited node with the smallest current distance as the next node to operate on. If the path from the current node to a neighboring node is smaller than the original distance value of that node, update the distance value of that node. At the same time, mark the current node as visited, indicating that the shortest path has been determined. Starting from the target node, backtrack along the predecessor nodes of each node until the starting node is reached to obtain the shortest path, i.e., the puncture path.

[0150] It should be noted that after generating the puncture path, the lesion is also subjected to precise automated rotary cutting. The specific process includes:

[0151] The controller first determines the position of the ultrasound probe in the XY plane, then calculates the coordinates of the puncture needle tip in a fixed coordinate system, and enhances these coordinates using a Kalman filter to reduce noise and predict subsequent states. A compensator is added on the X-axis to ensure the ultrasound probe always maintains zero distance from the puncture needle tip. This servo control of the ultrasound probe enables real-time tracking of the puncture needle tip.

[0152] After real-time tracking and observation of the puncture needle by an ultrasound probe, the probe provides positioning information. The puncture needle then performs puncture according to the generated puncture path. A data-driven, nonlinear autoregressive (NARX) dynamic neural network algorithm with external input controls the insertion depth and number of cuts of the blade to remove diseased tissue. A computer-controlled vacuum-assisted high-speed rotary cutting device utilizes the principle of vacuum negative pressure suction to perform minimally invasive cutting of breast tissue, achieving precise and automated rotary cutting of the target area (i.e., tumor / malignant tissue).

[0153] y(t)=f[y(t-1),y(t-2),…,y(t-ny),

[0154] x(t-1),x(t-2),…,x(t-nx)]

[0155] In the formula: f(·) represents the nonlinear process function implemented using a neural network. It extends along the time axis of the data. This formula represents the data correlation modeling idea of ​​using a neural network to simulate functions in time series data.

[0156] See Figure 3 The breast tumor biopsy path planning system provided in this embodiment of the invention includes:

[0157] Image acquisition module 100 is used to acquire target images;

[0158] Specifically, the image acquisition module 100 is used with a SIEMENS ACUSON S2000 ultrasound instrument, with a probe frequency of 4–9 MHz. The patient lies supine with both arms raised and abducted to fully expose the breast and axilla. Multiple-section scans of the breast are performed, and multiple images of the lesion site are captured.

[0159] To preserve details in the image that are useful for later analysis and diagnosis, and to obtain better images, the following measures are also included:

[0160] The image enhancement module is used to perform histogram equalization enhancement and homomorphic filtering enhancement on the target image respectively, to obtain two enhanced breast ultrasound images that are spatially and temporally correlated and complementary in information.

[0161] The Gaussian pyramid decomposition module is used to perform Gaussian pyramid decomposition on the enhanced breast ultrasound images that have spatiotemporal correlation and information complementarity, respectively, to obtain images of different resolutions;

[0162] The image subtraction module subtracts images of different resolutions to obtain a series of difference images;

[0163] The image fusion module is used to fuse the difference images using a weighted average to obtain a series of new pyramid images;

[0164] The image overlay module is used to overlay a series of new pyramid images layer by layer to obtain the final merged image.

[0165] The region segmentation module 200 is used to segment the target image into regions to obtain a breast contour image;

[0166] Specifically, the region segmentation module 200 includes:

[0167] The Empty Image Creation Unit is used to create an empty image with a black background that has the same size as the target image. All pixels in the empty image have a value of 0.

[0168] The pixel traversal unit is used to perform pixel traversal on the empty image based on the target image; specifically, the pixels traverse the image in five directions: from right to left, from left to right, from bottom to top, along the left diagonal, and along the right diagonal.

[0169] Morphological gradient calculation unit, used to calculate gradient using the formula grad B (X)=δ B (X)-β B (X) Calculate the morphological gradient grad of the corresponding region in the empty image. B (X); where δ B (X) is the image after dilation of the target image, β B (X) is the image after the target image has undergone erosion;

[0170] The morphological gradient comparison unit is used to compare the morphological gradient with the gradient. B (X) is compared with a preset threshold;

[0171] Breast contour image creation unit, used for if morphological gradient gradient grad B If (X) is greater than the preset threshold, it means that pixel X belongs to the breast contour region. The corresponding pixel value is generated in the corresponding region in the empty image until the entire breast contour is generated.

[0172] The 3D reconstruction module 300 is used to perform 3D reconstruction from the breast contour image to obtain a 3D image of the breast.

[0173] Specifically, the 3D reconstruction module 300 includes:

[0174] Image stacking unit, used to superimpose a series of breast contour images on a pre-created image stack to form a three-dimensional volume to be explored;

[0175] Mesh generation unit, used to divide a three-dimensional volume into meshes, resulting in multiple cubic units;

[0176] The vertex state determination unit is used to compare the attribute values ​​of the vertices of each cube unit with preset attribute thresholds to determine the state of each vertex.

[0177] Specifically, the vertex state determination unit is used to compare the attribute value of each vertex of the cube unit with a preset attribute threshold. If the attribute value of the vertex is greater than the preset attribute threshold, it means that the vertex is outside the breast surface; otherwise, it means that the vertex is inside the breast surface. In this embodiment, an 8-bit binary number (0 or 1) is used to represent the state of each vertex, where 0 indicates that it is outside the breast surface and 1 indicates that it is inside the breast surface.

[0178] The lookup unit is used to query a predefined triangle lookup table based on the combination of the states of each vertex to obtain the corresponding triangle structure.

[0179] Interpolation units are used to interpolate the triangular structures generated inside each cube unit according to the state positions of its vertices to obtain accurate triangles;

[0180] The 3D image building unit is used to connect all the precise triangles to form the final 3D image of the breast.

[0181] The lesion region extraction module 400 is used to input the three-dimensional image of the breast into a pre-built hybrid adaptive attention model to extract the lesion region of the breast tumor.

[0182] Specifically, the lesion area extraction module 400 includes:

[0183] The parallel processing unit is used to process the 3D breast image in parallel through three convolutional layers with different kernel sizes in a hybrid adaptive attention model, resulting in three feature maps with different receptive fields:

[0184] F 3 =W 3×3 ×F input

[0185] F 5 =W 5×5 ·F input

[0186]

[0187] Among them, F input It is a 3D image of the breast, W 3×3 W 5×5 , There are three convolution kernels, F 3 F 5 F D These are three feature maps with different receptive fields;

[0188] Global average pooling unit is used to pool feature maps. and A new feature map is obtained by using global average pooling. Right now

[0189] Feature map activation units are used to activate feature map F. G Perform Sigmoid activation to obtain F D Channel attention map α and F 5 Channel attention map α′=1-α;

[0190] The first feature map calibration unit is used to... F were obtained respectively D and F 5 Calibrated feature map and in, It is F D The calibrated feature map It is F 5 The calibrated feature map.

[0191] Feature map convolutional units are used to pass through For feature map F respectively 3 , and Perform a convolution operation to obtain the convolutional feature map F. S1 and

[0192] Spatial attention map acquisition unit, used to obtain spatial attention maps through For feature map F S1 and Perform convolution operations and activation using the sigmoid function to obtain the spatial attention map β; where σ is the sigmoid activation function. r It is the ReLU activation function;

[0193] The second feature map calibration unit is used to... Obtained respectively and F S1 Calibrated feature map and F S1′ ; where β ′ =1-β;

[0194] Lesion area acquisition unit, used to obtain lesion area The output F of the entire hybrid adaptive attention model for the breast tumor lesion region is obtained. out .

[0195] The weighted graph construction module 500 is used to model the breast region as a weighted graph, where each region is a node, the connection path between regions is an edge, and each edge has a weight, representing the difficulty or distance of puncture from one region to another, and the tumor lesion region is the target node.

[0196] The puncture path planning module 600 is used to plan the puncture path by comparing the distance values ​​between adjacent nodes, with the target node and the starting node as the two endpoints.

[0197] Specifically, the puncture path planning module 600 is used to calculate the distance from each node to all its neighboring nodes, starting from the starting node, and update the distance values ​​and predecessor nodes. The unvisited node with the smallest current distance is selected as the next node to operate on. If the path from the current node to a neighboring node is smaller than the original distance value of that node, the distance value of that node is updated. Simultaneously, the current node is marked as visited, indicating that the shortest path has been determined. Starting from the target node, the process backtracks along the predecessor nodes of each node until the starting node is reached, thus obtaining the shortest path, i.e., the puncture path.

[0198] The embodiments of the present invention can detect important targets in the early stage of breast cancer, effectively improve the accuracy of breast lesion detection, enhance the standardized screening and diagnosis capabilities of breast cancer, and have significant practical implications for breast cancer treatment technology.

[0199] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0200] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0201] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0202] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0203] Any aspects of this invention not described in detail in the embodiments are well-known techniques to those skilled in the art. Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this invention and not to limit it. Although this invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this invention without departing from the spirit and scope of this invention, and all such modifications and substitutions should be covered within the scope of the claims of this invention.

Claims

1. A method for planning a puncture path for breast tumors, characterized in that, include: Acquire target images; The target image is segmented to obtain a breast contour image; A three-dimensional image of the breast is obtained by reconstructing the breast contour image. The three-dimensional image of the breast is input into a pre-built hybrid adaptive attention model to extract the breast tumor lesion area; The breast region is modeled as a weighted graph, where each region is a node, the connection path between regions is an edge, and each edge has a weight representing the difficulty or distance of puncture from one region to another. The tumor lesion region is the target node. Using the target node and the starting node as the two endpoints, the puncture path is planned by comparing the distance values ​​between adjacent nodes.

2. The method for planning a breast tumor biopsy path as described in claim 1, characterized in that, After acquiring the target image, the following is also included: Histogram equalization enhancement and homomorphic filtering enhancement are performed on the target image respectively to obtain two enhanced breast ultrasound images that are spatially and temporally correlated and complementary in information. Gaussian pyramid decomposition was performed on the enhanced breast ultrasound images that possessed spatiotemporal correlation and information complementarity to obtain images of different resolutions; Subtracting the images at different resolutions yields a series of difference images; The difference images are fused using a weighted average to obtain a series of new pyramid images; The series of new pyramid images are superimposed layer by layer to obtain the final merged image.

3. The method for planning a breast tumor biopsy path as described in claim 1, characterized in that, The step of performing region segmentation on the target image to obtain a breast contour image includes: Create an empty image with the same size as the target image; Perform pixel traversal on the empty image based on the target image; Using the formula grad B (X)=δ B (X)-β B (X) Calculate the morphological gradient grad of the corresponding region in the empty image. B (X); where δ B (X) is the image after dilation of the target image, β B (X) is the image after the target image has undergone an erosion operation; The morphological gradient grad B (X) is compared with a preset threshold; If the morphological gradient grad B If (X) is greater than the preset threshold, corresponding pixel values ​​are generated in the corresponding area of ​​the empty image until the entire breast contour is generated.

4. The method for planning a breast tumor biopsy path as described in claim 1, characterized in that, The process of reconstructing a three-dimensional image of the breast from the breast contour image includes: A series of breast contour images are superimposed on a pre-created image stack to form a three-dimensional volume to be explored; The three-dimensional volume is meshed to obtain multiple cubic units; The attribute values ​​of the vertices of each cube unit are compared with preset attribute thresholds to determine the state of each vertex; The corresponding triangle structure is obtained by querying a predefined triangle lookup table based on the combination of the states of each vertex. The triangles generated inside each cube unit are interpolated according to the state positions of their vertices to obtain accurate triangles; Connect all the precise triangles to form the final three-dimensional image of the breast.

5. The method for planning a breast tumor biopsy path as described in any one of claims 1-4, characterized in that, The process of inputting the three-dimensional image of the breast into a pre-constructed hybrid adaptive attention model to extract the breast tumor lesion region includes: The breast 3D image is processed in parallel using three convolutional layers with different kernel sizes in the hybrid adaptive attention model to obtain three feature maps with different receptive fields: F 3 =W 3×3 ×F input F 5 =W 5×5 ·F input Among them, F input It is the three-dimensional image of the breast, W 3×3 W 5×5 , There are three convolution kernels, F 3 F 5 F D These are three feature maps with different receptive fields; The feature map and A new feature map is obtained by using global average pooling. For the feature map F G Perform Sigmoid activation to obtain F D Channel attention map α and F 5 Channel attention map α′=1-α; pass F were obtained respectively D and F 5 Calibrated feature map and pass For feature map F respectively 3 , and Perform a convolution operation to obtain the convolutional feature map F. S1 and pass For the feature map F S1 and Perform convolution operations and activation using the sigmoid function to obtain the spatial attention map β; where σ is the sigmoid activation function. r It is the ReLU activation function; pass Obtained respectively and F S1 Calibrated feature map and FS1′, where β′=1-β; pass The output F of the entire breast tumor lesion region of the hybrid adaptive attention model is obtained. out .

6. A breast tumor biopsy path planning system, characterized in that, include: The image acquisition module is used to acquire target images; The region segmentation module is used to segment the target image into regions to obtain a breast contour image; A 3D reconstruction module is used to perform 3D reconstruction based on the breast contour image to obtain a 3D image of the breast. The lesion area extraction module is used to input the three-dimensional image of the breast into a pre-constructed hybrid adaptive attention model to extract the lesion area of ​​the breast tumor; The weighted graph construction module is used to model the breast region as a weighted graph, where each region is a node, the connection path between regions is an edge, and each edge has a weight, representing the difficulty or distance of puncture from one region to another. The tumor lesion region is the target node. The puncture path planning module is used to plan the puncture path by comparing the distance values ​​between adjacent nodes, with the target node and the starting node as the two endpoints.

7. The breast tumor biopsy path planning system as described in claim 6, characterized in that, Also includes: The image enhancement module is used to perform histogram equalization enhancement and homomorphic filtering enhancement on the target image respectively, to obtain two enhanced breast ultrasound images that are spatially and temporally correlated and complementary in information. The Gaussian pyramid decomposition module is used to perform Gaussian pyramid decomposition on the enhanced breast ultrasound images that have spatiotemporal correlation and information complementarity to obtain images of different resolutions. The image subtraction module is used to subtract the images of different resolutions to obtain a series of difference images; An image fusion module is used to fuse the difference images using a weighted average to obtain a series of new pyramid images; The image overlay module is used to overlay the series of new pyramid images layer by layer to obtain the final fused image.

8. The breast tumor biopsy path planning system as described in claim 6, characterized in that, The region segmentation module includes: An empty image creation unit is used to create an empty image with the same size as the target image; A pixel traversal unit is used to perform pixel traversal on the empty image based on the target image; Morphological gradient calculation unit, used to calculate gradient using the formula grad B (X)=δ B (X)-β B (X) Calculate the morphological gradient grad of the corresponding region in the empty image. B (X); where δ B (X) is the image after dilation of the target image, β B (X) is the image after the target image has undergone an erosion operation; A morphological gradient comparison unit is used to compare the morphological gradient grad B (X) is compared with a preset threshold; The breast contour image creation unit is used to create a breast contour image if the morphological gradient grad B If (X) is greater than the preset threshold, corresponding pixel values ​​are generated in the corresponding area of ​​the empty image until the entire breast contour is generated.

9. The breast tumor puncture path planning system as described in claim 6, characterized in that, The three-dimensional reconstruction module includes: An image stacking unit is used to superimpose a series of breast contour images on a pre-created image stack to form a three-dimensional volume to be explored; Mesh division unit, used to divide the three-dimensional volume into meshes to obtain multiple cubic units; A vertex state determination unit is used to compare the attribute values ​​of the vertices of each cube unit with a preset attribute threshold to determine the state of each vertex. The lookup unit is used to query a predefined triangle lookup table based on the combination of the states of each vertex to obtain the corresponding triangle structure. An interpolation unit is used to interpolate the triangular structure generated inside each cube unit according to the state position of its vertices to obtain an accurate triangle; A 3D image construction unit is used to connect all the precise triangles to form the final 3D image of the breast.

10. The breast tumor biopsy path planning system as described in any one of claims 6-9, characterized in that, The lesion area extraction module includes: The parallel processing unit is used to perform parallel processing on the three-dimensional breast image through three convolutional layers with different kernel sizes in the hybrid adaptive attention model, resulting in three feature maps with different receptive fields: F 3 =W 3×3 ×F input F 5 =W 5×5 ·F input Among them, F input It is the three-dimensional image of the breast, W 3×3 W 5×5 , There are three convolution kernels, F 3 F 5 F D These are three feature maps with different receptive fields; A global average pooling unit is used to process the feature map. and A new feature map is obtained by using global average pooling. Feature map activation unit, used for activating the feature map F G Perform Sigmoid activation to obtain F D Channel attention map α and F 5 Channel attention map α′=1-α; The first feature map calibration unit is used to... F were obtained respectively D and F 5 Calibrated feature map and Feature map convolutional units are used to pass through For feature map F respectively 3 , and Perform a convolution operation to obtain the convolutional feature map F. S1 and Spatial attention map acquisition unit, used to obtain spatial attention maps through For the feature map F S1 and Perform convolution operations and activation using the sigmoid function to obtain the spatial attention map β; where σ is the sigmoid activation function. r It is the ReLU activation function; The second feature map calibration unit is used to... Obtained respectively and F S1 Calibrated feature map and F S1′ Where, β′=1-β; Lesion area acquisition unit, used to obtain lesion area The output F of the entire breast tumor lesion region of the hybrid adaptive attention model is obtained. out .

Images