Mammary gland molybdenum target image classification detection method, electronic equipment and storage medium
By extracting coarse-grained and full-resolution semantic features of mammography images and combining them with attention mechanism and cross-attention fusion technology, the problem of poor mammography image classification in existing technologies is solved, and more efficient early breast cancer screening is achieved.
Patent Information
- Application Number
- CN202510464077.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-14
- Publication Date
- 2025-09-12
AI Technical Summary
The existing computer-aided diagnosis system is not effective in classifying benign and malignant breast mammography images and is unable to meet actual needs.
By acquiring mammographic images, coarse-grained image features are extracted, and classification detection is performed based on full-resolution semantic features, including the probability of lesion area, lesion type, and BIRADS grade. The attention module is used to learn the contextual relationship of image features, and a multi-layer cross-attention mechanism is combined for feature fusion.
It retains fine image features to the greatest extent, improves the accuracy and efficiency of classification detection, and enhances the effect of mammography image diagnosis.
Smart Images
Figure CN120635520A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and specifically provides a breast mammography image classification detection method, electronic equipment and storage medium. Background Art
[0002] Breast cancer is the most common malignant tumor in women worldwide. In China, its incidence is increasing year by year and is affecting younger people. Early diagnosis of breast cancer is crucial to improving patient survival rates.
[0003] Mammography is a core method for early breast cancer screening. It offers the following advantages: it clearly displays subtle breast structures (such as tiny calcifications); it is easy to use, relatively low-cost, and suitable for large-scale screening. However, the core challenges of mammography diagnosis are its low image reading efficiency and high dependence on medical resources.
[0004] Traditional computer-aided diagnosis systems mostly rely on traditional machine learning algorithms to classify breast mammography images as benign or malignant, but the detection effect obtained by this method is poor and cannot meet actual needs. Summary of the Invention
[0005] In order to overcome the above-mentioned defects, the present application is proposed to provide a solution or at least partially solve the technical problem of poor detection effect of existing computer-aided diagnosis methods. The present application provides a mammography image classification detection method, an electronic device and a storage medium.
[0006] In a first aspect, the present application provides a method for classifying and detecting mammary gland mammography images, the method comprising:
[0007] Obtain mammographic images;
[0008] Extracting coarse-grained image features of the mammographic image;
[0009] Obtaining full-resolution semantic features of the mammographic image based on the mammographic image and the coarse-grained image features;
[0010] Classification detection is performed based on the full-resolution semantic features of the mammography image to obtain a classification detection result, wherein the detection result includes whether there is a lesion area in the mammography image, the lesion type and probability of the lesion area, and the probability of BIRADS grading.
[0011] In one embodiment of the present application, before extracting the coarse-grained image features of the mammography image, the method further includes:
[0012] Dividing the mammographic image into a plurality of sub-regions according to preset sizes;
[0013] Determine the information entropy of each sub-region;
[0014] Determining whether the mammographic image is a positive film based on the information entropy of the sub-region;
[0015] If not, an inversion operation is performed on the mammographic image.
[0016] In one embodiment of the present application, before dividing the mammographic target image into a plurality of sub-regions according to a preset size, the method further includes:
[0017] Sorting all pixels in the mammography image according to brightness values;
[0018] Based on the sorting results, the pixels corresponding to the highest and lowest brightness values are removed.
[0019] In one embodiment of the present application, determining the information entropy of each sub-region includes:
[0020] Determining a global probability of occurrence of a brightness value for each pixel in the mammographic image;
[0021] Determining the local probability of occurrence of each pixel in the sub-region;
[0022] Determine the information entropy of each sub-region based on the global probability and the local probability; and / or,
[0023] The determining whether the mammographic image is a positive film based on the information entropy of the sub-region includes:
[0024] sorting the information entropy of all sub-regions in the mammographic image;
[0025] Obtaining a preset number of first information entropies and second information entropies from the information entropy sorting result, wherein the first information entropy is a first preset number of information entropies in the information entropy sorting result, and the second information entropy is a last preset number of information entropies in the information entropy sorting result;
[0026] Calculating a first brightness average value of all pixel points corresponding to a preset number of first information entropies, and calculating a second brightness average value of all pixel points corresponding to a preset number of second information entropies;
[0027] determining whether the mammographic image is a positive film based on a comparison result of the first brightness average value and the second brightness average value; and / or
[0028] The performing of the inversion operation on the mammography image includes determining a difference between a maximum brightness value and a brightness value of each pixel in the mammography image, and updating the brightness value of the pixel based on the difference.
[0029] In one embodiment of the present application, extracting coarse-grained image features of the mammography image includes:
[0030] reducing the mammographic image to a preset size;
[0031] Using a feature extraction network to obtain coarse features of the mammographic image;
[0032] Using a first attention module to learn the contextual relationship of the coarse feature to obtain a first enhanced feature;
[0033] fusing the first enhanced feature with the second enhanced feature using a second attention module to obtain a first fused feature, where the second enhanced feature is a feature obtained by the first attention module of the mammography image at the same projection position on the corresponding side of the mammography image;
[0034] The first fusion feature and the second fusion feature are fused using a third attention module to obtain the coarse-grained image feature, and the second fusion feature is a feature obtained after the mammography image on the same side and in the same projection position as the mammography image passes through the first attention module and the second attention module in sequence.
[0035] In one embodiment of the present application, the acquiring of full-resolution semantic features of the mammography image based on the mammography image and the coarse-grained image features includes:
[0036] Dividing the mammographic image into a plurality of sub-regions according to preset sizes;
[0037] Acquire a first image feature of each of the sub-regions based on the coarse-grained image feature;
[0038] Using a fourth attention module to learn the contextual relationship of the first image feature to obtain a third enhanced feature;
[0039] The third enhancement feature and the fourth enhancement feature are fused using a fifth attention module to obtain a third fused feature, where the fourth enhancement feature is a feature obtained by the fourth attention module after the first image feature of the corresponding subregion of the mammographic image of the same projection position on the corresponding side of the subregion is passed through the first image feature of the mammographic image of the corresponding subregion;
[0040] fusing the third fused feature with the fourth fused feature using a sixth attention module to obtain a fifth fused feature, where the fourth fused feature is a feature obtained by sequentially passing the first image feature of the corresponding subregion of the mammographic image of the corresponding side of the subregion and the same projection position through the fourth attention module and the fifth attention module;
[0041] The fifth fusion features of all sub-regions of the mammography image are merged to obtain a full-resolution semantic feature of the mammography image.
[0042] In one embodiment of the present application, obtaining the first image feature of each sub-region based on the coarse-grained image feature includes:
[0043] extracting semantic features of each of the sub-regions;
[0044] The semantic feature of the sub-region is spliced with the feature at the corresponding position in the coarse-grained image feature to obtain the first image feature of the sub-region.
[0045] In one embodiment of the present application, a concat function is used to concatenate the semantic features of the sub-region with the corresponding position features in the coarse-grained image features.
[0046] In a second aspect, an electronic device is provided, comprising:
[0047] at least one processor;
[0048] and, a memory communicatively coupled to the at least one processor;
[0049] Wherein, a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the aforementioned breast mammography image classification detection method.
[0050] In a third aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored in the computer-readable storage medium, wherein the program codes are suitable for being loaded and run by a processor to execute any of the aforementioned breast mammography image classification detection methods.
[0051] The above one or more technical solutions of this application have at least one or more of the following Beneficial effects:
[0052] The present application provides a mammography image classification and detection method, specifically comprising: acquiring a mammography image; extracting coarse-grained image features from the mammography image; obtaining full-resolution semantic features of the mammography image based on the mammography image and the coarse-grained image features; and performing classification detection based on the full-resolution semantic features of the mammography image to obtain classification detection results. In this manner, by combining the original mammography image with the coarse-grained image features to obtain full-resolution semantic features, fine image features are minimized, thereby improving the detection effect of classification detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] The disclosure of this application will be more easily understood with reference to the accompanying drawings. Those skilled in the art will readily appreciate that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this application. Furthermore, similar numbers in the figures represent similar components, where:
[0054] Figure 1 This is a schematic diagram of the main process of a mammography image classification detection method in one embodiment of the present application;
[0055] Figure 2 and Figure 3 1 is a schematic diagram of extracting coarse-grained image features of a mammography image in one embodiment of the present application;
[0056] Figure 4 and Figure 5 1 is a flow chart of obtaining full-resolution semantic features of a mammography image in one embodiment of the present application;
[0057] Figure 6 This is a schematic diagram of classifying and detecting full-resolution semantic features of a mammography image to obtain detection results in one embodiment of the present application;
[0058] Figure 7 This is a schematic diagram of the main structure of a mammography image classification and detection device in one embodiment of the present application;
[0059] Figure 8 It is a structural diagram of an electronic device in one embodiment of the present application. DETAILED DESCRIPTION
[0060] Some embodiments of the present application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application and are not intended to limit the scope of protection of the present application.
[0061] In the description of this application, "module" and "processor" may include hardware, software, or a combination of both. A module may include hardware circuitry, various suitable sensors, communication ports, and memory. It may also include software components, such as program code, or a combination of software and hardware. A processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor, or any other suitable processor. A processor has data and / or signal processing capabilities. A processor may be implemented in software, hardware, or a combination of both. Non-transitory computer-readable storage media include any suitable medium capable of storing program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, etc. The term "A and / or B" refers to all possible combinations of A and B, such as only A, only B, or both A and B. The terms "at least one of A or B" or "at least one of A and B" have similar meanings to "A and / or B" and may include only A, only B, or both A and B. The singular forms "a" and "the" may also include the plural forms.
[0062] Currently, most traditional computer-aided diagnosis systems rely on traditional machine learning algorithms to classify mammography images as benign or malignant. However, this approach yields poor detection results and is unable to meet practical needs. Therefore, this application proposes a mammography image classification detection method, electronic device, and storage medium.
[0063] See attached Figure 1 , Figure 1 This is a flow chart of the main steps of a method for classifying and detecting mammary mammography images according to an embodiment of the present application.
[0064] like Figure 1 As shown, the breast mammography image classification detection method in the embodiment of the present application mainly includes the following steps S10 to S40.
[0065] Step S10: Acquire a breast mammography image.
[0066] Mammography is a two-dimensional image of breast tissue captured from different angles using X-rays. It forms the basis of the International Breast Cancer Screening System (BIRADS) and is typically presented in grayscale. Mammography typically includes the lateral cranio-collateral (L-CC) and mediolateral oblique (L-MLO) views of the left breast, and the lateral cranio-collateral (R-CC) and mediolateral oblique (R-MLO) views of the right breast.
[0067] Step S20: extracting coarse-grained image features of the mammography image.
[0068] Coarse-grained image features refer to low-resolution semantic features.
[0069] Step S30: Obtaining full-resolution semantic features of the mammography image based on the mammography image and the coarse-grained image features.
[0070] Full-resolution semantic features refer to feature representations that retain the spatial resolution of the original image while also having semantic information.
[0071] Step S40: performing classification detection based on the full-resolution semantic features of the mammography image to obtain a classification detection result.
[0072] The test results may include whether there is a lesion area in the breast mammography image, the lesion type and probability of the lesion area, BIRADS grade and probability, etc.
[0073] Based on steps S10-S40 described above, a mammographic image is first acquired; coarse-grained image features of the mammographic image are extracted; full-resolution semantic features of the mammographic image are obtained based on the mammographic image and the coarse-grained image features; and classification detection is performed based on the full-resolution semantic features of the mammographic image to obtain classification detection results. In this way, by combining the original mammographic image with the coarse-grained image features to obtain full-resolution semantic features, fine-grained image features are minimized, thereby improving the detection effect of classification detection.
[0074] The above steps S20 to S40 are further explained below.
[0075] Before executing step S20, the mammography image may be preprocessed, where the preprocessing may include but is not limited to a first preprocessing and a second preprocessing, where the first preprocessing may be denoising the mammography image, and the second preprocessing may be determining whether the mammography image is a positive film, and performing an inversion operation when the mammography image is a negative film to invert the mammography image into a positive film.
[0076] In a specific embodiment of denoising a mammography image, the method includes: sorting all pixels in the mammography image according to brightness values; and removing pixels corresponding to the highest brightness value and the lowest brightness value based on the sorting result.
[0077] Specifically, all pixels in the breast mammography image are sorted according to brightness values, such as ascending or descending order, and the pixels corresponding to the highest and lowest brightness values can be deleted, thereby retaining the middle 98% of the pixel values, obtaining the retained range M of the pixel values, removing extreme noise, thereby reducing errors and improving classification accuracy.
[0078] In addition, the second preprocessing can be implemented through the following steps S111 to S114.
[0079] Step S111: Divide the mammography image into multiple sub-regions according to preset sizes.
[0080] The preset size may be a pre-set size, for example, 100×100 pixels, 80×80 pixels, etc. may be used as examples of the preset size.
[0081] Specifically, a sliding window method may be used to divide the mammographic image into a plurality of sub-regions Patches according to a preset size (eg, 100×100 pixels).
[0082] Step S112: Determine the information entropy of each sub-region.
[0083] Information entropy is used to describe the complexity and information richness of an image. A higher information entropy indicates a greater amount of information in the region, making it a non-background region. A lower information entropy indicates a smaller amount of information in the region, making it likely to be a background region.
[0084] Step S112 can be implemented through the following steps S1121 to S1123.
[0085] Step S1121: Determine the global probability of occurrence of the brightness value of each pixel in the mammography image.
[0086] Specifically, the global probability can be determined by the ratio of the number of times the brightness value v of each pixel in the mammography image appears in the image to the area of the image. The specific formula is as follows:
[0087] Among them, P global (p ij ) is the pixel p ij The global probability of occurrence, W and H are the length and width of the mammography image.
[0088] Step S1122: Determine the local probability of each pixel point appearing in the sub-region.
[0089] Specifically, for each sub-region, the local probability is obtained by the ratio of the occurrence frequency of each pixel point in the sub-region to the area of the sub-region. The specific calculation formula is as follows:
[0090] Among them, P local (p ij ) is the pixel p ij The local probability of occurrence, w and h are the length and width of the sub-region.
[0091] Step S1123: Determine the information entropy of each sub-region based on the global probability and the local probability.
[0092] Specifically, the calculation formula for the information entropy of each sub-region is as follows:
[0093] Among them, entropy patch is the information entropy, and P represents the mammography image.
[0094] Step S113: Determine whether the mammographic image is a positive film based on the information entropy of the sub-region.
[0095] Step S113 can be implemented through the following steps S1131 to S1134.
[0096] Step S1131: sorting the information entropy of all sub-regions in the mammography image.
[0097] Specifically, the information entropy of all sub-regions is sorted according to a preset order, for example, from low to high.
[0098] Step S1132: Obtain a preset number of first information entropies and second information entropies from the information entropy sorting result, wherein the first information entropy is the first preset number of information entropies in the information entropy sorting result, and the second information entropy is the last preset number of information entropies in the information entropy sorting result.
[0099] The preset number may be pre-set, and 5, 10, the first 1 / 3 or the last 1 / 3 of the information entropy in the sorting results may be taken as examples of the preset number.
[0100] Step S1133: calculating a first brightness average value of all pixel points corresponding to a preset number of first information entropies, and calculating a second brightness average value of all pixel points corresponding to a preset number of second information entropies.
[0101] Specifically, the average brightness value of all pixels corresponding to a preset number of first information entropies is used as the first brightness average value, and the average brightness value of all pixels corresponding to a preset number of second information entropies is used as the second brightness average value.
[0102] Step S1134: Determine whether the mammographic image is a positive film based on the comparison result of the first brightness average value and the second brightness average value.
[0103] Specifically, when the information entropy sorting results are sorted from low to high, if the first brightness average is greater than the second brightness average, the current mammography image is a negative image; and if the first brightness average is less than the second brightness average, the current mammography image is a positive image. The principle of sorting the information entropy sorting results from high to low is similar to the above and will not be further described here.
[0104] For example, the first 1 / 3 of the information entropy can be selected from the sorting results of the information entropy as the first information entropy, and the last 1 / 3 of the information entropy can be selected as the second information entropy. Then, the brightness average value (first brightness average value) of all pixels corresponding to the first 1 / 3 of the information entropy and the brightness average value (second brightness average value) of all pixels corresponding to the last 1 / 3 of the information entropy are calculated. When the first brightness average value is greater than the second brightness average value, the current mammography image is a negative film, and when the first brightness average value is less than the second brightness average value, the current mammography image is a positive film.
[0105] Step S114: When the mammography image is a negative film, performing an inversion operation on the mammography image.
[0106] In a specific embodiment of the present application, performing the inversion operation on the mammography image includes: determining a difference between a maximum brightness value and a brightness value of each pixel in the mammography image, and updating the brightness value of the pixel based on the difference.
[0107] The maximum brightness value is the maximum value of the pixel brightness range M after the first preprocessing has been performed on the mammography image.
[0108] Specifically, when the mammography image is a negative film, the difference between the highest brightness value and the brightness value of each pixel in the mammography image is determined (that is, the brightness value of the mammography image is inverted), and the inverted brightness value is kept within the maximum value of the pixel brightness range M after the mammography image has been first preprocessed.
[0109] By determining whether the mammography image is positive and reversing it if it is negative, it is ensured that all post-processing is performed on the basis of the positive film, which is beneficial to improving the detection accuracy of the image.
[0110] Specifically, step S20 can be implemented through the following steps S201 to S205.
[0111] Step S201: reducing the mammographic image to a preset size.
[0112] The preset size may be a pre-set size, for example, 1024×1024 pixels, 512×512 pixels, etc. may be used as examples of the preset size.
[0113] Step S202: using a feature extraction network to obtain coarse features of the mammography image.
[0114] The feature extraction network can be a neural network structure such as ResNet, CNN, Transformer, etc., and there is no specific limitation on this, as long as the network can extract coarse features from breast mammography images.
[0115] Coarse features refer to semantic features with low spatial resolution.
[0116] Feature extraction networks, such as ResNet, can be used to extract coarse features from mammography images.
[0117] Step S203: Use the first attention module to learn the contextual relationship of the coarse feature to obtain the first enhanced feature.
[0118] The first attention module can be implemented through the attention mechanism, which can highlight important information and suppress irrelevant information by learning the contextual relationship of image features.
[0119] Specifically, the attention module can be used to learn the contextual relationship of the coarse features of the mammography image, thereby obtaining the first enhanced feature.
[0120] Step S204: Using a second attention module, the first enhancement feature and the second enhancement feature are fused to obtain a first fused feature. The second enhancement feature is a feature obtained after the mammography image of the same projection position on the corresponding side of the mammography image passes through the first attention module.
[0121] The second attention module can be a cross attention mechanism, which can fuse the first enhanced feature with the second enhanced feature to obtain a first fused feature.
[0122] Exemplarily, when the mammography image is a lateral cranio-collateral view (L-CC) of the left breast, the corresponding lateral isotropic mammography image of the mammography image is a lateral cranio-collateral view (R-CC) of the right breast; when the mammography image is a mediolateral oblique view (L-MLO) of the left breast, the corresponding lateral isotropic mammography image of the mammography image is a mediolateral oblique view (R-MLO) of the right breast.
[0123] Step S205: Use a third attention module to fuse the first fusion feature and the second fusion feature to obtain the coarse-grained image feature, where the second fusion feature is a feature obtained after the mammography image of the same side and projection position of the mammography image passes through the first attention module and the second attention module in sequence.
[0124] The third attention module can be a cross-attention mechanism, which can fuse the first fused features with the second fused image to obtain coarse-grained image features.
[0125] Exemplarily, when the mammography image is a lateral cranio-collateral view (L-CC) of the left breast, the mammography image on the same side and projection position of the mammography image is a mediolateral oblique view (L-MLO) of the left breast; when the mammography image is a lateral cranio-collateral view (R-CC) of the right breast, the mammography image on the corresponding side and projection position of the mammography image is a mediolateral oblique view (R-MLO) of the right breast.
[0126] After the above step S20, the coarse-grained image features of all breast mammography images can be obtained, which provides a basic support for subsequent detection and classification, and is conducive to improving the detection and classification accuracy.
[0127] For example, Figure 2 and Figure 3 It can serve as a complete schematic diagram for extracting coarse-grained image features from all mammography images.
[0128] Specific as Figure 2As shown in the figure, feature extraction is performed on the reduced mammography images L-CC, L-MLO, R-CC, and R-MLO, for example, using a ResNet network to extract features, thereby obtaining coarse image features of the four images. Furthermore, multi-layer cross-hybrid attention is performed on the four images to obtain coarse-grained image features corresponding to the four images.
[0129] The way to perform multi-layer cross-mixed attention on the four images is as follows Figure 3 As shown in the figure, taking the image coarse feature L-CC as an example, self-attention (first attention module) is first used to learn the internal information of the image coarse feature L-CC to enhance important information, thereby obtaining the first enhanced feature; the cross-attention module (second attention module) is used to fuse the first enhanced feature with the coarse feature of the mammography image on the corresponding side of the same projection position (image coarse feature R-CC) to obtain the first fused feature; the cross-attention module (third attention module) is used to fuse the first fused feature with the fused feature on the same side of the mammography image (the fused image obtained by sequentially passing the image coarse feature L-MLO through the self-attention module and the cross-attention module on the corresponding side of the same projection position) to obtain the coarse-grained image feature corresponding to the L-CC image.
[0130] The above is a further description of step S20 , and the following further describes step S30 .
[0131] Specifically, step S30 can be implemented through the following steps S301 to S306.
[0132] Step S301: Divide a mammographic image (eg, L-CC) into multiple sub-regions according to preset sizes.
[0133] The preset size may be a pre-set size, for example, 1024×1024 pixels, 512×512 pixels, etc. may be used as examples of the preset size.
[0134] Specifically, the breast mammography image can be divided into multiple sub-regions Patches by using a sliding window method.
[0135] Step S302: obtaining a first image feature of each sub-region based on the coarse-grained image feature.
[0136] Step S302 can be implemented through the following steps S3021 to S3022.
[0137] Step S3021: extracting semantic features of each sub-region.
[0138] Specifically, feature extraction networks such as ResNet and CNN can be used to extract coarse features from each sub-region of the breast mammography image.
[0139] Step S3022: splicing the semantic features of the sub-region with the features of the corresponding positions in the coarse-grained image features to obtain the first image features of the sub-region.
[0140] Specifically, the semantic features of each sub-region are spliced with the features of the corresponding position in the coarse-grained image features, so as to obtain the first image features of each sub-region.
[0141] Specifically, the concat function can be used to concatenate the semantic features of the sub-region with the corresponding position features in the coarse-grained image features to obtain the first image features of the sub-region.
[0142] According to the above method, the same operation (operations corresponding to steps S301 and S302) can also be performed on the corresponding side projection position of the mammographic image (the corresponding side projection position of L-CC is R-CC), thereby obtaining the first image feature corresponding to the corresponding side projection position image. Also, the first image feature corresponding to the same side projection position (the corresponding side projection position of L-CC is L-MLO) image is obtained.
[0143] Step S303: Use the fourth attention module to learn the contextual relationship of the first image feature to obtain the third enhanced feature.
[0144] The fourth attention module can be implemented through the attention mechanism, which can highlight important information and suppress irrelevant information by learning the contextual relationship of image features.
[0145] Specifically, an attention module can be used to learn the contextual relationship of the first image feature to obtain the third enhanced feature.
[0146] Step S304: Use the fifth attention module to fuse the third enhancement feature with the fourth enhancement feature to obtain a third fused feature, where the fourth enhancement feature is a feature obtained by the fourth attention module based on the first image feature of the corresponding sub-region of the mammographic image of the same projection position on the corresponding side of the sub-region.
[0147] The fifth attention module can be a cross attention mechanism, which can fuse the third enhanced feature with the fourth enhanced feature to obtain a third fused feature.
[0148] Step S305: Use the sixth attention module to fuse the third fusion feature and the fourth fusion feature to obtain a fifth fusion feature, where the fourth fusion feature is the feature obtained after the first image feature of the corresponding sub-region of the breast mammography image of the same projection position on the corresponding side of the sub-region passes through the fourth attention module and the fifth attention module in sequence.
[0149] The sixth attention module can be a cross attention mechanism, which can fuse the third fusion feature with the fourth fusion feature to obtain the fifth fusion feature.
[0150] Step S306: merging the fifth fusion features of all sub-regions of the mammography image to obtain the full-resolution semantic features of the mammography image.
[0151] Specifically, the fifth fusion features corresponding to all sub-regions of the same image are merged according to the corresponding positions to obtain the full-resolution semantic features of the mammography image.
[0152] Figure 4 and Figure 5 This is a complete flowchart for obtaining full-resolution semantic features of mammography images.
[0153] like Figure 4 As shown, perform the following operations on two projection positions on each side of the image, a total of four images:
[0154] For the original image (for example, L-CC), the image slice Patch sub-region is obtained in a sliding window manner according to the size of 1024*1024 pixels, and then ResNet is used to extract features. The concat function is used to splice the extracted features with the features of the corresponding positions in the coarse-grained image features of the entire image to obtain the first image features of each sub-region.
[0155] The same operation is performed on the corresponding side of the image with the same projection position (the corresponding side of the L-CC is the R-CC), and the image slice Patch (sub-region) is obtained in a sliding window manner with a size of 1024*1024 pixels. Then, ResNet is used to extract features, and the concat function is used to splice the extracted features with the features of the corresponding position in the coarse-grained image features of the entire image to obtain the first image feature of each sub-region.
[0156] The ipsilateral corresponding projection position of the image (the ipsilateral corresponding projection position of L-CC is L-MLO) is divided into sub-regions to extract coarse-grained image features of the sub-regions.
[0157] The first image features of the current side current projection position of each sub-region, the first image features of the current projection position on the corresponding side, and the coarse-grained image features of the current side current projection position are further input into the multi-layer cross-mixed attention, and the full-resolution semantic features of each sub-region are output. The full-resolution semantic features of all sub-regions are spliced to obtain the full-resolution semantic features of the breast mammography image.
[0158] The process of inputting the first image feature of the current projection position of the current side of each sub-region, the first image feature of the current projection position of the corresponding side, and the coarse-grained image feature into the multi-layer cross-hybrid attention module for processing is as follows: Figure 5 As shown. Figure 5 As shown, taking the sub-region features of the current projection position on the current side as an example, the self-attention mechanism is first used to perform contextual learning on the sub-region features of the current projection position on the front side to obtain the third enhanced feature. Then, the cross-attention mechanism is used to fuse the third enhanced feature and the fourth enhanced feature to obtain the third fused feature, where the fourth enhanced feature is the feature obtained by the fourth attention module after the first image feature of the corresponding sub-region of the mammography image of the same projection position on the corresponding side of the sub-region is passed through the self-attention module. Finally, the third fused feature and the fourth fused feature are fused using the same-side cross-attention mechanism to obtain the fifth fused feature (that is, the full-resolution semantic feature of the sub-region), where the fourth fused feature is the feature obtained by the self-attention mechanism after the sub-region features of the corresponding projection position on the current side are passed through the self-attention mechanism.
[0159] Figure 6 This is a schematic diagram of performing classification detection on full-resolution semantic features of a mammography image to obtain classification detection results.
[0160] Specific as Figure 6 As shown in the figure, a multi-task decoder is used, that is, three independent CNN networks plus activation functions are used to classify and detect the full-resolution semantic features of the image, so as to obtain whether there is a lesion area in the breast mammography image, the lesion type and probability of the lesion area, and the BIRADS grade and probability.
[0161] Specifically, the detection result includes a 0 / 1 mask image corresponding to the size of the original image, where 0 corresponds to areas without lesions in the original image and 1 corresponds to areas with lesions in the original image. Lesion types include masses, calcifications, dense shadows, architectural distortions, lymph nodes, skin lesions, and nipple lesions.
[0162] The lesion grade is the probability of BIRADS grade at each location, including the probability of BIRADS 1, BIRADS 2, BIRADS 3, BIRADS 4A, BIRADS 4B, BIRADS 4C, BIRADS 5, etc.
[0163] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art will understand that in order to achieve the effect of the present application, different steps do not have to be performed in such an order. They can be performed simultaneously (in parallel) or in other orders. These changes are within the scope of protection of the present application.
[0164] Furthermore, the present application also provides a breast mammography image classification and detection device.
[0165] See attached Figure 7 , Figure 7 This is a main structural block diagram of a breast mammography image classification and detection device according to an embodiment of the present application.
[0166] like Figure 7 As shown, the breast mammography image classification and detection device in the embodiment of the present application mainly includes a first acquisition module 11, an extraction module 12, a second acquisition module 13, and a classification and detection module 14. In some embodiments, one or more of the first acquisition module 11, the extraction module 12, the second acquisition module 13, and the classification and detection module 14 can be combined into one module.
[0167] In some embodiments, the acquisition module 11 may be configured to acquire a mammographic image.
[0168] The extraction module 12 may be configured to extract coarse-grained image features of the mammographic image.
[0169] The second acquisition module 13 may be configured to acquire full-resolution semantic features of the mammography image based on the mammography image and the coarse-grained image features.
[0170] The classification detection module 14 may be configured to perform classification detection based on the full-resolution semantic features of the mammography image to obtain a classification detection result.
[0171] In one embodiment, the description of the specific implementation functions can be found in steps S10-S40.
[0172] The above-mentioned breast mammography image classification detection device is used to perform Figure 1 The embodiments of the breast mammography image classification and detection method shown in the figure are similar in technical principles, technical problems solved, and technical effects produced. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process and related instructions of the breast mammography image classification and detection device can refer to the contents described in the embodiments of the breast mammography image classification and detection method, and will not be repeated here.
[0173] Furthermore, it should be understood that since the configuration of each module is merely for the purpose of illustrating the functional units of the apparatus of the present application, the physical devices corresponding to these modules may be the processor itself, or a portion of the software in the processor, a portion of the hardware, or a combination of software and hardware. Therefore, the number of modules in the figure is merely illustrative.
[0174] Those skilled in the art will appreciate that the various modules in the device can be adaptively split or merged. Such splitting or merging of specific modules will not cause the technical solution to deviate from the principles of this application. Therefore, the technical solutions after splitting or merging will fall within the scope of protection of this application.
[0175] It will be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment of the present application can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of the above-mentioned various method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device, medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electric carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code.
[0176] Furthermore, the present application also provides an electronic device, which may include at least one processor; and a memory in communication with the at least one processor; wherein the memory stores a computer program, and when the computer program is executed by the at least one processor, it implements the breast mammography image classification detection method described in any of the above embodiments. Figure 8 As shown, Figure 8 exemplarily shows the structure of an electronic device, which includes a processor 100 and a memory 200.
[0177] Furthermore, the present application also provides a computer-readable storage medium. In a computer-readable storage medium embodiment according to the present application, the computer-readable storage medium can be configured to store a program for executing the breast mammography image classification detection method of the above-mentioned method embodiment, and the program can be loaded and run by the processor to implement the above-mentioned breast mammography image classification detection method. For ease of explanation, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present application. The computer-readable storage medium can be a memory device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present application is a non-temporary computer-readable storage medium.
[0178] The relevant user personal information that may be involved in the various embodiments of this application is strictly in accordance with the requirements of laws and regulations, following the principles of legality, legitimacy and necessity, and based on the reasonable purposes of business scenarios, to process the personal information that users actively provide during the use of products / services or generated due to the use of products / services, as well as the personal information obtained with the user's authorization.
[0179] The user personal information processed in this application will vary depending on the specific product / service scenario and will be based on the specific scenario in which the user uses the product / service. This may involve the user's account information, device information, driving information, vehicle information, or other related information. The applicant will treat the user's personal information and its processing with a high degree of diligence.
[0180] This application attaches great importance to the security of user personal information and has taken reasonable and feasible security protection measures that comply with industry standards to protect user information and prevent personal information from being accessed, disclosed, used, modified, damaged or lost without authorization.
[0181] Thus far, the technical solutions of the present application have been described in conjunction with the specific embodiments shown in the accompanying drawings. However, it is readily understood by those skilled in the art that the scope of protection of the present application is obviously not limited to these specific embodiments. Without departing from the principles of the present application, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present application.
Claims
1. A method for classifying and detecting mammary gland mammography images, characterized in that: The method comprises: Obtain mammographic images; Extracting coarse-grained image features of the mammographic image; Obtaining full-resolution semantic features of the mammographic image based on the mammographic image and the coarse-grained image features; Classification detection is performed based on the full-resolution semantic features of the mammography image to obtain a classification detection result.
2. The method for classifying and detecting mammary gland mammography images according to claim 1, wherein: Before extracting the coarse-grained image features of the mammography image, the method further includes: Dividing the mammographic image into a plurality of sub-regions according to preset sizes; Determine the information entropy of each sub-region; Determining whether the mammographic image is a positive film based on the information entropy of the sub-region; If not, an inversion operation is performed on the mammographic image.
3. The method for classifying and detecting mammary gland mammography images according to claim 2, wherein: Before dividing the mammographic target image into a plurality of sub-regions according to preset sizes, the method further includes: Sorting all pixels in the mammography image according to brightness values; Based on the sorting results, the pixels corresponding to the highest and lowest brightness values are removed.
4. The method for classifying and detecting mammary gland mammography images according to claim 2, wherein: Determining the information entropy of each sub-region includes: Determining a global probability of occurrence of a brightness value for each pixel in the mammographic image; Determining the local probability of occurrence of each pixel in the sub-region; Determine the information entropy of each sub-region based on the global probability and the local probability; and / or, The determining whether the mammographic image is a positive film based on the information entropy of the sub-region includes: sorting the information entropy of all sub-regions in the mammographic image; Obtaining a preset number of first information entropies and second information entropies from the information entropy sorting result, wherein the first information entropy is a first preset number of information entropies in the information entropy sorting result, and the second information entropy is a last preset number of information entropies in the information entropy sorting result; Calculating a first brightness average value of all pixel points corresponding to a preset number of first information entropies, and calculating a second brightness average value of all pixel points corresponding to a preset number of second information entropies; determining whether the mammographic image is a positive film based on a comparison result of the first brightness average value and the second brightness average value; and / or The performing of the inversion operation on the mammography image includes determining a difference between a maximum brightness value and a brightness value of each pixel in the mammography image, and updating the brightness value of the pixel based on the difference.
5. The method for classifying and detecting mammary gland mammography images according to claim 1, wherein: The extracting of coarse-grained image features of the mammography image comprises: reducing the mammographic image to a preset size; Using a feature extraction network to obtain coarse features of the mammographic image; Using a first attention module to learn the contextual relationship of the coarse feature to obtain a first enhanced feature; fusing the first enhanced feature with the second enhanced feature using a second attention module to obtain a first fused feature, where the second enhanced feature is a feature obtained by the first attention module of the mammography image at the same projection position on the corresponding side of the mammography image; The first fusion feature and the second fusion feature are fused using a third attention module to obtain the coarse-grained image feature, and the second fusion feature is a feature obtained after the mammography image on the same side and in the same projection position as the mammography image passes through the first attention module and the second attention module in sequence.
6. The method for classifying and detecting mammary gland mammography images according to claim 1, wherein: The obtaining of full-resolution semantic features of the mammography image based on the mammography image and the coarse-grained image features includes: Dividing the mammographic image into a plurality of sub-regions according to preset sizes; Acquire a first image feature of each of the sub-regions based on the coarse-grained image feature; Using a fourth attention module to learn the contextual relationship of the first image feature to obtain a third enhanced feature; The third enhancement feature and the fourth enhancement feature are fused using a fifth attention module to obtain a third fused feature, where the fourth enhancement feature is a feature obtained by the fourth attention module after the first image feature of the corresponding subregion of the mammographic image of the same projection position on the corresponding side of the subregion is passed through the first image feature of the mammographic image of the corresponding subregion; fusing the third fused feature with the fourth fused feature using a sixth attention module to obtain a fifth fused feature, where the fourth fused feature is a feature obtained by sequentially passing the first image feature of the corresponding subregion of the mammographic image of the corresponding side of the subregion and the same projection position through the fourth attention module and the fifth attention module; The fifth fusion features of all sub-regions of the mammography image are merged to obtain a full-resolution semantic feature of the mammography image.
7. The method for classifying and detecting mammary gland mammography images according to claim 6, wherein: Acquiring a first image feature of each of the sub-regions based on the coarse-grained image feature includes: extracting semantic features of each of the sub-regions; The semantic feature of the sub-region is spliced with the feature at the corresponding position in the coarse-grained image feature to obtain the first image feature of the sub-region.
8. The method for classifying and detecting mammary gland mammography images according to claim 7, wherein: The concat function is used to concatenate the semantic features of the sub-region with the corresponding position features in the coarse-grained image features.
9. An electronic device, characterized in that: include: at least one processor; and, a memory communicatively coupled to the at least one processor; Wherein, a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the breast mammography image classification detection method according to any one of claims 1 to 8 is implemented.
10. A computer-readable storage medium storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by a processor to execute the breast mammography image classification detection method according to any one of claims 1 to 8.