A breast implant prediction method and system based on mammography

Abstract

The present invention provides a breast filler prediction method and system based on mammary gland molybdenum target, specifically, obtaining left and right breast CC views and MLO views, as well as personal information, embedding the personal information to obtain a personal information vector; inputting the personal information vector into a first encoder to obtain a personal information coding vector; determining a target CC feature map and a target MLO feature map according to a target breast selected by a user; obtaining a CC and MLO view difference map according to the view feature map and the target feature map, and further obtaining a CC vector and an MLO vector; inputting the CC vector into a second encoder to obtain a CC coding vector, and inputting the MLO vector into a third encoder to obtain an MLO coding vector; fusing the CC coding vector, the MLO coding vector and the personal information coding vector to obtain a target feature vector, and using the target feature vector as an input of a decoder to obtain a prediction result. The present invention improves the prediction accuracy of the size of breast fillers.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a method and system for predicting breast fillers based on nuclear magnetic resonance images. Background Art

[0002] Breast augmentation is a method of increasing the size, shape or fullness of the breasts through implants or fat. It is a common method of breast plastic surgery and correction of breast deformities. Breast augmentation is also used in reconstruction after partial mastectomy. Existing breast augmentation relies heavily on the doctor's experience and subjective judgment, but there are many factors that affect the actual effect of breast augmentation, such as size, shape, tissue distribution, breast distribution and other breast characteristics. Mammography is a non-invasive, high-resolution imaging technology that can provide detailed breast structure information. Compared with other medical imaging technologies, mammography provides more accurate and comprehensive data on the density, morphology and distribution of breast tissue. In mammography images, breast tissue can be clearly segmented and located, and can provide information on the three-dimensional structure of the breast, such as the distribution of glandular tissue, adipose tissue, blood vessels and mammary ducts. In addition, mammography can also show lumps, cysts and other abnormal lesions in the breast, providing an important basis for preoperative evaluation and intraoperative navigation of breast augmentation surgery.

[0003] The breast filling prediction method based on mammography images uses breast structure information and image processing and machine learning techniques to establish a breast filling prediction model. Such a model can predict the impact of different filling positions and filling amounts on breast morphology according to the individual characteristics and needs of the patient, and provide individualized preoperative planning and intraoperative guidance. This can not only reduce the errors of subjective judgment in traditional filling methods and improve the accuracy and consistency of filling effects, but also help doctors and patients simulate and evaluate breast morphology before surgery, reducing the risk of postoperative dissatisfaction. However, mammography images are only one aspect of breast development. The patient's body shape and age will affect the final filling effect. It is difficult to predict the size of breast fillers using only mammography images. Summary of the invention

[0004] In order to solve the above problems, in a first aspect, the present invention provides a breast implant prediction method based on mammography, the method comprising the following steps:

[0005] Perform image segmentation on the mammary gland mammography image to obtain the CC view and MLO view of the right breast, as well as the CC view and MLO view of the left breast; obtain the patient's personal information, embed each of the personal information to obtain a personal information vector; input the personal information vector into the first encoder to obtain a personal information encoding vector;

[0006] Determine the target CC feature map and the target MLO feature map according to the target breast selected by the user; when predicting the right or left breast filler, extract the feature map of the corresponding CC view and the feature map of the MLO view; obtain a CC view difference map according to the feature map of the CC view and the target CC feature map, obtain an MLO view difference map according to the feature map of the MLO view and the target MLO feature map, and perform patch bedding and position encoding on the difference maps to obtain a CC vector and an MLO vector; input the CC vector into the second encoder to obtain a CC encoding vector, and input the MLO vector into the third encoder to obtain an MLO encoding vector;

[0007] The CC coding vector, MLO coding vector and personal information coding vector are fused to obtain the target feature vector, and the target feature vector is used as the input of the decoder to obtain the prediction result.

[0008] Preferably, obtaining a CC view difference map according to a CC view feature map and a target CC view feature map, and obtaining an MLO view difference map according to a MLO view feature map and a target MLO view feature map, is specifically:

[0009] Subtract the target CC feature map from the CC view feature map to obtain the first tensor, normalize all negative values ​​in the first tensor to [0, 0.5], normalize 0 to 0.5, and normalize positive values ​​to (0.5, 1], thereby obtaining the CC view difference map;

[0010] The feature map of the MLO view is subtracted from the target MLO feature map to obtain the first tensor, all negative values ​​in the first tensor are normalized to [0, 0.5), 0 is normalized to 0.5, and positive values ​​are normalized to (0.5, 1] ​​to obtain the MLO view difference map.

[0011] Preferably, the difference map is patched and position-encoded to obtain a CC vector and an MLO vector, specifically:

[0012] The CC view difference map and the MLO view difference map are divided into multiple image blocks, respectively, and the image blocks with all values ​​of 0.5 are deleted. Then, the remaining image blocks are patched and position encoded to obtain the CC vector and MLO vector.

[0013] Preferably, the CC coding vector, the MLO coding vector and the personal information coding vector are fused to obtain the target feature vector, specifically:

[0014] Normalize the personal information coding vector to [0,1], and concatenate the CC coding vector, the MLO coding vector, and the normalized personal information coding vector;

[0015] The concatenated vector is used as the input of the fully connected layer, and the output of the fully connected layer is the target feature vector; wherein, the dimension of the target feature vector is the same as that of the personal information encoding vector.

[0016] Preferably, the second encoder and the third encoder are the same encoder;

[0017] The difference map is patched and position-encoded to obtain a CC vector and an MLO vector, specifically:

[0018] Divide the CC view difference map and the MLO view difference map into N image block patches respectively. If there are image blocks with all values ​​of 0.5 corresponding to the CC view difference map and there are image blocks with all values ​​of 0.5 corresponding to the MLO view difference map, count the number of image blocks with all values ​​of 0.5 in the two maps N1 and N2, delete the image blocks with min{N1,N2} CC view difference maps with all values ​​of 0.5, and delete the image blocks with min{N1,N2} MLO view difference maps with all values ​​of 0.5. During the deletion process, the image blocks with all values ​​of 0.5 corresponding to the CC view difference map and the MLO view difference map are deleted first. Figure 4 Corner image patches;

[0019] Perform patchbedding and position encoding on the remaining image blocks to obtain CC vectors and MLO vectors;

[0020] Among them, N, N1, and N2 are positive integers, and N>N1, N>N2.

[0021] In addition, the present invention also provides a breast filler prediction system based on mammography, the system comprising the following modules:

[0022] The personal information encoding module is used to perform image segmentation on the mammary gland mammography image to obtain the CC view and MLO view of the right breast, as well as the CC view and MLO view of the left breast; obtain the patient's personal information, embed each of the personal information to obtain a personal information vector; input the personal information vector into the first encoder to obtain a personal information encoding vector;

[0023] A molybdenum target image encoding module is used to determine a target CC feature map and a target MLO feature map according to a target breast selected by a user; when predicting a right or left breast filler, extract the feature map of the corresponding CC view and the feature map of the MLO view; obtain a CC view difference map according to the feature map of the CC view and the target CC feature map, obtain an MLO view difference map according to the feature map of the MLO view and the target MLO feature map, and perform patchembedding and position encoding on the difference maps to obtain a CC vector and an MLO vector; input the CC vector into a second encoder to obtain a CC encoding vector, and input the MLO vector into a third encoder to obtain an MLO encoding vector;

[0024] The prediction module is used to fuse the CC coding vector, the MLO coding vector and the personal information coding vector to obtain a target feature vector, and use the target feature vector as the input of the decoder to obtain a prediction result.

[0025] Preferably, the obtaining of the CC view difference map according to the feature map of the CC view and the target CC feature map, and the obtaining of the MLO view difference map according to the feature map of the MLO view and the target MLO feature map are specifically:

[0026] Subtract the target CC feature map from the CC view feature map to obtain the first tensor, normalize all negative values ​​in the first tensor to [0, 0.5], normalize 0 to 0.5, and normalize positive values ​​to (0.5, 1], thereby obtaining the CC view difference map;

[0027] The feature map of the MLO view is subtracted from the target MLO feature map to obtain the first tensor, all negative values ​​in the first tensor are normalized to [0, 0.5), 0 is normalized to 0.5, and positive values ​​are normalized to (0.5, 1] ​​to obtain the MLO view difference map.

[0028] Preferably, the difference map is patched and position-encoded to obtain a CC vector and an MLO vector, specifically:

[0029] The CC view difference map and the MLO view difference map are divided into multiple image blocks, respectively, and the image blocks with all values ​​of 0.5 are deleted. Then, the remaining image blocks are patched and position encoded to obtain the CC vector and MLO vector.

[0030] Preferably, the CC coding vector, the MLO coding vector and the personal information coding vector are fused to obtain the target feature vector, specifically:

[0031] Normalize the personal information coding vector to [0,1], and concatenate the CC coding vector, the MLO coding vector, and the normalized personal information coding vector;

[0032] The concatenated vector is used as the input of the fully connected layer, and the output of the fully connected layer is the target feature vector; wherein, the dimension of the target feature vector is the same as that of the personal information encoding vector.

[0033] Preferably, the second encoder and the third encoder are the same encoder;

[0034] The difference map is patched and position-encoded to obtain a CC vector and an MLO vector, specifically:

[0035] Divide the CC view difference map and the MLO view difference map into N image block patches respectively. If there are image blocks with all values ​​of 0.5 corresponding to the CC view difference map and there are image blocks with all values ​​of 0.5 corresponding to the MLO view difference map, count the number of image blocks with all values ​​of 0.5 in the two maps N1 and N2, delete the image blocks with min{N1,N2} CC view difference maps with all values ​​of 0.5, and delete the image blocks with min{N1,N2} MLO view difference maps with all values ​​of 0.5. During the deletion process, the image blocks with all values ​​of 0.5 corresponding to the CC view difference map and the MLO view difference map are deleted first. Figure 4 Corner image patches;

[0036] Perform patchbedding and position encoding on the remaining image blocks to obtain CC vectors and MLO vectors;

[0037] Among them, N, N1, and N2 are positive integers, and N>N1, N>N2.

[0038] In addition, the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described in the first aspect is implemented.

[0039] In addition, the present invention also provides a computer device, which includes a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the method described in the first aspect is implemented.

[0040] The present invention utilizes breast mammography, personal information, and target breast information when predicting breast implants, and adopts Transformer as the basic architecture, so that the prediction result is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 This is a flow chart of Embodiment 1;

[0043] Figure 2 For CC view and MLO view;

[0044] Figure 3 is a network structure diagram of an embodiment;

[0045] Figure 4 It is a structural diagram of embodiment 2. DETAILED DESCRIPTION

[0046] In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] First embodiment

[0049] The present invention provides a breast implant prediction method based on mammography. Figure 1 As shown, the method comprises the following steps:

[0050] S1, performing image segmentation on the mammary gland mammography image to obtain the CC view and MLO view of the right breast, as well as the CC view and MLO view of the left breast; obtaining the patient's personal information, embedding each of the personal information to obtain a personal information vector; inputting the personal information vector into the first encoder to obtain a personal information encoding vector;

[0051] The CC view (craniocaudal view) and the MLO view (mediolateral oblique view) are the two most commonly used views in mammography. In the CC view, the X-ray passes from the upper part of the breast to the lower part, and the upper and lower parts of the breast can be clearly seen. The MLO view is a mediolateral oblique view, which is taken from the inside of the breast to the outside, providing a lateral image of the breast tissue and showing the deep tissue of the breast. The CC view and MLO view of the breast are as follows: Figure 2 shown.

[0052] The overall condition of the breast is related to many factors, including but not limited to age, BMI (Body Mass Index), number of breastfeeding times, smoking, drinking, exercise, etc. It is not rigorous to judge the breast filling condition only based on the mammographic target image of the breast, and the result is not very accurate. The present invention predicts the size of the breast filling according to the mammographic target image and personal information at the same time.

[0053] In one embodiment, the present invention preferably uses Transformer as the basic network architecture. When encoding personal information, the embedding layer of Transformer is used to convert the personal information into an embedding vector. Since the various factors in the personal information are independent of each other, the position encoding process is skipped when the embedding layer of Transformer is used.

[0054] S2, determining a target CC feature map and a target MLO feature map according to a target breast selected by a user; when predicting a right or left breast filler, extracting a feature map of the corresponding CC view and a feature map of the MLO view; obtaining a CC view difference map according to the feature map of the CC view and the target CC feature map, obtaining an MLO view difference map according to the feature map of the MLO view and the target MLO feature map, and performing patch bedding and position encoding on the difference maps to obtain a CC vector and an MLO vector; inputting the CC vector into a second encoder to obtain a CC encoding vector, and inputting the MLO vector into a third encoder to obtain an MLO encoding vector;

[0055] Different users have different requirements for the size of their breasts after plastic surgery. In the present invention, the doctor will provide the patient with some templates or target breasts, and the user can select the target breast according to their own needs. The target breast is a breast molybdenum target image with different sizes and / or chest shapes. Since the molybdenum target images taken by the left and right breasts are different, and the left and right breasts of the user are not completely symmetrical, separate predictions are required. After determining the target breast, the CC feature map and the MLO feature map corresponding to the target breast can be determined, wherein the CC feature map includes the CC feature map of the right breast and the CC feature map of the left breast, and the MLO feature map includes the MLO feature map of the right breast and the MLO feature map of the left breast.

[0056] When predicting the filler size of the left or right breast, find the corresponding target CC feature map and target MLO feature map. For example, when predicting the filler size of the left breast, obtain the CC feature map and MLO feature map of the left breast, and then obtain the CC feature map and MLO feature map of the target breast. According to the CC feature map of the left breast and the CC feature map of the target breast, obtain the CC view difference map. Similarly, the MLO view difference map can be obtained.

[0057] In the above process, the CC feature map and the MLO feature map are extracted from the CC view and the MLO view, and the extraction method includes but is not limited to convolution and the backbone network of the convolutional neural network. The backbone network of the convolutional neural network includes AlexNet, VGGNet, Inception or ResNet.

[0058] Personal information is text information. The present invention mainly adopts the Transformer architecture, while the molybdenum target image is image information. ViT is preferably used as the basis. The specific relationship is as follows: Figure 3 shown.

[0059] S3, the CC coding vector, MLO coding vector and personal information coding vector are fused to obtain the target feature vector, and the target feature vector is used as the input of the decoder to obtain the prediction result.

[0060] The CC encoding vector, the MLO encoding vector, and the personal information encoding vector all contain information about the patient's breast. Compared with relying solely on mammographic images, the above method also takes into account the impact of individual differences on breast implants. In S3, the decoder preferably uses a Transformer decoder.

[0061] In a specific implementation, the CC view difference map is obtained according to the feature map of the CC view and the target CC feature map, and the MLO view difference map is obtained according to the feature map of the MLO view and the target MLO feature map, specifically:

[0062] Subtract the target CC feature map from the CC view feature map to obtain the first tensor, normalize all negative values ​​in the first tensor to [0, 0.5], normalize 0 to 0.5, and normalize positive values ​​to (0.5, 1], thereby obtaining the CC view difference map;

[0063] The feature map of the MLO view is subtracted from the target MLO feature map to obtain the first tensor, all negative values ​​in the first tensor are normalized to [0, 0.5), 0 is normalized to 0.5, and positive values ​​are normalized to (0.5, 1] ​​to obtain the MLO view difference map.

[0064] The feature map itself is a vector, and each position has a corresponding value. One case is that the value of the target feature map is greater than the value of the corresponding position of the patient's corresponding feature map. The other two cases are less than and equal to. In order to avoid negative values, in the present invention, negative values ​​are normalized to [0, 0.5), 0 is normalized to 0.5, and positive values ​​are normalized to (0.5, 1). This not only avoids the influence of negative values, but also normalization can prevent gradient disappearance or gradient explosion.

[0065] The segmented CC view and MLO view will contain a part of the area without breast information, such as the upper left corner and the upper right corner. This part does not contain breast information. In one embodiment, the difference map is patched and position-encoded to obtain the CC vector and the MLO vector, specifically:

[0066] The CC view difference map and the MLO view difference map are divided into multiple image blocks, respectively, and the image blocks with all values ​​of 0.5 are deleted. Then, the remaining image blocks are patched and position encoded to obtain the CC vector and MLO vector.

[0067] When the second encoder and the third encoder use the same encoder, the dimensions of the input data need to be the same; in this case, the difference map is patched and position-encoded to obtain the CC vector and the MLO vector, specifically:

[0068] Divide the CC view difference map and the MLO view difference map into N image block patches respectively. If there are image blocks with all values ​​of 0.5 corresponding to the CC view difference map and there are image blocks with all values ​​of 0.5 corresponding to the MLO view difference map, count the number of image blocks with all values ​​of 0.5 in the two maps N1 and N2, delete the image blocks with min{N1,N2} CC view difference maps with all values ​​of 0.5, and delete the image blocks with min{N1,N2} MLO view difference maps with all values ​​of 0.5. During the deletion process, the image blocks with all values ​​of 0.5 corresponding to the CC view difference map and the MLO view difference map are deleted first. Figure 4 Corner image patches;

[0069] For example, both the CC view difference map and the MLO view difference map are divided into 16×16 patches, where the pixel values ​​or vector elements of 7 patches in the CC view difference map are all 0, and the pixel values ​​or vector elements of 5 patches in the MLO view difference map are all 0, then N1=5, N2=7, min{5,7}=5, then delete 5 of the 7 patches in the CC view difference map and delete all 5 patches in the MLO view difference map. When deleting, first delete the patches in the difference map. Figure 4 The patch of the week, located at the difference Figure 4 The patches of a week can be determined by position, for example the first patch is located in the upper left corner.

[0070] Perform patchbedding and position encoding on the remaining image blocks to obtain CC vectors and MLO vectors;

[0071] Among them, N, N1, and N2 are positive integers, and N>N1, N>N2.

[0072] In another embodiment, when patching the remaining image blocks, the mapped embedding vectors are adjusted according to the number of remaining image blocks. For example, if the remaining image blocks are relatively small, the dimension of each image block mapped is large. Conversely, the dimension of the embedded vector mapped to each image block is large. This can ensure that the dimensions of the CC vector and the MLO vector input to the encoder are adapted to the dimensions required by the encoder.

[0073] In one embodiment, the CC coding vector, the MLO coding vector and the personal information coding vector are fused to obtain the target feature vector, specifically:

[0074] Normalize the personal information coding vector to [0,1], and concatenate the CC coding vector, the MLO coding vector and the normalized personal information coding vector; the concatenation preferably adopts the concat operation.

[0075] The concatenated vector is used as the input of the fully connected layer, and the output of the fully connected layer is the target feature vector; wherein, the dimension of the target feature vector is the same as that of the personal information encoding vector.

[0076] In another embodiment, the CC coding vector, the MLO coding vector and the personal information coding vector are fused by using a convolution method or a fully connected method.

[0077] The present invention is based on text information, which requires the fusion of molybdenum target image information and personal information. Specifically, in the Transformer, the personal information will be fused with the molybdenum target information after passing through the Encoder and used as the input of the Decoder, thereby improving the accuracy of the prediction.

[0078] Second embodiment

[0079] The present invention also provides a breast implant prediction system based on mammography. Figure 4 As shown, the system includes the following modules:

[0080] The personal information encoding module is used to perform image segmentation on the mammary gland mammography image to obtain the CC view and MLO view of the right breast, as well as the CC view and MLO view of the left breast; obtain the patient's personal information, embed each of the personal information to obtain a personal information vector; input the personal information vector into the first encoder to obtain a personal information encoding vector;

[0081] A molybdenum target image encoding module is used to determine a target CC feature map and a target MLO feature map according to a target breast selected by a user; when predicting a right or left breast filler, extract the feature map of the corresponding CC view and the feature map of the MLO view; obtain a CC view difference map according to the feature map of the CC view and the target CC feature map, obtain an MLO view difference map according to the feature map of the MLO view and the target MLO feature map, and perform patchembedding and position encoding on the difference maps to obtain a CC vector and an MLO vector; input the CC vector into a second encoder to obtain a CC encoding vector, and input the MLO vector into a third encoder to obtain an MLO encoding vector;

[0082] The prediction module is used to fuse the CC coding vector, the MLO coding vector and the personal information coding vector to obtain a target feature vector, and use the target feature vector as the input of the decoder to obtain a prediction result.

[0083] Preferably, the obtaining of the CC view difference map according to the feature map of the CC view and the target CC feature map, and the obtaining of the MLO view difference map according to the feature map of the MLO view and the target MLO feature map are specifically:

[0084] Subtract the target CC feature map from the CC view feature map to obtain the first tensor, normalize all negative values ​​in the first tensor to [0, 0.5], normalize 0 to 0.5, and normalize positive values ​​to (0.5, 1], thereby obtaining the CC view difference map;

[0085] The feature map of the MLO view is subtracted from the target MLO feature map to obtain the first tensor, all negative values ​​in the first tensor are normalized to [0, 0.5), 0 is normalized to 0.5, and positive values ​​are normalized to (0.5, 1] ​​to obtain the MLO view difference map.

[0086] Preferably, the difference map is patched and position-encoded to obtain a CC vector and an MLO vector, specifically:

[0087] The CC view difference map and the MLO view difference map are divided into multiple image blocks, respectively, and the image blocks with all values ​​of 0.5 are deleted. Then, the remaining image blocks are patched and position encoded to obtain the CC vector and MLO vector.

[0088] Preferably, the CC coding vector, the MLO coding vector and the personal information coding vector are fused to obtain the target feature vector, specifically:

[0089] Normalize the personal information coding vector to [0,1], and concatenate the CC coding vector, the MLO coding vector, and the normalized personal information coding vector;

[0090] The concatenated vector is used as the input of the fully connected layer, and the output of the fully connected layer is the target feature vector; wherein, the dimension of the target feature vector is the same as that of the personal information encoding vector.

[0091] Preferably, the second encoder and the third encoder are the same encoder;

[0092] The difference map is patched and position-encoded to obtain a CC vector and an MLO vector, specifically:

[0093] Divide the CC view difference map and the MLO view difference map into N image block patches respectively. If there are image blocks with all values ​​of 0.5 corresponding to the CC view difference map and there are image blocks with all values ​​of 0.5 corresponding to the MLO view difference map, count the number of image blocks with all values ​​of 0.5 in the two maps N1 and N2, delete the image blocks with min{N1,N2} CC view difference maps with all values ​​of 0.5, and delete the image blocks with min{N1,N2} MLO view difference maps with all values ​​of 0.5. During the deletion process, the image blocks with all values ​​of 0.5 corresponding to the CC view difference map and the MLO view difference map are deleted first. Figure 4 Corner image patches;

[0094] Perform patchbedding and position encoding on the remaining image blocks to obtain CC vectors and MLO vectors;

[0095] Among them, N, N1, and N2 are positive integers, and N>N1, N>N2.

[0096] Third embodiment

[0097] The present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described in the first embodiment is implemented.

[0098] Fourth embodiment

[0099] The present invention further provides a computer device, which includes a processor and a memory. The memory stores a computer program, and when the computer program is executed by the processor, the method described in the first embodiment is implemented.

[0100] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course can also be implemented by combining hardware and software. Based on such an understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a computer product, and the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A breast implant prediction method based on mammography, characterized in that: The method comprises the following steps: Perform image segmentation on the mammary gland mammography image to obtain the CC view and MLO view of the right breast, as well as the CC view and MLO view of the left breast; obtain the patient's personal information, embed each of the personal information to obtain a personal information vector; input the personal information vector into the first encoder to obtain a personal information encoding vector; Determine a target CC feature map and a target MLO feature map according to a target breast selected by a user; when predicting a right or left breast filler, extract a feature map of the corresponding CC view and a feature map of the MLO view; obtain a CC view difference map according to the feature map of the CC view and the target CC feature map, obtain an MLO view difference map according to the feature map of the MLO view and the target MLO feature map, and perform patch embedding and position encoding on the difference map to obtain a CC vector and an MLO vector; input the CC vector into the second encoder to obtain a CC encoding vector, and input the MLO vector into the third encoder to obtain an MLO encoding vector; The CC coding vector, MLO coding vector and personal information coding vector are fused to obtain the target feature vector, and the target feature vector is used as the input of the decoder to obtain the prediction result.

2. The method according to claim 1, characterized in that The CC view difference map is obtained according to the feature map of the CC view and the target CC feature map, and the MLO view difference map is obtained according to the feature map of the MLO view and the target MLO feature map, specifically: Subtract the target CC feature map from the CC view feature map to obtain the first tensor, normalize all negative values ​​in the first tensor to [0, 0.5], normalize 0 to 0.5, and normalize positive values ​​to (0.5, 1], thereby obtaining the CC view difference map; The feature map of the MLO view is subtracted from the target MLO feature map to obtain the first tensor, all negative values ​​in the first tensor are normalized to [0, 0.5), 0 is normalized to 0.5, and positive values ​​are normalized to (0.5, 1] ​​to obtain the MLO view difference map.

3. The method according to claim 2, characterized in that The difference map is patched and position-encoded to obtain a CC vector and an MLO vector, specifically: The CC view difference map and the MLO view difference map are divided into multiple image blocks, respectively, and the image blocks with all values ​​of 0.5 are deleted. Then, the remaining image blocks are subjected to patch embedding and position encoding to obtain the CC vector and MLO vector.

4. The method according to any one of claims 1 to 3, characterized in that: The CC coding vector, the MLO coding vector and the personal information coding vector are fused to obtain the target feature vector, specifically: Normalize the personal information coding vector to [0,1], and concatenate the CC coding vector, the MLO coding vector, and the normalized personal information coding vector; The concatenated vector is used as the input of the fully connected layer, and the output of the fully connected layer is the target feature vector; wherein, the dimension of the target feature vector is the same as that of the personal information encoding vector.

5. The method according to claim 2, characterized in that The second encoder and the third encoder are the same encoder; The difference map is subjected to patch embedding and position encoding to obtain a CC vector and an MLO vector, specifically: Divide the CC view difference map and the MLO view difference map into N image block patches respectively. If there are image blocks with all values ​​of 0.5 corresponding to the CC view difference map and there are image blocks with all values ​​of 0.5 corresponding to the MLO view difference map, count the number of image blocks with all values ​​of 0.5 in the two maps, N1 and N2, delete the image blocks with min{N1, N2} CC view difference maps with all values ​​of 0.5, and delete the image blocks with min{N1, N2} MLO view difference maps with all values ​​of 0.

5. During the deletion process, give priority to deleting the image blocks at the four corners of the CC view difference map and the MLO view difference map. Perform patch embedding and position encoding on the remaining image blocks to obtain CC vectors and MLO vectors; Among them, N, N1, and N2 are positive integers, and N>N1, N>N2.

6. A breast implant prediction system based on mammography, characterized in that: The system includes the following modules: The personal information encoding module is used to perform image segmentation on the mammary gland mammography image to obtain the CC view and MLO view of the right breast, as well as the CC view and MLO view of the left breast; obtain the patient's personal information, embed each of the personal information to obtain a personal information vector; input the personal information vector into the first encoder to obtain a personal information encoding vector; A molybdenum target image encoding module is used to determine a target CC feature map and a target MLO feature map according to a target breast selected by a user; when predicting a right or left breast filler, extract the feature map of the corresponding CC view and the feature map of the MLO view; obtain a CC view difference map according to the feature map of the CC view and the target CC feature map, obtain an MLO view difference map according to the feature map of the MLO view and the target MLO feature map, and perform patch embedding and position encoding on the difference map to obtain a CC vector and an MLO vector; input the CC vector into a second encoder to obtain a CC encoding vector, and input the MLO vector into a third encoder to obtain an MLO encoding vector; The prediction module is used to fuse the CC coding vector, the MLO coding vector and the personal information coding vector to obtain a target feature vector, and use the target feature vector as the input of the decoder to obtain a prediction result.

7. The system according to claim 6, characterized in that The CC view difference map is obtained according to the feature map of the CC view and the target CC feature map, and the MLO view difference map is obtained according to the feature map of the MLO view and the target MLO feature map, specifically: Subtract the target CC feature map from the CC view feature map to obtain the first tensor, normalize all negative values ​​in the first tensor to [0, 0.5], normalize 0 to 0.5, and normalize positive values ​​to (0.5, 1], thereby obtaining the CC view difference map; The feature map of the MLO view is subtracted from the target MLO feature map to obtain the first tensor, all negative values ​​in the first tensor are normalized to [0, 0.5), 0 is normalized to 0.5, and positive values ​​are normalized to (0.5, 1] ​​to obtain the MLO view difference map.

8. The system according to claim 7, characterized in that The difference map is patched and position-encoded to obtain a CC vector and an MLO vector, specifically: The CC view difference map and the MLO view difference map are divided into multiple image blocks, respectively, and the image blocks with all values ​​of 0.5 are deleted. Then, the remaining image blocks are subjected to patch embedding and position encoding to obtain the CC vector and MLO vector.

9. The system according to any one of claims 6 to 8, characterized in that: The CC coding vector, the MLO coding vector and the personal information coding vector are fused to obtain the target feature vector, specifically: Normalize the personal information coding vector to [0,1], and concatenate the CC coding vector, the MLO coding vector, and the normalized personal information coding vector; The concatenated vector is used as the input of the fully connected layer, and the output of the fully connected layer is the target feature vector; wherein, the dimension of the target feature vector is the same as that of the personal information encoding vector.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the method according to any one of claims 1 to 5.

Images