Breast cancer analysis and prediction system and method based on artificial intelligence

Abstract

The present disclosure relates to a breast cancer analysis and prediction system and method based on artificial intelligence. The method comprises: obtaining a breast X-ray image of a detected patient object; extracting X-ray image features of the breast X-ray image to obtain a breast X-ray image feature map; performing feature constraints and feature significant processing on the breast X-ray image feature map to obtain a significant breast X-ray image feature map; and determining a prediction result based on the significant breast X-ray image feature map. In this way, the risk level of breast cancer of the detected patient object can be intelligently identified, assisting doctors in making more accurate diagnoses.

Description

Technical Field

[0001] The present disclosure relates to the technical field of breast cancer analysis, and in particular, to a breast cancer analysis and prediction system and method based on artificial intelligence. Background Art

[0002] Breast cancer is one of the most common cancers among women worldwide. Early detection and diagnosis are crucial to improving patients' survival rates. However, traditional breast cancer screening usually relies on doctors' interpretation of mammograms, which is highly subjective and leads to inconsistent diagnostic results, and may even lead to misdiagnosis and wrong diagnosis.

[0003] In recent years, artificial intelligence (AI) has been increasingly used in the medical field, especially in image recognition and analysis. That is, with the development and application of artificial intelligence, a new idea is provided to solve the above technical problems. Summary of the invention

[0004] This summary is provided to introduce concepts in a brief form that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0005] In a first aspect, the present disclosure provides a breast cancer analysis and prediction system based on artificial intelligence, the system comprising:

[0006] A mammary gland X-ray image acquisition module, used for acquiring a mammary gland X-ray image of a detected patient object;

[0007] An image feature extraction module, used for extracting X-ray image features of the mammary gland X-ray image to obtain a mammary gland X-ray image feature map;

[0008] A feature constraint and manifestation module, used for performing feature constraint and feature salient processing on the mammary X-ray image feature map to obtain a salient mammary X-ray image feature map;

[0009] The prediction result generating module is used to determine the prediction result based on the significant breast X-ray image feature map.

[0010] Optionally, the image feature extraction module includes: a brightness compensation unit, used to perform brightness compensation on the breast X-ray image to obtain a brightness compensated breast X-ray image; and a feature extraction unit, used to perform feature extraction on the brightness compensated breast X-ray image using a deep learning network model to obtain a feature map of the breast X-ray image.

[0011] Optionally, the feature extraction unit is used to: pass the brightness compensated breast X-ray image through an image feature extractor based on a convolutional neural network model to obtain the breast X-ray image feature map.

[0012] Optionally, the feature constraint and manifestation module includes: a feature constraint and enhancement unit, used to input the breast X-ray image feature map into a feature map enhancer based on a reparameterized layer to obtain an enhanced breast X-ray image feature map; a feature manifestation unit, used to input the enhanced breast X-ray image feature map into a feature salient device based on a convolutional kernel attention network to obtain the salient breast X-ray image feature map.

[0013] Optionally, the feature constraint and enhancement unit is used to: process the mammary X-ray image feature map using the following reparameterization formula to obtain the enhanced mammary X-ray image feature map; wherein the reparameterization formula is:

[0014]

[0015] in, is the mean value of the mammographic image feature map, is the variance of the mammographic image feature map, ∈ θ is the θth value randomly sampled from a Gaussian distribution, is the θth eigenvalue in the enhanced breast X-ray image feature map.

[0016] Optionally, the feature manifestation unit includes: a multi-scale representation feature extraction sub-unit, used to extract multi-scale nonlinear convolution global representation features of the breast X-ray image feature map to obtain a concentrated global representation feature vector; and an attention application sub-unit, used to perform attention application processing based on the concentrated global representation feature vector to obtain the significant breast X-ray image feature map.

[0017] Optionally, the multi-scale characterization feature extraction subunit is used to: perform a nonlinear convolution operation on the breast X-ray image feature map through a multi-scale convolution kernel group to obtain a first nonlinear convolution feature map, a second nonlinear convolution feature map, a third nonlinear convolution feature map and a fourth nonlinear convolution feature map; and process the first nonlinear convolution feature map, the second nonlinear convolution feature map, the third nonlinear convolution feature map and the fourth nonlinear convolution feature map according to the following global representation formula to obtain a global characterization feature vector; wherein the global representation formula is:

[0018] U=U1+U2+U3+U4

[0019]

[0020] Among them, U1, U2, U3 and U4 are the first nonlinear convolution feature map, the second nonlinear convolution feature map, the third nonlinear convolution feature map and the fourth nonlinear convolution feature map respectively, U is a multi-scale nonlinear convolution fusion feature map, + represents the element-by-element addition and summation processing, S c represents the cth eigenvalue in the global characterization eigenvector, F gp (U c ) represents the global mean pooling processing of the c-th feature matrix in the multi-scale nonlinear convolution fusion feature map, H and W are the height and width of the c-th feature matrix in the multi-scale nonlinear convolution fusion feature map, respectively, and U c (i, j) is the eigenvalue at the position (i, j) in the cth feature matrix in the multi-scale nonlinear convolution fusion feature map; the global representation feature vector is activated to obtain the concentrated global representation feature vector.

[0021] Optionally, the attention applying subunit is used to: process the first nonlinear convolution feature map, the second nonlinear convolution feature map, the third nonlinear convolution feature map, the fourth nonlinear convolution feature map and the concentrated global representation feature vector using the following attention weight applying formula to obtain the significant breast X-ray image feature map; wherein the attention weight applying formula is:

[0022]

[0023]

[0024]

[0025]

[0026] a c +b c +c c +d c =1

[0027] F c =a c U1+b c U2+c c U3+d c U4

[0028] Among them, a c 、b c 、c c d crespectively represent the weight probability vectors of the first nonlinear convolution feature map U1, the second nonlinear convolution feature map U2, the third nonlinear convolution feature map U3 and the fourth nonlinear convolution feature map U4, Acz represents the vector element of the c channel corresponding to the first nonlinear convolution feature map in the concentrated global representation feature vector, Bcz, Ccz and Dcz respectively represent the vector elements of the c channel corresponding to the second nonlinear convolution feature map, the third nonlinear convolution feature map and the fourth nonlinear convolution feature map in the concentrated global representation feature vector, F c is the significant breast X-ray image feature map, and + represents the element-by-element addition and summation processing.

[0029] Optionally, the prediction result generation module includes: a feature distribution optimization unit, which is used to optimize the feature distribution of the significant breast X-ray image feature map to obtain an optimized significant breast X-ray image feature map; a result identification and division unit, which passes the optimized significant breast X-ray image feature map through a classifier-based breast cancer analysis predictor to obtain the prediction result, and the prediction result is used to represent the risk level label of breast cancer.

[0030] In a second aspect, the present disclosure provides a breast cancer analysis and prediction method based on artificial intelligence, the method comprising:

[0031] Acquiring a mammographic image of a patient subject;

[0032] Extracting X-ray image features of the mammary gland X-ray image to obtain a mammary gland X-ray image feature map;

[0033] Performing feature constraint and feature saliency processing on the mammary gland X-ray image feature map to obtain a saliency mammary gland X-ray image feature map;

[0034] Based on the significant breast X-ray image feature map, a prediction result is determined.

[0035] By adopting the above technical scheme, by acquiring a breast X-ray image of a detected patient object; extracting the X-ray image features of the breast X-ray image to obtain a breast X-ray image feature map; performing feature constraints and feature significant processing on the breast X-ray image feature map to obtain a significant breast X-ray image feature map; and determining a prediction result based on the significant breast X-ray image feature map, it is possible to intelligently identify the risk level of breast cancer of the detected patient object and assist doctors in making a more accurate diagnosis.

[0036] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale. In the drawings:

[0038] Figure 1 It is a block diagram of a breast cancer analysis and prediction system based on artificial intelligence according to an exemplary embodiment.

[0039] Figure 2 The present invention is a flowchart of a method for analyzing and predicting breast cancer based on artificial intelligence according to an exemplary embodiment.

[0040] Figure 3 It is a block diagram of an electronic device according to an exemplary embodiment.

[0041] Figure 4 This is an application scenario diagram of a breast cancer analysis and prediction system based on artificial intelligence according to an exemplary embodiment. DETAILED DESCRIPTION

[0042] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0043] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0044] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0045] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0046] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0047] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0048] Breast cancer is a malignant tumor that seriously endangers women's health. Early detection and treatment are the key to improving patients' survival rate and quality of life. However, current breast cancer diagnostic methods have certain limitations. For example, the diagnosis of breast cancer depends largely on the experience and judgment of doctors, and the experience and judgment of different doctors may differ, resulting in inconsistent diagnostic results. There is currently a lack of objective and unified diagnostic criteria for breast cancer. Different diagnostic criteria may lead to different diagnostic results, affecting patients' treatment plans and prognosis. The diagnosis of breast cancer is easily affected by human factors, such as doctors' subjective judgment, patients' descriptions, and the accuracy of examination equipment, which may lead to misdiagnosis or missed diagnosis. The currently used breast cancer diagnostic methods, such as mammography and breast ultrasound, are not sensitive and specific enough. Insufficient sensitivity may lead to missed diagnosis, while insufficient specificity may lead to misdiagnosis. Some breast cancer diagnostic methods, such as biopsy, may cause discomfort or pain to patients, which may affect patients' compliance and lead to delayed diagnosis. Some breast cancer diagnostic methods, such as magnetic resonance imaging (MRI) examinations, are expensive, which may bring financial burden to patients and affect their diagnosis and treatment. In some cases, there may be uncertainty in the diagnosis of breast cancer, which may cause anxiety and worry to patients and affect their mental health.

[0049] To solve these problems, this paper proposes an artificial intelligence-based breast cancer analysis and prediction system, which uses deep learning technology to extract features from breast X-ray images, establish a classification model, and realize automatic detection and grading of breast cancer.

[0050] The system also combines clinical data and genetic data to construct a multivariate regression model to predict the prognosis and treatment effect of breast cancer patients. The system has the advantages of high accuracy, high efficiency, and high interpretability, providing a powerful tool for early diagnosis and personalized treatment of breast cancer.

[0051] In view of the above technical problems, the technical concept of the present application is to use image processing methods and intelligent algorithms based on artificial intelligence technology to extract and capture significant feature information related to breast cancer from the breast X-ray images of the detected patient object, so as to intelligently identify the risk level of breast cancer of the detected patient object. In this way, it can assist doctors in making more accurate diagnoses.

[0052] In order to solve the above problems, the present disclosure provides a breast cancer analysis and prediction system and method based on artificial intelligence, which obtains a breast X-ray image of a detected patient object; extracts the X-ray image features of the breast X-ray image to obtain a breast X-ray image feature map; performs feature constraints and feature significant processing on the breast X-ray image feature map to obtain a significant breast X-ray image feature map; and determines a prediction result based on the significant breast X-ray image feature map. In this way, the risk level of breast cancer of the detected patient object can be intelligently identified, assisting doctors in making more accurate diagnoses.

[0053] The specific embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0054] Figure 1 FIG. 1 is a block diagram of a breast cancer analysis and prediction system based on artificial intelligence according to an exemplary embodiment. Figure 1 As shown, the breast cancer analysis and prediction system 100 includes:

[0055] A mammary gland X-ray image acquisition module 101 is used to acquire a mammary gland X-ray image of a detected patient object;

[0056] An image feature extraction module 102, used to extract X-ray image features of the breast X-ray image to obtain a breast X-ray image feature map;

[0057] A feature constraint and manifestation module 103, used for performing feature constraint and feature significance processing on the mammary X-ray image feature map to obtain a significant mammary X-ray image feature map;

[0058] The prediction result generating module 104 is used to determine the prediction result based on the significant breast X-ray image feature map.

[0059] The mammary gland X-ray image acquisition module 101 can obtain high-quality mammary gland X-ray images, provide a good foundation for subsequent image processing and analysis, reduce the risk of misdiagnosis and missed diagnosis, and improve the detection rate of breast cancer. The image feature extraction module 102 selects a suitable feature extraction algorithm to extract features that can effectively distinguish breast cancer from normal tissues, ensure the robustness of the feature extraction algorithm, and be able to cope with various changes in mammary gland X-ray images, such as noise, artifacts, etc. The features that can effectively distinguish breast cancer from normal tissues are extracted to provide a basis for subsequent feature constraints and manifestation, and improve the diagnostic accuracy of breast cancer. The feature constraint and manifestation module 103 selects a suitable feature constraint and manifestation algorithm to enhance the significance of breast cancer-related features and suppress the interference of irrelevant features. Ensure the effectiveness and robustness of the feature constraint and manifestation algorithm, and be able to cope with various changes in mammary gland X-ray images. Enhance the significance of breast cancer-related features, suppress the interference of irrelevant features, improve the detection rate of breast cancer, reduce the risk of misdiagnosis and missed diagnosis, and improve the diagnostic accuracy of breast cancer. The prediction result generation module 104 selects a suitable classifier or regression model to generate a prediction result based on the significant breast X-ray image feature map. It ensures that the performance of the classifier or regression model is good and can accurately distinguish breast cancer from normal tissue. By generating accurate prediction results based on the significant breast X-ray image feature map, the diagnosis accuracy of breast cancer is improved, and doctors are assisted in the diagnosis of breast cancer, thereby improving the efficiency and accuracy of diagnosis.

[0060] Based on this, in the technical solution of the present application, first, a mammographic X-ray image of the patient being tested is obtained. Here, as a common medical imaging examination method, X-ray imaging technology can display the structure and density of the breast tissue of the patient being tested. At the same time, this information will be reflected in the mammographic X-ray image.

[0061] In one embodiment of the present disclosure, the image feature extraction module includes: a brightness compensation unit, used to perform brightness compensation on the breast X-ray image to obtain a brightness compensated breast X-ray image; and a feature extraction unit, used to perform feature extraction on the brightness compensated breast X-ray image using a deep learning network model to obtain a feature map of the breast X-ray image.

[0062] Considering that the breast X-ray image may be affected by environmental factors during the acquisition process, resulting in uneven brightness of the breast X-ray image. It should be understood that during the imaging process of the breast X-ray image, its display device is usually not linear. This means that a small change in the input signal may lead to a large change in the output brightness. For example, some small noise generated by environmental factors will be amplified in the breast X-ray image. This will affect the subsequent model's analysis and interpretation of the breast X-ray image. Therefore, in order to alleviate or solve this problem, in the technical solution of the present application, it is expected to perform brightness compensation on the breast X-ray image to adjust the overall brightness display of the breast X-ray image, thereby obtaining a breast X-ray image after brightness compensation.

[0063] In a specific example of the present application, a specific encoding process of performing brightness compensation on the breast X-ray image to obtain a brightness compensated breast X-ray image includes: processing the breast X-ray image using the following brightness compensation formula to obtain the brightness compensated breast X-ray image; wherein the brightness compensation formula is:

[0064]

[0065] Wherein, L0 is the normalized result of converting the mammary X-ray image from the RGB space to the HSV space, A, B, C and D are adjustment parameters with different values, and L1 is the mammary X-ray image after brightness compensation. Here, the brightness compensation formula compensates for the nonlinear response of the display device of the mammary X-ray image during the imaging process by applying a gamma curve, thereby adjusting the darker or brighter areas in the mammary X-ray image to be improved accordingly, reducing the adverse effects caused by the brightness difference, and enhancing the detail information in the mammary X-ray image.

[0066] Next, the brightness-compensated mammary X-ray image is passed through an image feature extractor based on a convolutional neural network model to obtain a mammary X-ray image feature map. Among them, the convolutional neural network (CNN) model is a deep learning model that is widely used in image recognition and processing and has a strong feature learning ability. It can automatically learn the abstract features in the brightness-compensated mammary X-ray image, which helps to discover potential signs of breast cancer and lesion information.

[0067] In one embodiment of the present disclosure, the feature extraction unit is used to: pass the brightness compensated breast X-ray image through an image feature extractor based on a convolutional neural network model to obtain the breast X-ray image feature map.

[0068] Although the problem of poor image quality caused by factors such as light can be alleviated to a certain extent by brightness compensation. However, since breast tissue itself has complex structure and tissue characteristics, there may be significant differences in the density and morphology of breast tissue between different individuals and different parts, which will also cause some features to appear sparse or missing in the image. At the same time, various breast lesions (such as cysts, calcifications, masses, etc.) may exist in the breast X-ray image of the detected patient object. These lesions may block or change the original breast tissue characteristics, resulting in some features that cannot be accurately extracted or presented. Therefore, in the technical solution of the present application, it is expected that the breast X-ray image feature map is input into a feature map enhancer based on a reparameterization layer to learn a more representational and highly abstract feature representation in the breast X-ray image feature map, thereby obtaining an enhanced breast X-ray image feature map. Specifically, the feature map enhancer based on the reparameterization layer guides and standardizes the feature distribution by introducing a priori distribution, such as Gaussian random distribution. In this way, by introducing the reparameterization technique, the breast X-ray image feature map is made random after reconstruction, thereby cleverly achieving the effect of data enhancement in a high-dimensional feature space, so that subsequent models can make full use of all available information.

[0069] In one embodiment of the present disclosure, the feature constraint and manifestation module includes: a feature constraint and enhancement unit, which is used to input the breast X-ray image feature map into a feature map enhancer based on a reparameterization layer to obtain an enhanced breast X-ray image feature map; a feature manifestation unit, which is used to input the enhanced breast X-ray image feature map into a feature salient device based on a convolutional kernel attention network to obtain the salient breast X-ray image feature map.

[0070] Furthermore, the feature constraint and enhancement unit is used to: process the mammary X-ray image feature map using the following reparameterization formula to obtain the enhanced mammary X-ray image feature map; wherein the reparameterization formula is:

[0071]

[0072] in, is the mean value of the mammographic image feature map, is the variance of the mammographic image feature map, ∈ θ is the θth value randomly sampled from a Gaussian distribution, is the θth eigenvalue in the enhanced breast X-ray image feature map.

[0073] Furthermore, the enhanced breast X-ray image feature map is input into a feature saliency device based on a convolution kernel attention network to obtain a salient breast X-ray image feature map. The feature saliency device based on a convolution kernel attention network performs a nonlinear convolution operation on the enhanced breast X-ray image feature map through a multi-scale convolution kernel group to extract the associated features in different local spatial neighborhoods in the enhanced breast X-ray image feature map, thereby improving the expressiveness and discrimination of the features. Afterwards, the importance of both the channel and the convolution kernel is considered simultaneously, and the network is guided by weight information to assign different attentions to convolution kernels of different sizes. In this way, the feature distribution captured by the convolution kernel of useful features is extracted, and the goal of the convolution kernel that has learned useless background features or noise is ignored.

[0074] In one embodiment of the present disclosure, the feature manifestation unit includes: a multi-scale representation feature extraction sub-unit, used to extract multi-scale nonlinear convolution global representation features of the breast X-ray image feature map to obtain a concentrated global representation feature vector; an attention application sub-unit, used to perform attention application processing based on the concentrated global representation feature vector to obtain the significant breast X-ray image feature map.

[0075] Furthermore, the multi-scale characterization feature extraction subunit is used to: perform a nonlinear convolution operation on the breast X-ray image feature map through a multi-scale convolution kernel group to obtain a first nonlinear convolution feature map, a second nonlinear convolution feature map, a third nonlinear convolution feature map and a fourth nonlinear convolution feature map; and process the first nonlinear convolution feature map, the second nonlinear convolution feature map, the third nonlinear convolution feature map and the fourth nonlinear convolution feature map according to the following global representation formula to obtain a global characterization feature vector; wherein the global representation formula is:

[0076] U=U1+U2+U3+U4

[0077]

[0078] Among them, U1, U2, U3 and U4 are the first nonlinear convolution feature map, the second nonlinear convolution feature map, the third nonlinear convolution feature map and the fourth nonlinear convolution feature map respectively, U is a multi-scale nonlinear convolution fusion feature map, + represents the element-by-element addition and summation processing, S c represents the cth eigenvalue in the global characterization eigenvector, F gp (U c ) represents the global mean pooling processing of the c-th feature matrix in the multi-scale nonlinear convolution fusion feature map, H and W are the height and width of the c-th feature matrix in the multi-scale nonlinear convolution fusion feature map, respectively, and U c(i, j) is the eigenvalue at the position (i, j) in the cth feature matrix in the multi-scale nonlinear convolution fusion feature map; the global representation feature vector is activated to obtain the concentrated global representation feature vector.

[0079] Furthermore, the attention applying subunit is used to: process the first nonlinear convolution feature map, the second nonlinear convolution feature map, the third nonlinear convolution feature map, the fourth nonlinear convolution feature map and the concentrated global representation feature vector using the following attention weight applying formula to obtain the significant breast X-ray image feature map; wherein the attention weight applying formula is:

[0080]

[0081]

[0082]

[0083]

[0084] a c +b c +c c +d c =1

[0085] F c =a c U1+b c U2+c c U3+d c U4

[0086] Among them, a c 、b c 、c c d c respectively represent the weight probability vectors of the first nonlinear convolution feature map U1, the second nonlinear convolution feature map U2, the third nonlinear convolution feature map U3 and the fourth nonlinear convolution feature map U4, Acz represents the vector element of the c channel corresponding to the first nonlinear convolution feature map in the concentrated global representation feature vector, Bcz, Ccz and Dcz respectively represent the vector elements of the c channel corresponding to the second nonlinear convolution feature map, the third nonlinear convolution feature map and the fourth nonlinear convolution feature map in the concentrated global representation feature vector, F c is the significant breast X-ray image feature map, and + represents the element-by-element addition and summation processing.

[0087] In one embodiment of the present disclosure, the prediction result generation module includes: a feature distribution optimization unit, which is used to optimize the feature distribution of the significant breast X-ray image feature map to obtain an optimized significant breast X-ray image feature map; a result identification and division unit, which passes the optimized significant breast X-ray image feature map through a classifier-based breast cancer analysis predictor to obtain the prediction result, and the prediction result is used to represent a risk level label for breast cancer.

[0088] In the technical solution of the present application, each feature matrix of the enhanced breast X-ray image feature map expresses the enhanced image semantic features of the breast X-ray image after brightness compensation. In this way, after the enhanced breast X-ray image feature map is input into the feature saliency device based on the convolution kernel attention network, the spatial distribution of local semantic features within the feature matrix of the image semantic features will be enhanced based on the convolution kernel attention mechanism. This will not only enhance the image semantic feature representation of the significant breast X-ray image feature map, but also make the feature matrices of the significant breast X-ray image feature map have significant channel distribution difference representation in the channel dimension, so that the overall distribution of the significant breast X-ray image feature map in units of feature matrix feature distribution is reduced, thereby affecting the class probability convergence effect of the significant breast X-ray image feature map through the classifier, that is, affecting the speed of classification training and the accuracy of classification results.

[0089] Based on this, the applicant of the present application first converts each feature matrix in the significant breast X-ray image feature map into a square matrix through a linear transformation, and then optimizes the significant breast X-ray image feature map, which is specifically expressed as: converting each feature matrix in the significant breast X-ray image feature map into a square matrix through a linear transformation, and then optimizing the significant breast X-ray image feature map with the following optimization formula to obtain an optimized significant breast X-ray image feature map; wherein, the optimization formula is:

[0090]

[0091] Among them, M i and M i+1 are respectively the i-th and i+1-th feature matrices in the salient mammographic image feature map, and and They are the feature matrices M i and M i+1 The global mean, M i+1 T is the transposed matrix of the i+1th feature matrix in the salient mammographic image feature map, M i+1 -1 is the inverse of the i+1th feature matrix in the salient mammographic image feature map, M′ i+1 is the i+1th feature matrix in the optimized and salient breast X-ray image feature map, ⊙ represents the point multiplication by position, Represents matrix multiplication.

[0092] Here, by taking the center of the distribution of the feature matrix of the significant breast X-ray image feature map along the channel as the seed point of the scene transfer in the channel dimension, each eigenvalue of the feature matrix of the significant breast X-ray image feature map is robustly aggregated and sub-sampled through matrix multiplication, so that the distribution bounding box transfer directional constraints of adjacent feature matrices are performed on the basis of the participation of each eigenvalue of the feature matrix of the significant breast X-ray image feature map, so as to improve the integrity of the feature representation of the multi-scale feature map of the sample image on the basis of the overall bottom-up context association of the significant breast X-ray image feature map along the channel dimension, thereby improving the class probability convergence effect of the significant breast X-ray image feature map through the classifier, that is, improving the speed of classification training and the accuracy of classification results.

[0093] Then, the optimized significant breast X-ray image feature map is passed through a classifier-based breast cancer analysis predictor to obtain a prediction result, and the prediction result is used to represent a risk level label for breast cancer.

[0094] In summary, the above scheme uses an image processing method and intelligent algorithm based on artificial intelligence technology to extract and capture significant feature information related to breast cancer from the breast X-ray image of the detected patient object, thereby realizing intelligent identification of the risk level of breast cancer of the detected patient object. In this way, it assists doctors to make more accurate diagnosis.

[0095] Figure 2 is a flowchart of a method for analyzing and predicting breast cancer based on artificial intelligence according to an exemplary embodiment. Figure 2 As shown, the method includes:

[0096] Step 201, obtaining a breast X-ray image of a detected patient object;

[0097] Step 202: extracting X-ray image features of the breast X-ray image to obtain a breast X-ray image feature map;

[0098] Step 203, performing feature constraint and feature saliency processing on the mammary gland X-ray image feature map to obtain a saliency mammary gland X-ray image feature map;

[0099] Step 204: Determine a prediction result based on the significant breast X-ray image feature map.

[0100] Reference below Figure 3, which shows a schematic diagram of the structure of an electronic device 600 suitable for implementing the embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0101] like Figure 3 As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0102] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0103] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0104] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0105] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0106] The computer-readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0107] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0108] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0109] The modules involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a module does not limit the module itself in some cases. For example, a test parameter acquisition module may also be described as a "module for acquiring device test parameters corresponding to a target device".

[0110] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0111] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0112] According to one or more embodiments of the present disclosure, Example 1 provides a breast cancer analysis and prediction system based on artificial intelligence, the system comprising:

[0113] A mammary gland X-ray image acquisition module, used for acquiring a mammary gland X-ray image of a detected patient object;

[0114] An image feature extraction module, used for extracting X-ray image features of the mammary gland X-ray image to obtain a mammary gland X-ray image feature map;

[0115] A feature constraint and manifestation module, used for performing feature constraint and feature salient processing on the mammary X-ray image feature map to obtain a salient mammary X-ray image feature map;

[0116] The prediction result generating module is used to determine the prediction result based on the significant breast X-ray image feature map.

[0117] According to one or more embodiments of the present disclosure, Example 2 provides the system of Example 1, wherein the image feature extraction module comprises:

[0118] A brightness compensation unit, used for performing brightness compensation on the breast X-ray image to obtain a brightness compensated breast X-ray image;

[0119] A feature extraction unit is used to extract features of the brightness-compensated breast X-ray image using a deep learning network model to obtain the breast X-ray image feature map.

[0120] According to one or more embodiments of the present disclosure, Example 3 provides the system of Example 2, wherein the feature extraction unit is configured to:

[0121] The brightness-compensated breast X-ray image is passed through an image feature extractor based on a convolutional neural network model to obtain the breast X-ray image feature map.

[0122] According to one or more embodiments of the present disclosure, Example 4 provides the system of Example 3, wherein the feature constraint and manifestation module includes:

[0123] A feature constraint and enhancement unit, used for inputting the mammary gland X-ray image feature map into a feature map enhancer based on a re-parameterized layer to obtain an enhanced mammary gland X-ray image feature map;

[0124] The feature manifestation unit is used to input the enhanced breast X-ray image feature map into a feature salient device based on a convolution kernel attention network to obtain the salient breast X-ray image feature map.

[0125] According to one or more embodiments of the present disclosure, Example 5 provides the system of Example 4, wherein the feature constraint and enhancement unit is configured to:

[0126] The mammary X-ray image feature map is processed by the following reparameterization formula to obtain the enhanced mammary X-ray image feature map; wherein the reparameterization formula is:

[0127]

[0128] in, is the mean value of the mammographic image feature map, is the variance of the mammographic image feature map, ∈ θ is the θth value randomly sampled from a Gaussian distribution, is the θth eigenvalue in the enhanced breast X-ray image feature map.

[0129] According to one or more embodiments of the present disclosure, Example 6 provides the system of Example 5, wherein the feature manifestation unit includes:

[0130] A multi-scale representation feature extraction subunit, used for extracting multi-scale nonlinear convolution global representation features of the mammary X-ray image feature map to obtain a concentrated global representation feature vector;

[0131] The attention applying subunit is used to perform attention applying processing based on the concentrated global representation feature vector to obtain the salient breast X-ray image feature map.

[0132] According to one or more embodiments of the present disclosure, Example 7 provides the system of Example 6, wherein the multi-scale representation feature extraction subunit is used to:

[0133] Performing a nonlinear convolution operation on the breast X-ray image feature map through a multi-scale convolution kernel group to obtain a first nonlinear convolution feature map, a second nonlinear convolution feature map, a third nonlinear convolution feature map, and a fourth nonlinear convolution feature map;

[0134] The first nonlinear convolution feature map, the second nonlinear convolution feature map, the third nonlinear convolution feature map, and the fourth nonlinear convolution feature map are processed using the following global representation formula to obtain a global characterization feature vector; wherein the global representation formula is:

[0135] U=U1+U2+U3+U4

[0136]

[0137] Among them, U1, U2, U3 and U4 are the first nonlinear convolution feature map, the second nonlinear convolution feature map, the third nonlinear convolution feature map and the fourth nonlinear convolution feature map respectively, U is a multi-scale nonlinear convolution fusion feature map, + represents the element-by-element addition and summation processing, S c represents the cth eigenvalue in the global characterization eigenvector, F gp (U c ) represents the global mean pooling processing of the c-th feature matrix in the multi-scale nonlinear convolution fusion feature map, H and W are the height and width of the c-th feature matrix in the multi-scale nonlinear convolution fusion feature map, respectively, and U c (i, j) is the eigenvalue at position (i, j) in the c-th feature matrix of the multi-scale nonlinear convolution fusion feature map;

[0138] The global representation feature vector is activated to obtain the condensed global representation feature vector.

[0139] According to one or more embodiments of the present disclosure, Example 8 provides the system of Example 7, wherein the attention applying subunit is configured to:

[0140] The first nonlinear convolution feature map, the second nonlinear convolution feature map, the third nonlinear convolution feature map, the fourth nonlinear convolution feature map and the concentrated global representation feature vector are processed using the following attention weight application formula to obtain the significant breast X-ray image feature map; wherein the attention weight application formula is:

[0141]

[0142]

[0143]

[0144]

[0145] a c +b c +c c +dc =1

[0146] F c =a c U1+b c U2+c c U3+d c U4

[0147] Among them, a c 、b c 、c c d c respectively represent the weight probability vectors of the first nonlinear convolution feature map U1, the second nonlinear convolution feature map U2, the third nonlinear convolution feature map U3 and the fourth nonlinear convolution feature map U4, Acz represents the vector element of the c channel corresponding to the first nonlinear convolution feature map in the concentrated global representation feature vector, Bcz, Ccz and Dcz respectively represent the vector elements of the c channel corresponding to the second nonlinear convolution feature map, the third nonlinear convolution feature map and the fourth nonlinear convolution feature map in the concentrated global representation feature vector, F c is the significant breast X-ray image feature map, and + represents the element-by-element addition and summation processing.

[0148] According to one or more embodiments of the present disclosure, Example 9 provides the system of Example 8, wherein the prediction result generating module includes:

[0149] A feature distribution optimization unit, configured to optimize the feature distribution of the significant breast X-ray image feature map to obtain an optimized significant breast X-ray image feature map;

[0150] The result recognition and classification unit passes the optimized significant breast X-ray image feature map through a classifier-based breast cancer analysis predictor to obtain the prediction result, and the prediction result is used to represent the risk level label of breast cancer.

[0151] According to one or more embodiments of the present disclosure, Example 10 provides a breast cancer analysis and prediction method based on artificial intelligence, the method comprising:

[0152] Acquiring a mammographic image of a patient subject;

[0153] Extracting X-ray image features of the mammary gland X-ray image to obtain a mammary gland X-ray image feature map;

[0154] Performing feature constraint and feature saliency processing on the mammary gland X-ray image feature map to obtain a saliency mammary gland X-ray image feature map;

[0155] Based on the significant breast X-ray image feature map, a prediction result is determined.

[0156] Figure 4 FIG. 1 is an application scenario diagram of a breast cancer analysis and prediction system based on artificial intelligence according to an exemplary embodiment. Figure 4 As shown, in this application scenario, first, a breast X-ray image of a detected patient object is obtained (for example, Figure 4 Then, the acquired breast X-ray image is input to a server (e.g., Figure 4 In S) shown in , the server is capable of processing the breast X-ray image based on an artificial intelligence-based breast cancer analysis and prediction algorithm to determine a prediction result.

[0157] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0158] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0159] Although the subject matter has been described in language specific to structural features and / or method logic actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims. Regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Claims

1. A breast cancer analysis and prediction system based on artificial intelligence, characterized in that: include: A mammary gland X-ray image acquisition module, used for acquiring a mammary gland X-ray image of a detected patient object; An image feature extraction module, used for extracting X-ray image features of the mammary gland X-ray image to obtain a mammary gland X-ray image feature map; A feature constraint and manifestation module, used for performing feature constraint and feature salient processing on the mammary X-ray image feature map to obtain a salient mammary X-ray image feature map; A prediction result generating module, used for determining a prediction result based on the significant breast X-ray image feature map; Wherein, the image feature extraction module includes: A brightness compensation unit, used for performing brightness compensation on the breast X-ray image to obtain a brightness compensated breast X-ray image; A feature extraction unit, configured to extract features of the brightness-compensated breast X-ray image using a deep learning network model to obtain a feature map of the breast X-ray image; Wherein, the feature extraction unit is used to: Passing the brightness-compensated breast X-ray image through an image feature extractor based on a convolutional neural network model to obtain the breast X-ray image feature map; The feature constraint and manifestation module includes: A feature constraint and enhancement unit, used for inputting the mammary gland X-ray image feature map into a feature map enhancer based on a re-parameterized layer to obtain an enhanced mammary gland X-ray image feature map; A feature manifestation unit, configured to input the enhanced mammary X-ray image feature map into a feature salient device based on a convolution kernel attention network to obtain the salient mammary X-ray image feature map; Wherein, the feature constraint and enhancement unit is used to: The mammary X-ray image feature map is processed by the following reparameterization formula to obtain the enhanced mammary X-ray image feature map; wherein the reparameterization formula is: in, is the mean value of the mammographic image feature map, is the variance of the mammographic image feature map, is randomly sampled from a Gaussian distribution. values, is the first eigenvalues; Wherein, the feature manifestation unit includes: A multi-scale representation feature extraction subunit, used for extracting multi-scale nonlinear convolution global representation features of the mammary X-ray image feature map to obtain a concentrated global representation feature vector; an attention applying subunit, configured to perform attention applying processing based on the concentrated global representation feature vector to obtain the salient breast X-ray image feature map; Wherein, the multi-scale characterization feature extraction subunit is used to: Performing a nonlinear convolution operation on the breast X-ray image feature map through a multi-scale convolution kernel group to obtain a first nonlinear convolution feature map, a second nonlinear convolution feature map, a third nonlinear convolution feature map, and a fourth nonlinear convolution feature map; The first nonlinear convolution feature map, the second nonlinear convolution feature map, the third nonlinear convolution feature map, and the fourth nonlinear convolution feature map are processed using the following global representation formula to obtain a global characterization feature vector; wherein the global representation formula is: in, , , and are respectively the first nonlinear convolution feature map, the second nonlinear convolution feature map, the third nonlinear convolution feature map and the fourth nonlinear convolution feature map, is a multi-scale nonlinear convolution fusion feature map, It means the bitwise addition and summation of elements. represents the first eigenvalues, Represents the multi-scale nonlinear convolution fusion feature map The feature matrix is ​​processed by global mean pooling. and They are respectively the first The height and width of the feature matrix, is the first In the feature matrix The eigenvalue at the position; Performing activation processing on the global representation feature vector to obtain the concentrated global representation feature vector; Wherein, the attention applying subunit is used for: The first nonlinear convolution feature map, the second nonlinear convolution feature map, the third nonlinear convolution feature map, the fourth nonlinear convolution feature map and the concentrated global representation feature vector are processed using the following attention weight application formula to obtain the significant breast X-ray image feature map; wherein the attention weight application formula is: in, , , , Respectively represent the first nonlinear convolution feature map , the second nonlinear convolution feature map , the third nonlinear convolution feature map And the fourth nonlinear convolution feature map The weighted probability vector, represents the condensed global representation feature vector corresponding to the first nonlinear convolution feature map The vector elements of the channel, , and Then, the condensed global representation feature vectors corresponding to the second nonlinear convolution feature map, the third nonlinear convolution feature map, and the fourth nonlinear convolution feature map are represented respectively. The vector elements of the channel, is the significant breast X-ray image feature map, Indicates the bitwise addition and summation of elements; Wherein, the prediction result generation module includes: A feature distribution optimization unit, configured to optimize the feature distribution of the significant breast X-ray image feature map to obtain an optimized significant breast X-ray image feature map; The result recognition and classification unit passes the optimized significant breast X-ray image feature map through a classifier-based breast cancer analysis predictor to obtain the prediction result, and the prediction result is used to represent the risk level label of breast cancer.

2. A method for analyzing and predicting breast cancer based on artificial intelligence, using the breast cancer analysis and prediction system based on artificial intelligence according to claim 1, characterized in that: include: Acquiring a mammographic image of a patient subject; Extracting X-ray image features of the mammary gland X-ray image to obtain a mammary gland X-ray image feature map; Performing feature constraint and feature saliency processing on the mammary gland X-ray image feature map to obtain a saliency mammary gland X-ray image feature map; Based on the significant breast X-ray image feature map, a prediction result is determined.

Images