Image evaluation method and device based on neural network

Abstract

The present application proposes a neural network-based image evaluation method, which relates to the field of image processing technology, wherein the method comprises: obtaining a pre-exposure image of a target object; pre-processing the pre-exposure image, inputting the pre-processed pre-exposure image into a pre-trained first neural network, and obtaining a first target parameter in the pre-exposure image; updating the initial exposure parameter according to the first target parameter to obtain a target exposure parameter, and obtaining a formal exposure image of the target object according to the target exposure parameter; pre-processing the formal exposure image, inputting the pre-processed formal exposure image into a pre-trained second neural network, and obtaining a second target parameter in the formal exposure image. The present invention adopting the above scheme improves the image exposure efficiency, and improves the image evaluation accuracy while ensuring the image quality.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a neural network-based image evaluation method and device. Background Art

[0002] Early detection is a decisive factor in the successful treatment of breast cancer. Therefore, for many years, medical institutions have recommended the use of mammography for regular screening of asymptomatic women. Currently, the main screening method for breast lesions is mammography. In recent years, deep learning has been used to classify and recognize mammography scan images. Quantitative evaluation of female mammography images is mainly used to guide the optimization of mammography exposure parameter settings, which can reduce the radiation risk to patients while ensuring image quality.

[0003] At present, the problem of evaluating breast X-ray scanning images is very important and needs to be solved urgently. Most of the existing image evaluation methods use scales to evaluate and classify images. However, no matter which scale is used, the subjective evaluation of images has a common disadvantage, that is, it is not reliable and repeatable, and the image segmentation is mainly manual. The segmentation result depends largely on the operator's experience and skills. The degree of intelligence and automation is low, which greatly affects the evaluation efficiency. Summary of the invention

[0004] The present application aims to solve one of the technical problems in the related art at least to some extent.

[0005] To this end, the first purpose of this application is to propose an image evaluation method based on a neural network, which solves the problem of existing methods in improving the efficiency of image exposure and improves the evaluation accuracy while ensuring image quality.

[0006] The second objective of the present application is to provide an image evaluation device based on a neural network.

[0007] To achieve the above-mentioned objectives, the first aspect of the present application proposes a neural network-based image evaluation method, including: obtaining a pre-exposure image of a target object; preprocessing the pre-exposure image, inputting the preprocessed pre-exposure image into a pre-trained first neural network, and obtaining a first target parameter in the pre-exposure image; updating the initial exposure parameter according to the first target parameter to obtain a target exposure parameter, and obtaining a formal exposure image of the target object according to the target exposure parameter; preprocessing the formal exposure image, inputting the preprocessed formal exposure image into a pre-trained second neural network, and obtaining a second target parameter in the formal exposure image.

[0008] Optionally, in one embodiment of the present application, preprocessing the pre-exposure image, or preprocessing the formal exposure image, includes:

[0009] For the pre-exposure image or the formal exposure image, a first region and a second region are separated from each image, wherein the second region is a region in the image other than the first region;

[0010] Use histogram equalization to adjust the intensity distribution of all images to the same range, and adjust the resolution of all images to a preset resolution;

[0011] Generate a mean image and subtract the mean from each image so that the mean of the feature pixels of each image is zero.

[0012] Optionally, in one embodiment of the present application, before inputting the pre-processed pre-exposure image into the pre-trained first neural network, the method further includes:

[0013] A training data set is collected, where the training data set includes a number of paired training images, and training layer images and training pre-exposure images corresponding to the training images.

[0014] Using the training data set to train the deep learning network model to obtain the trained first neural network and the second neural network, wherein using the training data set to train the deep learning network model includes:

[0015] Perform image processing and segmentation on the training layer images in the training data set to obtain the numerical values ​​of the target parameters as the labels of the training data;

[0016] Preprocessing the training pre-exposure images and the training images;

[0017] According to the generated labels, the preprocessed training pre-exposure images and training images are respectively input into the deep learning network model for training to obtain the trained first neural network and the second neural network.

[0018] To achieve the above-mentioned purpose, the second embodiment of the present invention proposes an image evaluation device based on a neural network, including: an acquisition module, a first target parameter generation module, a formal exposure image generation module, and a second target parameter generation module, wherein:

[0019] An acquisition module, used for acquiring a pre-exposure image of a target object;

[0020] A first target parameter generating module, configured to preprocess the pre-exposure image using the preprocessing module, and input the preprocessed pre-exposure image into a pre-trained first neural network to obtain a first target parameter in the pre-exposure image;

[0021] A formal exposure image generation module, used to update the initial exposure parameter according to the first target parameter to obtain a target exposure parameter, and obtain a formal exposure image of the target object according to the target exposure parameter;

[0022] The second target parameter generating module is used to preprocess the formal exposure image using the preprocessing module, input the preprocessed formal exposure image into the pre-trained second neural network, and obtain the second target parameter in the formal exposure image.

[0023] Optionally, in one embodiment of the present application, the preprocessing module is used to:

[0024] For the pre-exposure image or the formal exposure image, a first region and a second region are separated from each image, wherein the second region is a region in the image other than the first region;

[0025] Use histogram equalization to adjust the intensity distribution of all images to the same range, and adjust the resolution of all images to a preset resolution;

[0026] Generate a mean image and subtract the mean from each image so that the mean of the feature pixels of each image is zero.

[0027] Optionally, in one embodiment of the present application, a training module is further included, for training the first neural network and the second neural network before inputting the pre-processed pre-exposure image into the pre-trained first neural network, wherein training the first neural network and the second neural network includes:

[0028] A training data set is collected, where the training data set includes a number of paired training images, and training layer images and training pre-exposure images corresponding to the training images.

[0029] Using the training data set to train the deep learning network model to obtain the trained first neural network and the second neural network, wherein using the training data set to train the deep learning network model includes:

[0030] Perform image processing and segmentation on the training layer images in the training data set to obtain the numerical values ​​of the target parameters as the labels of the training data;

[0031] Preprocessing the training pre-exposure images and the training images;

[0032] According to the generated labels, the preprocessed training pre-exposure images and training images are respectively input into the deep learning network model for training to obtain the trained first neural network and the second neural network.

[0033] In order to achieve the above-mentioned purpose, a third aspect of the present invention proposes a non-temporary computer-readable storage medium, which, when the instructions in the storage medium are executed by a processor, enables the execution of the above-mentioned neural network-based image evaluation method.

[0034] The neural network-based image evaluation method of the present application obtains a pre-exposure image of a target object; pre-processes the pre-exposure image, inputs the pre-processed pre-exposure image into a pre-trained first neural network, and obtains a first target parameter in the pre-exposure image; updates the initial exposure parameter according to the first target parameter to obtain a target exposure parameter, and obtains a formal exposure image of the target object according to the target exposure parameter; pre-processes the formal exposure image, inputs the pre-processed formal exposure image into a pre-trained second neural network, and obtains a second target parameter in the formal exposure image, thereby improving the exposure efficiency of the image and improving the evaluation accuracy while ensuring the image quality.

[0035] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0037] Figure 1 A flowchart of a neural network-based image evaluation method provided in Example 1 of the present application;

[0038] Figure 2 A flow chart of a method for evaluating glandular density of mammary gland X-ray pre-exposure and formal exposure images provided in an embodiment of the present application;

[0039] Figure 3 A flowchart of the application phase of glandular density assessment applicable to mammography pre-exposure and formal exposure images according to an embodiment of the present application;

[0040] Figure 4 A schematic diagram of a glandular density assessment training network structure applicable to mammographic pre-exposure and formal exposure images in an embodiment of the present application;

[0041] Figure 5 A schematic diagram of the structure of a neural network-based image evaluation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0043] The following describes the neural network-based image evaluation method and device according to an embodiment of the present application with reference to the accompanying drawings.

[0044] Figure 1 A flowchart of a neural network-based image evaluation method provided in Example 1 of the present application.

[0045] like Figure 1 As shown, the neural network-based image evaluation method includes the following steps:

[0046] Step 101, obtaining a pre-exposure image of a target object;

[0047] Step 102, preprocessing the pre-exposure image, inputting the pre-processed pre-exposure image into a pre-trained first neural network, and obtaining a first target parameter in the pre-exposure image;

[0048] Step 103, updating the initial exposure parameter according to the first target parameter to obtain a target exposure parameter, and acquiring a formal exposure image of the target object according to the target exposure parameter;

[0049] Step 104 , preprocessing the formal exposure image, inputting the preprocessed formal exposure image into a pre-trained second neural network, and obtaining a second target parameter in the formal exposure image.

[0050] The first target parameter and the second target parameter belong to the same indicator.

[0051] The neural network-based image evaluation method of the embodiment of the present application obtains a pre-exposure image of a target object; pre-processes the pre-exposure image, and inputs the pre-processed pre-exposure image into a pre-trained first neural network to obtain a first target parameter in the pre-exposure image; updates the initial exposure parameter according to the first target parameter to obtain a target exposure parameter, and obtains a formal exposure image of the target object according to the target exposure parameter; pre-processes the formal exposure image, and inputs the pre-processed formal exposure image into a pre-trained second neural network to obtain a second target parameter in the formal exposure image, thereby improving the exposure efficiency of the picture and improving the evaluation accuracy while ensuring the image quality.

[0052] Further, in the embodiment of the present application, preprocessing the pre-exposure image, or preprocessing the formal exposure image, includes:

[0053] For the pre-exposure image or the formal exposure image, a first region and a second region are separated from each image, wherein the second region is a region in the image other than the first region;

[0054] The intensity distribution of all images was adjusted to the same range using histogram equalization. All images were resized to a smaller resolution for higher computational efficiency;

[0055] Generate a mean image and subtract the mean from each image to ensure that the mean of each feature pixel is zero.

[0056] Specifically, in the embodiment of the present application, before inputting the pre-processed pre-exposure image into the pre-trained first neural network, the method further includes:

[0057] A training data set is collected, where the training data set includes a number of paired training images, and training layer images and training pre-exposure images corresponding to the training images.

[0058] Using the training data set to train the deep learning network model to obtain the trained first neural network and the second neural network, wherein using the training data set to train the deep learning network model includes:

[0059] Perform image processing and segmentation on the training layer images in the training data set to obtain the numerical values ​​of the target parameters as the labels of the training data;

[0060] Preprocessing the training pre-exposure images and the training images;

[0061] According to the generated labels, the preprocessed training pre-exposure images and training images are respectively input into the deep learning network model for training to obtain the trained first neural network and the second neural network.

[0062] Taking breast gland density assessment as an example, the method of identifying and evaluating the dynamic characteristics of the convolutional neural network system in this application is introduced:

[0063] Breast tissue is mainly composed of fibroglandular tissue and adipose tissue, where fibroglandular tissue appears dense on mammographic (DM) images and adipose tissue appears almost transparent. In this context, the glandular density (MBD) of the breast refers to the relative mass proportion of the breast tissue occupied by fibroglandular tissue in mammographic X-ray images. Women with very high MBD may have a 4 to 6 times higher risk of breast cancer than women whose breast tissue is mainly adipose tissue. This may be caused by either etiological effects or masking effects. The etiological effect is reflected in the fact that breast cancer mainly occurs in the epithelial cells lining the mammary ducts, and a high breast MBD reflects that the breast is mainly composed of fibroglandular tissue, thus indicating an increased likelihood of developing breast cancer. The masking effect is due to the increased difficulty in detecting potential lesions in areas of high glandular density in mammographic X-ray images than in areas of adipose tissue, so the sensitivity is reduced. Therefore, supplementary screening for women with high breast density will be very necessary. The fifth edition of the American College of Radiology (ACR) Breast Imaging Reporting and Database System (BI-RADS) in 2013 divides the percentage of glandular content into: fatty type (glandular content <25%), fibroglandular type (glandular content approximately 25%-50%), heterogeneously dense type (glandular content approximately 51%-75%), and dense type (glandular content >75%). There are many methods for evaluating breast MBD, mainly based on qualitative evaluation of visual areas. Methods include BI-RADS classification scale, visual analog scale (VAS), and semi-automatic threshold segmentation algorithm. Many radiologists use other methods to classify breast components in mammographic X-ray images, such as Wolfe grade, Boyd's six-category method, Tabar pattern, and BI-RADS density grade.

[0064] Mammography is a two-dimensional imaging of the breast of the examinee through an X-ray tube after a certain target filtration, which is used to detect lesions such as masses and calcifications inside the breast of the examinee. At the same time, X-rays will also cause radiation risks to the glands. In order to reduce the radiation risk as much as possible and improve the quality of diagnostic images, it is particularly important to set appropriate beam parameters for the examinee when exposing the examinee, among which the main influence is the examinee's MBD. Therefore, it is very necessary to obtain the examinee's MBD before formal exposure during the photography process, so the research content such as MBD evaluation accuracy and evaluation efficiency is particularly important. At present, there are two main methods used for quantitative evaluation of MBD. One is that experienced physicians divide the 3D image model obtained by breast body layer synthesis photography layer by layer, establish a 3D model of the gland in the breast, and calculate the MBD; the other is to use software such as Volpara and Quantra to estimate the volume of dense fibroglandular and adipose tissue in the breast for the original breast X-ray photography image, so as to evaluate MBD.

[0065] The main disadvantages of existing breast density assessment technologies are:

[0066] 1) The prior art mentions the use of scales to assess and classify MBD. However, no matter which scale is used, the subjective assessment of MBD has a common shortcoming, namely, it lacks reliable repeatability.

[0067] 2) Gland segmentation in breast body layer synthetic photography exposure images is mainly based on manual division, and the division result depends largely on the operator's experience and skills. The degree of intelligence and automation is low, which greatly affects the diagnostic efficiency.

[0068] 3) The MBD assessment technology of Volpara and Quantra software cannot perform MBD assessment of the entire breast based on the local pre-exposure images of the breast, lacks guidance for the formal exposure process, and has a low degree of automation, which affects the exposure efficiency of breast X-ray photography.

[0069] 4) The current mainstream method of using neural networks for MBD assessment is mainly to classify MBD into scales, which cannot obtain accurate MBD values. In addition, this method is processed after the formal exposure image is obtained, and it is impossible to obtain the MBD beam parameters used to guide breast X-ray exposure before the formal exposure, which lacks clinical trial value.

[0070] Deep learning has been used for breast segmentation, breast lesion detection, breast mass detection, and breast mass segmentation. Various deep learning methods have been proposed for other breast cancer-related tasks, such as distinguishing benign masses from malignant masses and distinguishing masses from microcalcifications. Currently, breast MBD estimation methods based on deep learning have received increasing attention in recent years, but the number of published studies is small. A recent study used supervised deep learning to classify breast MBD into BI-RADS categories and showed good results.

[0071] Therefore, the method of identifying and evaluating the dynamic characteristics of the convolutional neural network system of the present application is applied to breast gland density assessment, and a deep learning method is used in breast gland density assessment to improve the image quality and diagnostic efficiency of breast X-ray photography.

[0072] Figure 2 A schematic diagram of a flow chart of a method for evaluating glandular density of mammary gland X-ray pre-exposure and formal exposure images provided in an embodiment of the present application,

[0073] like Figure 2 As shown, the glandular density assessment method applicable to mammographic pre-exposure and formal exposure images includes:

[0074] Acquire a breast image of the subject and perform pre-exposure to generate a pre-exposure image;

[0075] Performing image preprocessing on the pre-exposed image, and inputting the preprocessed image into the trained first neural network to complete the preliminary evaluation of the glandular density of the subject, and obtain the estimated glandular density;

[0076] Optimizing mammographic X-ray exposure parameters according to the estimated glandular density, and formally exposing the subject's breast image according to the optimized exposure parameters to obtain a formal exposure image, wherein the greater the estimated glandular density, the higher the mammographic X-ray exposure parameters, such as tube voltage and mAs, are;

[0077] The formal exposure image is preprocessed, and the preprocessed image is input into the trained second neural network to automatically evaluate and obtain the breast gland density MBD value.

[0078] Further, in the embodiment of the present application, preprocessing the pre-exposure image, or preprocessing the formal exposure image, includes:

[0079] In each image, the whole breast area is automatically separated from the non-breast area (i.e., air and pectoral muscle);

[0080] The intensity distribution of all images was adjusted to the same range using histogram equalization. All images were resized to a smaller resolution for higher computational efficiency;

[0081] Generate a mean image and subtract the mean from each input image to ensure that the mean of each feature pixel is zero.

[0082] Furthermore, in the embodiment of the present application, before the pre-processed pre-exposure image is input into the pre-trained first neural network, neural network training is also included. The training process is as follows: Figure 3 As shown, including:

[0083] Acquire several paired mammographic, breast tomosynthesis, and pre-exposure images;

[0084] Perform image processing and segmentation on the training layer images in the training data set to obtain the breast area (including the skin area, fat area and gland area) and non-breast area of ​​the training layer images, and stack the training layers layer by layer in the initial order to obtain a three-dimensional voxel model of the breast. Finally, the breast gland density is calculated based on the skin, gland and fat density of the breast as the label of the training data;

[0085] performing the above-mentioned preprocessing on the mammographic X-ray image and the pre-exposure image;

[0086] According to the generated labels, the pre-processed pre-exposure image and the mammographic X-ray image are respectively input into the deep learning network model for training to obtain the trained first neural network and the second neural network. The structure of the first neural network is as follows: Figure 4 As shown, it includes an input layer, an average pooling layer, a first convolutional layer, a second convolutional layer, a first maximum pooling layer, a third convolutional layer, a second maximum pooling layer, a fourth convolutional layer, a third maximum pooling layer and a fully connected layer. The convolution kernel in the average pooling layer is 4×4; the convolution kernel in the first convolutional layer is 2×2, and the output image feature of this layer is 32; the convolution kernel in the second convolutional layer is 4×4, and the output image feature of this layer is 128; the convolution kernel in the first maximum pooling layer is 2×2; the convolution kernel in the third convolutional layer is 4×4, and the output image feature of this layer is 256; the convolution kernel in the second maximum pooling layer is 2×2; the convolution kernel in the fourth convolutional layer is 4×4, and the output image feature of this layer is 512; the convolution kernel in the third maximum pooling layer is 2×2. The structure of the second neural network is exactly the same as the first neural network.

[0087] In this embodiment, although the use of craniocaudal (CC) or mediolateral oblique (MLO) view mammography images is generally described. As long as the dataset is changed for training, the network can also be applied to other views, such as mediolateral (ML), lateral medial (LM) and other views irradiated by the device. The premise of using this technology is that the images used in the final test are consistent with the views of the dataset.

[0088] The method of identifying and evaluating the dynamic characteristics of the convolutional neural network system of the present application can improve the exposure efficiency when evaluating breast glandular density, reduce the radiation risk to the subject while ensuring image quality, and re-evaluate the glandular density of the photographic image after formal exposure to provide an exposure reference for the next breast X-ray screening. The artificial intelligence method is used to achieve accurate estimation of MBD and interactive automatic adjustment of beam parameters during the formal exposure process, which is convenient for improving the image quality and diagnostic efficiency of breast X-ray photography.

[0089] In order to implement the above embodiment, the present application also proposes an image evaluation device based on a neural network.

[0090] Figure 5 A schematic diagram of the structure of a neural network-based image evaluation device provided in an embodiment of the present application.

[0091] like Figure 5 As shown, the neural network-based image evaluation device includes an acquisition module, a first target parameter generation module, a formal exposure image generation module, and a second target parameter generation module, wherein:

[0092] An acquisition module, used for acquiring a pre-exposure image of a target object;

[0093] A first target parameter generating module, configured to preprocess the pre-exposure image using the preprocessing module, and input the preprocessed pre-exposure image into a pre-trained first neural network to obtain a first target parameter in the pre-exposure image;

[0094] A formal exposure image generation module, used to update the initial exposure parameter according to the first target parameter to obtain a target exposure parameter, and obtain a formal exposure image of the target object according to the target exposure parameter;

[0095] The second target parameter generating module is used to preprocess the formal exposure image using the preprocessing module, input the preprocessed formal exposure image into the pre-trained second neural network, and obtain the second target parameter in the formal exposure image.

[0096] Furthermore, in the embodiment of the present application, the preprocessing module is used to:

[0097] For the pre-exposure image or the formal exposure image, a first region and a second region are separated from each image, wherein the second region is a region in the image other than the first region;

[0098] Use histogram equalization to adjust the intensity distribution of all images to the same range, and adjust the resolution of all images to a preset resolution;

[0099] Generate a mean image and subtract the mean from each image so that the mean of the feature pixels of each image is zero.

[0100] Specifically, in the embodiment of the present application, a training module is further included, which is used to train the first neural network and the second neural network before inputting the pre-processed pre-exposure image into the pre-trained first neural network, wherein the training of the first neural network and the second neural network includes:

[0101] A training data set is collected, where the training data set includes a number of paired training images, and training layer images and training pre-exposure images corresponding to the training images.

[0102] Using the training data set to train the deep learning network model to obtain the trained first neural network and the second neural network, wherein using the training data set to train the deep learning network model includes:

[0103] Perform image processing and segmentation on the training layer images in the training data set to obtain the numerical values ​​of the target parameters as the labels of the training data;

[0104] Preprocessing the training pre-exposure images and the training images;

[0105] According to the generated labels, the preprocessed training pre-exposure images and training images are respectively input into the deep learning network model for training to obtain the trained first neural network and the second neural network.

[0106] It should be noted that the aforementioned explanation of the embodiment of the neural network-based image evaluation method is also applicable to the neural network-based image evaluation device of this embodiment, and will not be repeated here.

[0107] In order to implement the above embodiments, the present invention further proposes a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method of the above embodiments is implemented.

[0108] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0109] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0110] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0111] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0112] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0113] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0114] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0115] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A neural network-based image evaluation method, characterized in that: The image is a mammographic X-ray image, and the method is used to assess breast glandular density, and the method comprises the following steps: Acquire a pre-exposure image of the target object; Preprocessing the pre-exposure image, and inputting the pre-processed pre-exposure image into a pre-trained first neural network to obtain an estimated gland density in the pre-exposure image; Optimizing mammographic X-ray exposure parameters according to the estimated glandular density, and obtaining a formal exposure image of the target object according to the optimized exposure parameters, wherein the mammographic X-ray exposure parameters are beam parameters, and the greater the estimated glandular density, the higher the optimized exposure parameters; Preprocessing the formal exposure image, inputting the preprocessed formal exposure image into a pre-trained second neural network to obtain breast density in the formal exposure image; The pre-processing of the pre-exposure image or the pre-processing of the formal exposure image includes: For the pre-exposure image or the formal exposure image, a first region and a second region are separated from each image, wherein the first region is a whole breast region, and the second region is a non-breast region, and the non-breast region includes air and pectoral muscles; Use histogram equalization to adjust the intensity distribution of all images to the same range, and adjust the resolution of all images to a preset resolution; Generate a mean image and subtract the mean from each image so that the mean of the feature pixels of each image is zero; Before inputting the pre-processed pre-exposure image into the pre-trained first neural network, the method further includes: Acquiring a training data set, the training data set comprising a plurality of paired mammographic images, breast tomosynthesis images and pre-exposure images, Using the training data set to train the deep learning network model to obtain a trained first neural network and a trained second neural network, wherein using the training data set to train the deep learning network model includes: Performing image processing and segmentation on the training layer images in the training data set to obtain breast areas and non-breast areas of the training layer images, wherein the breast area includes a skin area, a fat area, and a gland area, and superimposing the training layers in an initial order to obtain a three-dimensional voxel model of the breast, and calculating the breast gland density according to the skin, gland, and fat densities of the breast as a label of the training data; performing the preprocessing on the pre-exposure image and the mammographic image; According to the generated labels, the pre-processed pre-exposure image and the breast X-ray photography image are respectively input into the deep learning network model for training to obtain the trained first neural network and the second neural network, wherein the first neural network and the second neural network have the same structure, and the first neural network includes an input layer, an average pooling layer, a first convolutional layer, a second convolutional layer, a first maximum pooling layer, a third convolutional layer, a second maximum pooling layer, a fourth convolutional layer, a third maximum pooling layer and a fully connected layer; the convolution kernel in the average pooling layer is 4×4; the convolution kernel in the first convolutional layer is 2×2, and the output image feature of this layer is 32; the convolution kernel in the second convolutional layer is 4×4, and the output image feature of this layer is 128; the convolution kernel in the first maximum pooling layer is 2×2; the convolution kernel in the third convolutional layer is 4×4, and the output image feature of this layer is 256; the convolution kernel in the second maximum pooling layer is 2×2; the convolution kernel in the fourth convolutional layer is 4×4, and the output image feature of this layer is 512; the convolution kernel in the third maximum pooling layer is 2×2.

2. A neural network-based image evaluation device, characterized in that: The image is a breast X-ray photography image, and the device is used to evaluate breast glandular density. The device includes an acquisition module, a first target parameter generation module, a formal exposure image generation module, and a second target parameter generation module, wherein: The acquisition module is used to acquire a pre-exposure image of the target object; The first target parameter generating module is used to preprocess the pre-exposure image using the preprocessing module, and input the preprocessed pre-exposure image into a pre-trained first neural network to obtain an estimated gland density in the pre-exposure image; The formal exposure image generation module is used to optimize the mammographic X-ray exposure parameters according to the estimated glandular density, and obtain the formal exposure image of the target object according to the optimized exposure parameters, wherein the mammographic X-ray exposure parameters are beam parameters, and the greater the estimated glandular density, the higher the optimized exposure parameters; The second target parameter generating module is used to preprocess the formal exposure image using the preprocessing module, and input the preprocessed formal exposure image into a pre-trained second neural network to obtain the gland density in the formal exposure image; Wherein, the preprocessing module is used to: For the pre-exposure image or the formal exposure image, a first region and a second region are separated from each image, wherein the first region is a whole breast region, and the second region is a non-breast region, and the non-breast region includes air and pectoral muscles; Use histogram equalization to adjust the intensity distribution of all images to the same range, and adjust the resolution of all images to a preset resolution; Generate a mean image and subtract the mean from each image so that the mean of the feature pixels of each image is zero; The method further includes a training module for training the first neural network and the second neural network before inputting the pre-processed pre-exposure image into the pre-trained first neural network, wherein the training of the first neural network and the second neural network includes: Acquiring a training data set, the training data set comprising a plurality of paired mammographic images, breast tomosynthesis images and pre-exposure images, Using the training data set to train the deep learning network model to obtain a trained first neural network and a trained second neural network, wherein using the training data set to train the deep learning network model includes: Performing image processing and segmentation on the training layer images in the training data set to obtain breast areas and non-breast areas of the training layer images, wherein the breast area includes a skin area, a fat area, and a gland area, and superimposing the training layers in an initial order to obtain a three-dimensional voxel model of the breast, and calculating the breast gland density according to the skin, gland, and fat densities of the breast as a label of the training data; performing the preprocessing on the pre-exposure image and the mammographic image; According to the generated labels, the pre-processed pre-exposure image and the breast X-ray photography image are respectively input into the deep learning network model for training to obtain the trained first neural network and the second neural network, wherein the first neural network and the second neural network have the same structure, and the first neural network includes an input layer, an average pooling layer, a first convolutional layer, a second convolutional layer, a first maximum pooling layer, a third convolutional layer, a second maximum pooling layer, a fourth convolutional layer, a third maximum pooling layer and a fully connected layer; the convolution kernel in the average pooling layer is 4×4; the convolution kernel in the first convolutional layer is 2×2, and the output image feature of this layer is 32; the convolution kernel in the second convolutional layer is 4×4, and the output image feature of this layer is 128; the convolution kernel in the first maximum pooling layer is 2×2; the convolution kernel in the third convolutional layer is 4×4, and the output image feature of this layer is 256; the convolution kernel in the second maximum pooling layer is 2×2; the convolution kernel in the fourth convolutional layer is 4×4, and the output image feature of this layer is 512; the convolution kernel in the third maximum pooling layer is 2×2.

3. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method as claimed in claim 1 is implemented.

Images