A neural network-based image processing method, system, terminal and storage medium
Patent Information
- Application Number
- CN202510957846.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-11
- Publication Date
- 2026-11-03
AI Technical Summary
现有技术中，AI对甲状腺超声图像的“黑箱”特性使其对甲状腺结节的良恶性分类过程缺乏透明度，医生难以理解AI对甲状腺结节的决策过程缺乏透明度，尤其在甲状腺结节的分类和解释。
By using a neural network-based image processing method, thyroid ultrasound images are acquired, and a target detection model is used for identification. Multi-scale features are extracted, an interpreter heatmap is generated, and classification is performed to provide an intuitive interpretation.
It improves the accuracy and speed of identifying and analyzing the core feature regions of thyroid ultrasound images, enabling doctors to review and evaluate the reliability of the AI.
Smart Images
Figure CN123033018A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical image processing technology, and in particular to an image processing method, system, terminal, and computer-readable storage medium based on neural networks. Background Technology
[0002] The application of artificial intelligence in medical diagnosis has been widely recognized. For example, convolutional neural networks (CNNs) have been used to classify thyroid nodules as benign or malignant. However, the "black box" nature of existing AI systems makes their decision-making process lack transparency. Doctors find it difficult to understand the basis of AI predictions, and the lack of explanation may reduce doctors' trust in AI, especially when AI predictions are inconsistent with doctors' initial judgments. This makes it difficult for AI to accurately classify and interpret key areas in thyroid ultrasound images, which is a problem that urgently needs to be solved.
[0003] Therefore, existing technologies still need to be improved and developed. Summary of the Invention
[0004] The main objective of this invention is to provide an image processing method, system, terminal, and computer-readable storage medium based on neural networks, aiming to solve the problem in the prior art that when predicting thyroid ultrasound images using AI, it is impossible to directly and accurately classify and interpret the core feature regions of the image.
[0005] To achieve the above objectives, the present invention provides an image processing method based on a neural network, the image processing method based on a neural network comprising the following steps:
[0006] A thyroid ultrasound image is acquired, and the target region image is obtained by identifying the thyroid ultrasound image using a target detection model.
[0007] The feature extraction network of the target detection model is used to extract features from the target region image to obtain a multi-scale feature representation map;
[0008] The multi-scale feature representation map is convolved using the interpreter generator of the target detection model to generate an interpreter heatmap.
[0009] The image classification network of the target detection model is used to classify the interpreter heatmap to obtain the feature classification result.
[0010] Optionally, in the neural network-based image processing method, the target detection model includes RetinaNet;
[0011] The acquisition of thyroid ultrasound images, including identifying the target region image using a target detection model, specifically includes:
[0012] Thyroid ultrasound images are acquired, and the thyroid nodule regions are identified using RetinaNet.
[0013] The thyroid nodule region is adjusted using the detection bounding box of RetinaNet to obtain the target region image.
[0014] Optionally, in the neural network-based image processing method, the feature extraction network includes a feature pyramid network architecture;
[0015] The step of extracting features from the target region image using the feature extraction network of the target detection model to obtain a multi-scale feature representation map specifically includes:
[0016] The target region image is used to extract features through the feature pyramid network architecture of the target detection model to obtain deep features and shallow features;
[0017] The deep features and the shallow features are fused using the feature pyramid network architecture to obtain a multi-scale feature representation map;
[0018] The deep features include high semantic features and low resolution features, while the shallow features include low semantic features and high resolution features.
[0019] Optionally, in the neural network-based image processing method, the interpretation generator includes a forward feature interpretation generator;
[0020] The step of using the interpreter generator of the target detection model to perform convolution processing on the multi-scale feature representation map to generate an interpreter heatmap specifically includes:
[0021] The multi-scale feature representation map is convolved by the forward feature interpreter of the target detection model to obtain an activation heatmap. The activation heatmap is then processed by the convolutional neural network architecture of the target detection model to obtain an initial interpreter heatmap.
[0022] The multi-scale feature representation map is masked based on the numerical distribution of the initial interpreter heatmap to obtain the interpreter heatmap.
[0023] Optionally, in the neural network-based image processing method, the step of masking the multi-scale feature representation map based on the numerical distribution of the initial interpreter heatmap to obtain the interpreter heatmap specifically includes:
[0024] The multi-scale feature representation map is masked based on the numerical distribution of the initial interpreter heatmap to obtain the key region;
[0025] The initial interpreter heatmap is obtained by element-wise multiplying the key regions.
[0026] Optionally, in the neural network-based image processing method, the feature classification result includes any one of the following: benign / malignant classification result, TI-RADS feature classification result, edge result, component result, echogenicity result, and echo foci result;
[0027] The step of classifying the interpreter heatmap using the image classification network of the target detection model to obtain feature classification results specifically includes:
[0028] The image classification network of the target detection model classifies the interpreter heatmap according to a preset classification task to obtain classification results. Based on the interpreter heatmap, the information of the target region image is classified and predicted to obtain prediction information.
[0029] Based on the prediction information and classification results, the target region image is classified to obtain any one of the following: benign / malignant classification result, TI-RADS feature classification result, edge result, component result, echo result, and echo foci result.
[0030] Optionally, the neural network-based image processing method, wherein the step of classifying the interpreter heatmap through the image classification network of the target detection model to obtain feature classification results, further includes:
[0031] The resolution of the interpreter heatmap is adjusted based on the thyroid ultrasound image to obtain the target interpreter heatmap;
[0032] The target interpreter heatmap is fused with the thyroid ultrasound image to generate a fused image. The boundaries of the fused image are then smoothed to obtain an optimal heatmap for the user to visualize.
[0033] Furthermore, to achieve the above objectives, the present invention also provides an image processing system based on a neural network, wherein the image processing system based on a neural network:
[0034] An image recognition module is used to acquire thyroid ultrasound images and identify the target region image by using a target detection model.
[0035] The image feature extraction module is used to extract features from the target region image through the feature extraction network of the target detection model to obtain a multi-scale feature representation map;
[0036] The image interpretation module is used to perform convolution processing on the multi-scale feature representation map through the interpretation generator of the target detection model to generate an interpreter heatmap;
[0037] The image classification module is used to classify the interpreter heatmap through the image classification network of the target detection model to obtain the feature classification result.
[0038] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a neural network-based image processing program, which, when executed by a processor, implements the steps of the neural network-based image processing method as described above.
[0039] In this invention, a thyroid ultrasound image is acquired, and the target region image is obtained by identifying the thyroid ultrasound image through a target detection model. Features of the target region image are extracted through the feature extraction network of the target detection model to obtain a multi-scale feature representation map. The multi-scale feature representation map is convolved through the interpretation generator of the target detection model to generate an interpreter heatmap. The interpreter heatmap is classified through the image classification network of the target detection model to obtain a feature classification result. This invention, by acquiring a thyroid ultrasound image, performing target detection and region identification, extracting multi-scale features, generating a heatmap, and classifying the results, greatly improves the accuracy and speed of identifying core feature regions of the image, assesses the corresponding risk level, and presents the AI's understanding of the image to doctors in an intuitive way through the explainer (interpretation heatmap module), enabling doctors to review and evaluate the reliability of the AI. Attached Figure Description
[0040] Figure 1 This is a flowchart of a preferred embodiment of the neural network-based image processing method of the present invention;
[0041] Figure 2 This is a schematic diagram of the overall architecture design of a preferred embodiment of the neural network-based image processing method of the present invention;
[0042] Figure 3 This is a flowchart of S10 of a preferred embodiment of the neural network-based image processing method of the present invention;
[0043] Figure 4 This is a flowchart of S20 of a preferred embodiment of the neural network-based image processing method of the present invention;
[0044] Figure 5 This is a schematic diagram of the feature extraction network of a preferred embodiment of the neural network-based image processing method of the present invention;
[0045] Figure 6 This is a flowchart of S30 of a preferred embodiment of the neural network-based image processing method of the present invention;
[0046] Figure 7 This is a flowchart of S32 of a preferred embodiment of the neural network-based image processing method of the present invention;
[0047] Figure 8 This is a schematic diagram illustrating the key region selection in a preferred embodiment of the neural network-based image processing method of the present invention.
[0048] Figure 9 This is a flowchart of S40 of a preferred embodiment of the neural network-based image processing method of the present invention;
[0049] Figure 10 This is a flowchart of the classification network of a preferred embodiment of the neural network-based image processing method of the present invention;
[0050] Figure 11 This is a structural diagram of a preferred embodiment of the neural network-based image processing system of the present invention;
[0051] Figure 12 This is a structural diagram of a preferred embodiment of the terminal of the device of the present invention. Detailed Implementation
[0052] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0053] The application of artificial intelligence in medical diagnosis is now widely recognized. For example, Convolutional Neural Networks (CNNs) have been used to classify benign and malignant thyroid nodules. However, the "black box" nature of existing AI systems makes their decision-making process lack transparency. Doctors find it difficult to understand the basis of AI predictions, and the lack of explanation may reduce doctors' trust in AI, especially when AI predictions are inconsistent with doctors' initial judgments. This leads to AI's inability to accurately classify and interpret key areas in thyroid ultrasound images. Therefore, a neural network-based image processing method is needed. This method acquires thyroid ultrasound images, performs target detection and region recognition, extracts multi-scale features, generates heatmaps, and outputs classification results, greatly improving the accuracy and speed of identifying core feature regions in the image.
[0054] The preferred embodiment of the image processing method based on neural networks of the present invention, such as... Figure 1 and Figure 2 As shown, the neural network-based image processing method includes the following steps:
[0055] Step S10: Obtain a thyroid ultrasound image, and use a target detection model to identify the thyroid ultrasound image to obtain a target region image.
[0056] like Figure 3 As shown, step S10 includes:
[0057] Step S11: Obtain thyroid ultrasound images, and identify thyroid nodule regions using RetinaNet;
[0058] Step S12: Adjust the thyroid nodule region using the detection box of the RetinaNet to obtain the target region image.
[0059] Specifically, the target detection model includes RetinaNet (Automated Nodule Region Detection: Based on a target detection model (such as RetinaNet), high-quality target (ROI, Region of Interest) images are generated, where the target (ROI) image refers to... Figure 2The thyroid ultrasound image was obtained by capturing the thyroid nodule and the surrounding tissue area. The thyroid ultrasound image was then identified using RetinaNet to obtain the thyroid nodule region (which can automatically identify the nodule region in the thyroid ultrasound image). The thyroid nodule region was then adjusted using the detection box of RetinaNet to obtain the target region image (the thyroid nodule region was detected using a target detection model (such as RetinaNet). To ensure coverage of the relevant tissue, the detection box size was enlarged by 1.5 times to generate the target region image. Subsequently, the resolution of the target ROI image was adjusted to 320×320 pixels using bilinear interpolation).
[0060] Step S20: Extract features from the target region image using the feature extraction network of the target detection model to obtain a multi-scale feature representation map.
[0061] like Figure 4 As shown, step S20 includes:
[0062] Step S21: Extract features from the target region image using the feature pyramid network architecture of the target detection model to obtain deep features and shallow features;
[0063] Step S22: Perform feature fusion on the deep features and the shallow features through the feature pyramid network architecture to obtain a multi-scale feature representation map.
[0064] Specifically, such as Figure 5 As shown, the target region image is feature extracted through the feature pyramid network architecture of the target detection model to obtain deep features and shallow features (the obtained target ROI image is input into the backbone of a convolutional neural network (CNN) to extract multi-layer feature maps, and the CNN backbone adopts a feature pyramid architecture). The deep features and shallow features are fused through the feature pyramid network architecture to obtain a multi-scale feature representation map (generating a multi-scale feature representation (such as a 3-layer feature map of FP (Feature Pyramid Network) with a size of 20×20, and a 4-layer feature map of FP (Feature Pyramid Network) with a size of 10×10)). The deep features include high semantic features and low-resolution features, and the shallow features include low semantic features and high-resolution features.
[0065] Step S30: The multi-scale feature representation map is convolved using the interpreter generator of the target detection model to generate an interpreter heatmap.
[0066] like Figure 6 As shown, step S30 includes:
[0067] Step S31: The multi-scale feature representation map is convolved through the forward feature interpreter generator of the target detection model to obtain an activation heatmap. The activation heatmap is then processed by the convolutional neural network architecture of the target detection model to obtain an initial interpreter heatmap.
[0068] Step S32: Mask the multi-scale feature representation map according to the numerical distribution of the initial interpreter heatmap to obtain the interpreter heatmap.
[0069] Specifically, the multi-scale feature representation map is convolved by the forward feature interpreter generator of the object detection model to obtain an activation heatmap (constructing a convolutional neural network for the corresponding classification task). The activation heatmap is then processed by the convolutional neural network architecture of the object detection model (using a U-shaped convolutional network architecture (UNet, U-shaped Convolutional Networks for Biomedical Image Segmentation) to generate the corresponding initial interpreter heatmap, resulting in an initial interpreter heatmap (generated from the feature map through convolution operations). The multi-scale feature representation map is then masked based on the numerical distribution of the initial interpreter heatmap (by masking all regions in the original image where the corresponding heatmap value is 0 during inference), thus obtaining the interpreter heatmap.
[0070] like Figure 7 As shown, step S32 includes:
[0071] Step S321: Mask the multi-scale feature representation map according to the numerical distribution of the initial interpreter heatmap to obtain the key region;
[0072] Step S322: Multiply the initial interpreter heatmap element by element according to the key regions to obtain the interpreter heatmap.
[0073] Specifically, such as Figure 8 As shown, the multi-scale feature representation map is masked based on the numerical distribution of the initial interpreter heatmap to obtain key regions (masking non-key region information (pixels with values close to 0)). The initial interpreter heatmap is then element-wise multiplied based on these key regions (element-wise multiplication of E_map with the key regions, where E_map is the sum of the length, width, and feature values of a single channel). Figure 1 (Multiply each element individually) to obtain the interpreter heatmap.
[0074] like Figure 9As shown, in step S40, the interpreter heatmap is classified through the image classification network of the target detection model to obtain the feature classification result.
[0075] Step S41: The image classification network of the target detection model classifies the interpreter heatmap according to a preset classification task to obtain the classification result. Based on the interpreter heatmap, the information of the target region image is classified and predicted to obtain the prediction information.
[0076] Step S42: Perform feature classification on the target region image based on the prediction information and classification results to obtain any one of the following: benign / malignant classification result, TI-RADS feature classification result, edge result, component result, echo result, and echo foci result.
[0077] Specifically, such as Figure 10 As shown, the feature classification results include any one of the following: benign / malignant classification results, TI-RADS feature classification results, edge results, composition results, echogenicity results, and echogenicity results (each feature has an independent interpreter, and each interpreter focuses on different features, such as nature, edge, composition, echogenicity, and echogenicity; where benign / malignant means benign / malignant, composition means cystic / solid / cystic-solid, echogenicity means hyperechoic / isoechoic / hypoechoic, edge means smooth / irregular edge, and calcification means present / absent). The image classification network of the target detection model classifies the interpreter heatmap according to a preset classification task to obtain classification results. Based on the interpreter heatmap, the information of the target region image is classified and predicted to obtain predicted information (subsequently, the classification and interpretation branches complete feature extraction, classification prediction, and interpretation generation, finally outputting the classification result to assist doctors in evaluating the reliability of AI predictions, such as...). Figure 5 As shown, for example: the generator interprets the E_map generated through convolution operations. The value of the E_map is between 0 and 1, representing the importance of each pixel in the feature map to the prediction. To adapt to multi-layer feature maps, the size of the E_map is adjusted by upsampling or downsampling (e.g., adjusting FP4 to FP3 size). Based on the prediction information and classification results, the target region image is classified to obtain any one of the following: benign / malignant classification result, ultrasound feature (TI-RADS, Thyroid Imaging Reporting and Data System) feature classification result, edge result, component result, echo result, and echo foci result (only key regions are retained for classification prediction (e.g.)). Figure 8 )).
[0078] Furthermore, after step S40, the method further includes adjusting the resolution of the interpreter heatmap based on the thyroid ultrasound image to obtain a target interpreter heatmap; fusing the target interpreter heatmap with the thyroid ultrasound image to generate a fused image; smoothing the boundaries of the fused image (adjusting it to match the resolution of the original ROI image and fusing it with the original image using an image processing tool to generate a heatmap; to improve the visualization effect, the boundaries of the heatmap can be smoothed) to obtain an optimal heatmap for the user to visualize and observe the image.
[0079] Furthermore, such as Figure 11 As shown, based on the above-described neural network-based image processing method, the present invention also provides a neural network-based image processing system, wherein the neural network-based image processing system includes:
[0080] Image recognition module 51 is used to acquire thyroid ultrasound images and identify the thyroid ultrasound images using a target detection model to obtain target region images;
[0081] Image feature extraction module 52 is used to extract features from the target region image through the feature extraction network of the target detection model to obtain a multi-scale feature representation map;
[0082] Image interpretation module 53 is used to perform convolution processing on the multi-scale feature representation map through the interpretation generator of the target detection model to generate an interpreter heatmap;
[0083] The image classification module 54 is used to classify the interpreter heatmap through the image classification network of the target detection model to obtain the feature classification result.
[0084] Furthermore, such as Figure 12 As shown, based on the above-mentioned neural network-based image processing method and system, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 12 Only some of the terminal components are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.
[0085] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory. In other embodiments, the memory 20 may be an external storage device of the terminal, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc. Further, the memory 20 may include both internal and external storage devices. The memory 20 is used to store application software and various types of data installed on the terminal, such as program code installed on the terminal. The memory 20 can also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores a neural network-based image processing program 40, which can be executed by the processor 10 to implement the neural network-based image processing method of this application.
[0086] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run program code stored in the memory 20 or process data, such as executing the neural network-based image processing method.
[0087] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 30 is used to display information on the terminal and to display a visual user interface. The terminals communicate with each other via a system bus.
[0088] In one embodiment, when the processor 10 executes the neural network-based image processing program 40 in the memory 20, the following steps are performed:
[0089] A thyroid ultrasound image is acquired, and the target region image is obtained by identifying the thyroid ultrasound image using a target detection model.
[0090] The feature extraction network of the target detection model is used to extract features from the target region image to obtain a multi-scale feature representation map;
[0091] The interpreter generator of the target detection model performs convolution processing on the multi-scale feature representation map to generate an interpreter heatmap;
[0092] The image classification network of the target detection model is used to classify the interpreter heatmap to obtain the feature classification result.
[0093] The target detection model includes RetinaNet;
[0094] The acquisition of thyroid ultrasound images, including identifying the target region image using a target detection model, specifically includes:
[0095] Thyroid ultrasound images were acquired, and the thyroid nodule regions were identified using RetinaNet.
[0096] The thyroid nodule region is adjusted using the detection bounding box of RetinaNet to obtain the target region image.
[0097] The feature extraction network includes a feature pyramid network architecture;
[0098] The step of extracting features from the target region image using the feature extraction network of the target detection model to obtain a multi-scale feature representation map specifically includes:
[0099] The target region image is used to extract features through the feature pyramid network architecture of the target detection model to obtain deep features and shallow features;
[0100] The deep features and the shallow features are fused using the feature pyramid network architecture to obtain a multi-scale feature representation map;
[0101] The deep features include high semantic features and low resolution features, while the shallow features include low semantic features and high resolution features.
[0102] The interpretation generator includes a forward feature interpretation generator;
[0103] The step of using the interpreter generator of the target detection model to perform convolution processing on the multi-scale feature representation map to generate an interpreter heatmap specifically includes:
[0104] The multi-scale feature representation map is convolved by the forward feature interpreter of the target detection model to obtain an activation heatmap. The activation heatmap is then processed by the convolutional neural network architecture of the target detection model to obtain an initial interpreter heatmap.
[0105] The multi-scale feature representation map is masked based on the numerical distribution of the initial interpreter heatmap to obtain the interpreter heatmap.
[0106] Specifically, the step of masking the multi-scale feature representation map based on the numerical distribution of the initial interpreter heatmap to obtain the interpreter heatmap includes:
[0107] The multi-scale feature representation map is masked based on the numerical distribution of the initial interpreter heatmap to obtain the key region;
[0108] The initial interpreter heatmap is obtained by element-wise multiplying the key regions.
[0109] The feature classification results include any one of the following: benign / malignant classification results, TI-RADS feature classification results, edge results, component results, echogenicity results, and echo foci results;
[0110] The step of classifying the interpreter heatmap using the image classification network of the target detection model to obtain feature classification results specifically includes:
[0111] The image classification network of the target detection model classifies the interpreter heatmap according to a preset classification task to obtain classification results. Based on the interpreter heatmap, the information of the target region image is classified and predicted to obtain prediction information.
[0112] Based on the prediction information and classification results, the target region image is classified to obtain any one of the following: benign / malignant classification result, TI-RADS feature classification result, edge result, component result, echo result, and echo foci result.
[0113] The step of classifying the interpreter heatmap using the image classification network of the target detection model to obtain feature classification results further includes:
[0114] The resolution of the interpreter heatmap is adjusted based on the thyroid ultrasound image to obtain the target interpreter heatmap;
[0115] The target interpreter heatmap is fused with the thyroid ultrasound image to generate a fused image. The boundaries of the fused image are then smoothed to obtain an optimal heatmap for the user to visualize.
[0116] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a neural network-based image processing program, which, when executed by a processor, implements the steps of the neural network-based image processing method as described above.
[0117] In summary, this invention provides an image processing method, system, terminal, and storage medium based on neural networks. The method includes: acquiring a thyroid ultrasound image; identifying the thyroid ultrasound image using a target detection model to obtain a target region image; extracting features from the target region image using the feature extraction network of the target detection model to obtain a multi-scale feature representation map; performing convolution processing on the multi-scale feature representation map using the interpreter generator of the target detection model to generate an interpreter heatmap; and classifying the interpreter heatmap using the image classification network of the target detection model to obtain a feature classification result. This invention, by acquiring a thyroid ultrasound image, performing target detection and region identification on the image, extracting multi-scale features, generating a heatmap, and classifying the results, greatly improves the accuracy and speed of identifying core feature regions of the image and assesses the corresponding risk level. Furthermore, it presents the AI's understanding of the image to doctors in an intuitive way through the explainer (interpretation heatmap module), enabling doctors to review and evaluate the reliability of the AI.
[0118] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal system that includes that element.
[0119] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). The program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The computer-readable storage medium can be a memory, magnetic disk, optical disk, etc.
[0120] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.
Claims
1. An image processing method based on neural networks, characterized in that, The neural network-based image processing method includes: A thyroid ultrasound image is acquired, and the target region image is obtained by identifying the thyroid ultrasound image using a target detection model. The feature extraction network of the target detection model is used to extract features from the target region image to obtain a multi-scale feature representation map; The interpreter generator of the target detection model performs convolution processing on the multi-scale feature representation map to generate an interpreter heatmap; The image classification network of the target detection model is used to classify the interpreter heatmap to obtain the feature classification result.
2. The image processing method based on neural networks according to claim 1, characterized in that, The target detection model includes RetinaNet; The acquisition of thyroid ultrasound images, including identifying the target region image using a target detection model, specifically includes: Thyroid ultrasound images are acquired, and the thyroid nodule regions are identified using RetinaNet. The thyroid nodule region is adjusted using the detection bounding box of RetinaNet to obtain the target region image.
3. The image processing method based on neural networks according to claim 2, characterized in that, The feature extraction network includes a feature pyramid network architecture; The step of extracting features from the target region image using the feature extraction network of the target detection model to obtain a multi-scale feature representation map specifically includes: The target region image is used to extract features through the feature pyramid network architecture of the target detection model to obtain deep features and shallow features; The deep features and the shallow features are fused using the feature pyramid network architecture to obtain a multi-scale feature representation map; The deep features include high semantic features and low resolution features, while the shallow features include low semantic features and high resolution features.
4. The image processing method based on a neural network according to claim 3, characterized in that, The interpretation generator includes a forward feature interpretation generator; The step of using the interpreter generator of the target detection model to perform convolution processing on the multi-scale feature representation map to generate an interpreter heatmap specifically includes: The multi-scale feature representation map is convolved by the forward feature interpreter of the target detection model to obtain an activation heatmap. The activation heatmap is then processed by the convolutional neural network architecture of the target detection model to obtain an initial interpreter heatmap. The multi-scale feature representation map is masked based on the numerical distribution of the initial interpreter heatmap to obtain the interpreter heatmap.
5. The image processing method based on a neural network according to claim 4, characterized in that, The step of masking the multi-scale feature representation map based on the numerical distribution of the initial interpreter heatmap to obtain the interpreter heatmap specifically includes: The multi-scale feature representation map is masked based on the numerical distribution of the initial interpreter heatmap to obtain the key region; The initial interpreter heatmap is obtained by element-wise multiplying the key regions.
6. The image processing method based on a neural network according to claim 5, characterized in that, The feature classification results include any one of the following: benign / malignant classification results, TI-RADS feature classification results, edge results, component results, echogenicity results, and echo foci results; The step of classifying the interpreter heatmap using the image classification network of the target detection model to obtain feature classification results specifically includes: The image classification network of the target detection model classifies the interpreter heatmap according to a preset classification task to obtain classification results. Based on the interpreter heatmap, the information of the target region image is classified and predicted to obtain prediction information. Based on the prediction information and classification results, the target region image is classified to obtain any one of the following: benign / malignant classification result, TI-RADS feature classification result, edge result, component result, echo result, and echo foci result.
7. The image processing method based on a neural network according to claim 1, characterized in that, The step of classifying the interpreter heatmap using the image classification network of the target detection model to obtain feature classification results, and then further includes: The resolution of the interpreter heatmap is adjusted based on the thyroid ultrasound image to obtain the target interpreter heatmap; The target interpreter heatmap is fused with the thyroid ultrasound image to generate a fused image. The boundaries of the fused image are then smoothed to obtain an optimal heatmap for the user to visualize.
8. An image processing system based on a neural network, characterized in that, The neural network-based image processing system includes: The image recognition module is used to acquire thyroid ultrasound images and identify the target region image by using a target detection model. The image feature extraction module is used to extract features from the target region image through the feature extraction network of the target detection model to obtain a multi-scale feature representation map; The image interpretation module is used to perform convolution processing on the multi-scale feature representation map through the interpretation generator of the target detection model to generate an interpreter heatmap; The image classification module is used to classify the interpreter heatmap through the image classification network of the target detection model to obtain the feature classification result.
9. A terminal, characterized in that, The terminal includes: a memory, a processor, and a neural network-based image processing program stored in the memory and executable on the processor. When the neural network-based image processing program is executed by the processor, it implements the steps of the neural network-based image processing method as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a neural network-based image processing program, which, when executed by a processor, implements the steps of the neural network-based image processing method as described in any one of claims 1-7.