Image processing method for detecting malignant nodule based on benign thyroid

Abstract

The application belongs to the technical field of image processing, and discloses a malignant nodule detection image processing method based on benign thyroid; the method comprises the following steps: S2, processing a standard image data set to obtain a static feature vector; S4, processing a standard dynamic image data set to obtain a dynamic feature vector; S5, processing based on the static feature vector and the dynamic feature vector, and performing probability evaluation according to a processing result to obtain a malignant probability evaluation value; S6, analyzing the malignant probability evaluation value to obtain a final probability evaluation value and a system evaluation value; S7, analyzing based on the final probability evaluation value and the system evaluation value to obtain a patient diagnosis report and a system evaluation report; and S8, processing based on the system evaluation report to obtain a system optimization report; in general, the application has the remarkable advantages of high multi-source data utilization efficiency, strong detection sensitivity and good diagnosis process efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, more particularly, the present application relates to a benign thyroid-based malignant nodule detection image processing method. BACKGROUND

[0002] Thyroid malignant nodule detection based on medical images is an important application field of computer-aided diagnosis systems, mainly used for identifying and classifying the benign or malignant nature of nodules by analyzing image data such as ultrasound images.

[0003] However, the traditional benign thyroid-based malignant nodule detection image processing method still has the following shortcomings in use, first, the traditional method mostly adopts simple splicing or average fusion processing after feature extraction of the detection image, which makes the system prone to deviation in feature extraction position due to the differences in spatial consistency and information contribution between various modal images, second, the traditional method mostly uses static frame analysis in dynamic image nodule detection, lacking processing of motion artifacts in continuous cross-section images and extraction of deep spatio-temporal features, which easily causes the loss of important diagnostic basis of morphological changes, third, the traditional method is usually limited to single malignant nodule probability output, making it difficult to secondarily confirm the output result, which easily leads to poor accuracy of the output result due to problems in the original image or processing process, thereby affecting the doctor's diagnosis, in general, how to effectively deal with the problems of data fusion of detection images, mining of dynamic image information and evaluation of diagnosis results in the traditional method has become the urgent problem of the present benign thyroid-based malignant nodule detection image processing method.

[0004] In view of this, the present application proposes a benign thyroid-based malignant nodule detection image processing method to solve the above problems. SUMMARY

[0005] In order to overcome the above-mentioned defects of the prior art, in order to achieve the above-mentioned purpose, the present application provides the following technical scheme, comprising:

[0006] An image receiving preprocessing module is used to receive a patient data set and perform preprocessing to obtain a standard static image data set;

[0007] Further, the step of receiving a patient data set and performing preprocessing comprises:

[0008] S1.1: Obtain the specified patient data set through the information management platform, the patient data set including B-mode, elasticity map and blood flow map;

[0009] S1.2: based on the registration algorithm, the different modal images of the same nodule position are registered, and the mutual information of the two images is maximized to obtain a registration data set, and the specific expression of the registration algorithm is:

[0010]

[0011] The mutual information A(i, j) of the image I and the image J is obtained, wherein i and j are the pixel gray levels of the image I and the image J, p IJ (i, j) is the joint gray level probability distribution of the image I and the image J, p I (i) and p J (j) are the gray level probability distributions of the image I and the image J respectively;

[0012] S1.3: based on Z-score standardization, the registration data set is standardized to obtain a standard data set;

[0013] S1.4: using Gaussian filtering, the standard data set is denoised to obtain a denoised data set;

[0014] S1.5: based on the nodule position, the standard region is automatically reserved, and the image outside the reserved region is cropped to obtain a standard static image data set;

[0015] S1.6: the standard static image data set is output to a feature vector fusion module;

[0016] The feature vector fusion module is used for processing the standard image data set to obtain a static feature vector;

[0017] Further, the step of processing the standard image data set comprises:

[0018] S2.1: based on all modal images in the standard image data set, a pre-trained convolutional neural network is used for deep feature extraction respectively to obtain a feature vector set;

[0019] S2.2: based on the attention subnetwork, the weight of all feature vectors in the feature vector set is calculated, and the feature vector weight set is obtained after packing, and the specific formula is:

[0020] α m =ω α [f1;f2;...;f m ;]+b α ;

[0021] The mth feature vector weight is obtained, wherein ω α is a weight matrix, f m is the mth feature vector, and b α is a bias vector;

[0022] S2.3: Weighted sum calculation based on the feature vector weight set to obtain the static feature vector f all The specific formula for calculation is:

[0023] S2.4: Output the static feature vector to the multi-source feature calculation and evaluation module;

[0024] An image preprocessing enhancement module is configured to preprocess the continuous cross-section images of the nodule to obtain a standard dynamic image data set;

[0025] Further, the step of preprocessing the continuous cross-section images of the nodule includes:

[0026] S3.1: Inter-frame motion estimation of the continuous cross-section images is performed using the optical flow method, and inverse transformation is used to stabilize the inter-frame motion to obtain a stable image data set;

[0027] S3.2: Time-series neighborhood median value is taken for the pixel points of all images in the stable image data set using time-series median filtering to obtain a denoising image data set;

[0028] S3.3: Image enhancement is performed on the denoising image data set based on a contrast-limited adaptive histogram equalization tool to obtain an enhanced image data set;

[0029] S3.4: Linear interpolation is used to uniformly time-interval the images in the enhanced image data set to obtain a standard dynamic image data set;

[0030] S3.5: The standard dynamic image data set is output to the nodule feature extraction module;

[0031] A nodule feature extraction module is configured to process the standard dynamic image data set to obtain a dynamic feature vector;

[0032] Further, the step of processing the standard dynamic image data set includes:

[0033] S4.1: The motion speed and direction of each pixel in the image sequence in the standard dynamic image data set are calculated based on the optical flow method to obtain a motion vector field;

[0034] S4.2: Deformation rate calculation is performed based on the motion vector field in step S4.1, and the specific formula for calculation is:

[0035]

[0036] The deformation rate ε is obtained, where is the partial derivative, u and v are displacement vectors, and x and y are spatial coordinates;

[0037] S4.3: based on the motion vector field in step S4.1, and according to the 3D CNN model, spatio-temporal feature extraction is performed to obtain a spatio-temporal feature, and the specific calculation formula of the feature extraction is:

[0038]

[0039] The spatio-temporal feature F(x, y, t) is obtained, wherein P(x, y, t) is the intensity value of the motion vector field at the spatial position (x, y) and the time point t, and K(i, j, d) is the weight value of the convolution kernel at the offset (i, j, d);

[0040] S4.4: the deformation rate and the spatio-temporal feature are spliced to obtain a dynamic feature vector;

[0041] S4.5: the dynamic feature vector is output to a multi-source feature calculation and evaluation module;

[0042] The multi-source feature calculation and evaluation module is used for processing based on the static feature vector and the dynamic feature vector, and performing probability evaluation according to the processing result to obtain a malignant probability evaluation value;

[0043] Further, the step of processing based on the static feature vector and the dynamic feature vector and performing probability evaluation according to the processing result comprises:

[0044] S5.1: based on the static feature vector and the dynamic feature vector, a joint feature vector is obtained by splicing;

[0045] S5.2: based on the joint feature vector, a malignant probability evaluation value is calculated according to a deep neural network classifier, and the specific formula of the calculation is:

[0046] Y = σ(W Τ f c +b);

[0047] The malignant probability evaluation value Y is obtained, wherein σ is a Sigmoid activation function, W Τ is the transpose of the weight matrix of the classification layer, f c is the joint feature vector, and b is the bias parameter of the classification layer;

[0048] S5.3: the malignant probability evaluation value is output to a model diagnosis and evaluation module;

[0049] The model diagnosis and evaluation module is used for analyzing the malignant probability evaluation value to obtain a final probability evaluation value and a system evaluation value;

[0050] Further, the step of analyzing the malignant probability evaluation value comprises:

[0051] S6.1: N times of forward propagation are performed on the malignant probability evaluation value based on Monte Carlo Dropout, to obtain N malignant probability evaluation values;

[0052] S6.2: A final probability evaluation value is calculated based on the N malignant probability evaluation values, and the specific formula for calculation is:

[0053]

[0054] A final probability evaluation value ZY is obtained, wherein Y n is the nth malignant probability evaluation value;

[0055] S6.3: Variance calculation is performed based on the final probability evaluation value in step S6.2, to obtain a system evaluation value R, and the specific formula for variance calculation is:

[0056] S6.4: The final probability evaluation value and the system evaluation value are output to a visual report generation module;

[0057] The visual report generation module is configured to analyze the final probability evaluation value and the system evaluation value, to obtain a patient diagnosis report and a system evaluation report;

[0058] Further, the step of analyzing the final probability evaluation value and the system evaluation value to obtain the patient diagnosis report and the system evaluation report includes:

[0059] S7.1: A probability heat map is generated based on the final probability evaluation value and according to a gradient-weighted class activation mapping technique;

[0060] S7.2: A feature contribution map is obtained by calculating the contribution value of the static feature vector and the dynamic feature vector to the final probability evaluation value;

[0061] S7.3: Based on an evaluation threshold, when the system evaluation value is greater than or equal to the evaluation threshold, a system evaluation report is generated;

[0062] The factor report includes an indication that the multi-modal image quality in the patient data set is poor, the static feature vector and the dynamic feature vector conflict, and / or the image sequence has a problem;

[0063] S7.4: The probability heat map and the feature contribution map are packaged to obtain a patient diagnosis report;

[0064] S7.5: The patient diagnosis report is output to a system data management module, and the system evaluation report is output to a system interaction optimization module;

[0065] The system interaction optimization module is configured to process based on the system evaluation report, to obtain a system optimization report;

[0066] Further, the step of processing based on the system evaluation report comprises:

[0067] S8.1: reading the system evaluation report;

[0068] S8.2: generating a reacquisition instruction when the system evaluation report content is multimodal image quality difference;

[0069] The reacquisition instruction contains a set of characters representing the image receiving preprocessing module reacquiring the specified patient data set;

[0070] S8.3: generating a weight adjustment instruction when the system evaluation report content is static feature vector and dynamic feature vector conflict;

[0071] The weight adjustment instruction contains a set of characters representing adjusting the feature vector weight in the feature vector fusion module;

[0072] S8.4: generating a reprocessing instruction when the system evaluation report content is image sequence problem;

[0073] The reprocessing instruction contains a set of characters representing the nodule feature extraction module reprocessing the standard dynamic image data set;

[0074] S8.5: packaging the reacquisition instruction, the weight adjustment instruction and the reprocessing instruction to obtain a system optimization report;

[0075] A system data management module for separately processing system data;

[0076] Further, the way of separately processing system data comprises:

[0077] The system data includes patient data set, standard static image data set, standard dynamic image data set, static feature vector, dynamic feature vector, malignant probability evaluation value, final probability evaluation value, system evaluation value, patient diagnosis report, system evaluation report and system optimization report;

[0078] The patient diagnosis report is output to the doctor visual panel receiving end;

[0079] Further, S1: receiving a patient data set and preprocessing to obtain a standard static image data set;

[0080] S2: processing the standard image data set to obtain a static feature vector;

[0081] S3: preprocessing the continuous cross-sectional images of the nodule to obtain a standard dynamic image data set;

[0082] S4: processing the standard dynamic image data set to obtain a dynamic feature vector;

[0083] S5: processing based on the static feature vector and the dynamic feature vector, and probability evaluation according to a processing result to obtain a malignant probability evaluation value;

[0084] S6: analysis on the malignant probability evaluation value to obtain a final probability evaluation value and a system evaluation value;

[0085] S7: analysis based on the final probability evaluation value and the system evaluation value to obtain a patient diagnosis report and a system evaluation report;

[0086] S8: processing based on the system evaluation report to obtain a system optimization report;

[0087] S9: separate processing of system data.

[0088] The application has the technical effects and advantages of the malignant nodule detection image processing method of the benign thyroid:

[0089] The application receives a patient data set and pre-processes to obtain a standard static image data set, processes the standard image data set to obtain a static feature vector, pre-processes the continuous cross-section image of the nodule to obtain a standard dynamic image data set, processes the standard dynamic image data set to obtain a dynamic feature vector, processes based on the static feature vector and the dynamic feature vector, and performs probability evaluation according to the processing result to obtain a malignant probability evaluation value, analyzes the malignant probability evaluation value to obtain a final probability evaluation value and a system evaluation value, analyzes based on the final probability evaluation value and the system evaluation value to obtain a patient diagnosis report and a system evaluation report, processes based on the system evaluation report to obtain a system optimization report, and separately processes the system data, so that the system can effectively solve the problem of poor fusion effect of the traditional method on the multi-modal detection image data through multiple processing of the patient data set, in addition, the application also sets an image preprocessing enhancement module and a nodule feature extraction module to realize stable enhancement of the continuous cross-section image by the system and in-depth mining of dynamic image information, so that the system can effectively obtain the malignant symptom phenomenon that is difficult to express on the static image, greatly improving the sensitivity of the system in detecting malignant nodules, at the same time, the model diagnosis evaluation module further realizes the "gatekeeping" effect for the accuracy of the system diagnosis result, the visualization report generation module realizes the clearness and traceability of the malignant nodule probability and the "gatekeeping" result respectively, finally, according to the system interaction optimization module, the diagnosis evaluation after tracing generates targeted optimization adjustment measures, which maximizes the practicality and sustainable use of the system, in general, the application has the significant advantages of high multi-source data utilization efficiency, strong detection sensitivity and good diagnosis process efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0090] Figure 1A schematic diagram of an image processing system for detecting a malignant nodule based on a benign thyroid according to the present application;

[0091] Figure 2 A schematic diagram of an image processing method for detecting a malignant nodule based on a benign thyroid according to the present application. DETAILED DESCRIPTION

[0092] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all the other embodiments obtained by a person of ordinary skill in the art without creative work fall within the scope of the present application.

[0093] The terms used in the embodiments of the present application are merely for the purpose of describing the specific embodiments, and are not intended to limit the present application. The singular forms "a", "an" and "the" used in the embodiments of the present application are also intended to include the plural forms, unless the context clearly indicates otherwise. "Plural" generally includes at least two.

[0094] Depending on the context, the word "if" as used herein can be interpreted to mean "when" or "while" or "in response to determining" or "in response to detecting". Similarly, depending on the context, the phrase "if determined" or "if detecting (a stated condition or event)" can be interpreted to mean "when determined" or "in response to determining" or "when detecting (a stated condition or event)" or "in response to detecting (a stated condition or event)".

[0095] In addition, the sequence of steps in each of the following method embodiments is only an example, and is not strictly limited.

[0096] In fact, the server device deployed by the benign thyroid-based malignant nodule detection image processing system can be composed of one or more devices. The benign thyroid-based malignant nodule detection image processing system can be implemented as a business instance, a virtual machine, or a hardware device. For example, the benign thyroid-based malignant nodule detection image processing system can be implemented as a business instance deployed on one or more devices in a cloud node. In short, the benign thyroid-based malignant nodule detection image processing system can be understood as a software deployed on a cloud node, which provides a benign thyroid-based malignant nodule detection image processing system for each user terminal. Alternatively, the benign thyroid-based malignant nodule detection image processing system can also be implemented as a virtual machine deployed on one or more devices in a cloud node. The virtual machine has application software installed for managing each user terminal. Alternatively, the benign thyroid-based malignant nodule detection image processing system can also be implemented as a server composed of a plurality of same or different types of hardware devices, and one or more hardware devices are provided to provide a benign thyroid-based malignant nodule detection image processing system for each user terminal.

[0097] In an implementation form, the benign thyroid-based malignant nodule detection image processing system and the user terminal are mutually adaptive. That is, the benign thyroid-based malignant nodule detection image processing system is an application installed on a cloud service platform, and the user terminal is a client that establishes a communication connection with the application; or the benign thyroid-based malignant nodule detection image processing system is implemented as a website, and the user terminal is implemented as a webpage; or the benign thyroid-based malignant nodule detection image processing system is implemented as a cloud service platform, and the user terminal is implemented as an applet in an instant messaging application.

[0098] As shown in Figure 1 FIG. 1 is a system architecture diagram of a benign thyroid-based malignant nodule detection image processing system according to an embodiment of the present application.

[0099] The benign thyroid-based malignant nodule detection image processing system can be set in a cloud server, and in terms of implementation, can be used as one or more service devices, or can be installed as an application on a cloud (such as a server of a mobile service operator, a server cluster, or the like), or can be developed as a website. According to the functions implemented, the benign thyroid-based malignant nodule detection image processing system can include an image receiving preprocessing module, a feature vector fusion module, an image preprocessing enhancement module, a nodule feature extraction module, a multi-source feature calculation and evaluation module, a model diagnosis and evaluation module, a visualization report generation module, a system interaction optimization module, and a system data management module. The modules in the present application can also be referred to as units, which refer to a series of computer program segments that can be executed by an electronic device processor and can complete a fixed function, which are stored in the memory of the electronic device.

[0100] In the embodiment of the present application, each of the above-mentioned modules in the benign thyroid-based malignant nodule detection image processing system can be independently implemented and called by other modules. Here, calling can be understood as connecting a module to a plurality of modules of another type and providing corresponding services for the plurality of modules connected thereto. For example, the sharing evaluation module can call the same information collection module to obtain the information collected by the information collection module. Based on the above characteristics, the benign thyroid-based malignant nodule detection image processing system provided in the embodiment of the present application can adjust the application scope of the benign thyroid-based malignant nodule detection image processing system architecture by increasing modules and directly calling without modifying program codes, realize cluster-type horizontal expansion, and achieve the purpose of quickly and flexibly expanding the benign thyroid-based malignant nodule detection image processing system. In practical applications, the above-mentioned modules can be arranged in the same device or different devices, or can be arranged in a virtual device, such as a service instance in a cloud server.

[0101] Embodiment 1

[0102] Please refer to Figure 1 The benign thyroid-based malignant nodule detection image processing system of the present embodiment includes:

[0103] The image receiving preprocessing module is configured to receive a patient data set and perform preprocessing to obtain a standard static image data set.

[0104] Further, the step of receiving a patient data set and performing preprocessing includes:

[0105] S1.1: Obtain a specified patient data set through an information management platform, wherein the patient data set includes B-mode, elasticity map, and blood flow map.

[0106] S1.2: based on the registration algorithm, the different modal images of the same nodule position are registered, and the mutual information of the two images is maximized to obtain a registration data set, and the specific expression of the registration algorithm is:

[0107]

[0108] The mutual information A(i, j) of the image I and the image J is obtained, wherein i and j are the pixel gray levels of the image I and the image J, p IJ (i, j) is the joint gray level probability distribution of the image I and the image J, p I (i) and p J (j) are the gray level probability distributions of the image I and the image J respectively;

[0109] It needs to be explained that the mutual information is used to measure the similarity between two images;

[0110] S1.3: based on Z-score standardization, the registration data set is standardized to obtain a standard data set;

[0111] S1.4: using Gaussian filtering, the standard data set is denoised to obtain a denoised data set;

[0112] S1.5: based on the nodule position, the standard reserved area is automatically reserved, and the image outside the reserved area is cropped to obtain a standard static image data set;

[0113] S1.6: the standard static image data set is output to a feature vector fusion module;

[0114] The feature vector fusion module is used to process the standard image data set to obtain a static feature vector;

[0115] Further, the step of processing the standard image data set comprises:

[0116] S2.1: based on all modal images in the standard image data set, a pre-trained convolutional neural network is used for deep feature extraction respectively, and a feature vector set is obtained by packaging;

[0117] S2.2: based on the attention subnetwork, the weight of all feature vectors in the feature vector set is calculated, and a feature vector weight set is obtained by packaging, and the specific formula for calculation is:

[0118] α m =ω α [f1;f2;...;f m ;]+b α ;

[0119] The mth feature vector weight is obtained, wherein ω α is a weight matrix, and fm Let b be the m-th eigenvector. α Bias vector;

[0120] S2.3: Calculate the static feature vector f by performing a weighted summation based on the feature vector weight set. all The specific formula for calculation is:

[0121] S2.4: Output the static feature vector to the multi-source feature calculation and evaluation module;

[0122] The image preprocessing enhancement module is used to preprocess the continuous cross-sectional images of the nodules to obtain a standard dynamic image dataset.

[0123] Further, the preprocessing steps for the continuous cross-sectional images of the nodules include:

[0124] S3.1: Frame interval motion estimation is performed on continuously switching images using optical flow, and frame interval motion is stabilized using inverse transform to obtain a stable image dataset;

[0125] S3.2: Use temporal median filtering to perform temporal neighborhood median values ​​on the pixels of all images in the stable image dataset to obtain the denoised image dataset;

[0126] S3.3: Image enhancement is performed on the denoised image dataset based on the contrast-limited adaptive histogram equalization tool to obtain the enhanced image dataset;

[0127] S3.4: The time intervals of the images in the enhanced image dataset are uniformly adjusted by linear interpolation to obtain a standard dynamic image dataset;

[0128] S3.5: Output the standard dynamic image dataset to the nodule feature extraction module;

[0129] The nodule feature extraction module is used to process the standard dynamic image dataset to obtain dynamic feature vectors;

[0130] Further steps in processing standard dynamic image datasets include:

[0131] S4.1: Calculate the motion velocity and direction of each pixel in the image sequence of the standard dynamic image dataset based on the optical flow method to obtain the motion vector field;

[0132] S4.2: Calculate the deformation rate based on the motion vector field in step S4.1. The specific formula for the calculation is as follows:

[0133]

[0134] The deformation rate ε is obtained, where, Let be the partial derivatives, u and v be the displacement vectors, and x and y be the spatial coordinates;

[0135] S4.3: Based on the motion vector field in step S4.1, and using the 3D CNN model, spatiotemporal features are extracted to obtain spatiotemporal features. The specific calculation formula for feature extraction is as follows:

[0136]

[0137] The spatiotemporal features F(x, y, t) are obtained, where P(x, y, t) is the intensity value of the motion vector field at spatial position (x, y) and time point t, and K(i, j, d) is the weight value of the convolution kernel at offset (i, j, d).

[0138] S4.4: Concatenate the deformation rate and spatiotemporal features to obtain the dynamic feature vector;

[0139] S4.5: Output the dynamic feature vector to the multi-source feature calculation and evaluation module;

[0140] The multi-source feature calculation and evaluation module is used to process static feature vectors and dynamic feature vectors, and to perform probability evaluation based on the processing results to obtain a malignancy probability evaluation value.

[0141] Furthermore, the steps of processing based on static and dynamic feature vectors and evaluating probabilities based on the processing results include:

[0142] S5.1: A joint feature vector is obtained by concatenating static and dynamic feature vectors;

[0143] S5.2: Based on the joint feature vector and using a deep neural network classifier, the malignancy probability assessment value is calculated. The specific formula for the calculation is as follows:

[0144] Y = σ(W) Τ f c +b);

[0145] The malignancy probability assessment value Y is obtained, where σ is the Sigmoid activation function, and W... Τ f is the transpose of the weight matrix of the classification layer. c Here, b is the joint feature vector, and b is the bias parameter of the classification layer.

[0146] It should be explained that the Sigmoid activation function is used to constrain the output value within the immediately adjacent brackets to the range (0,1);

[0147] S5.3: Output the malignancy probability assessment value to the model diagnosis and assessment module;

[0148] The model diagnosis and evaluation module is used to analyze the malignancy probability evaluation value to obtain the final probability evaluation value and the system evaluation value.

[0149] Further steps in analyzing the probability assessment value of malignancy include:

[0150] S6.1: Based on Monte Carlo Dropout, perform N forward propagation operations on the malignancy probability assessment value to obtain N malignancy probability assessment values;

[0151] S6.2: Calculate the final probability assessment value based on N malignancy probability assessment values. The specific formula for the calculation is as follows:

[0152]

[0153] The final probability evaluation value ZY is obtained, where Y n This is the probability assessment value for the nth malignancy.

[0154] S6.3: Calculate the variance based on the final probability assessment value from step S6.2 to obtain the system assessment value R. The specific formula for variance calculation is as follows:

[0155] S6.4: Output the final probability assessment value and system assessment value to the visualization report generation module;

[0156] The visualization report generation module is used to analyze the final probability assessment value and system assessment value to obtain patient diagnosis report and system assessment report;

[0157] Furthermore, the steps for obtaining patient diagnosis reports and system evaluation reports based on the final probability assessment values ​​and system evaluation values ​​include:

[0158] S7.1: Generate a probability heatmap based on the final probability evaluation value and according to the gradient-weighted class activation mapping technique;

[0159] S7.2: By calculating the contribution values ​​of static and dynamic eigenvectors to the final probability evaluation value, a feature contribution map is obtained;

[0160] S7.3: Based on the evaluation threshold, when the system evaluation value is greater than or equal to the evaluation threshold, a system evaluation report is generated;

[0161] It should be explained that the evaluation threshold was set manually and entered into the system;

[0162] Factor reports include descriptions of poor multimodal image quality in the patient dataset, conflicts between static and dynamic feature vectors, and / or problems with the image sequences.

[0163] It should be explained that the poor quality of multimodal images is detected by the system when processing patient datasets through the image receiving preprocessing module; the conflict between static and dynamic feature vectors is determined by the model diagnosis and evaluation module after calculating the contribution values ​​of static and dynamic feature vectors, and when the contribution values ​​are similar but opposite in direction; the problem with the image sequence is determined when the variance or entropy of the motion vector field in the nodule feature extraction module exceeds the normal range.

[0164] S7.4: Pack the probability heatmap and feature contribution map to obtain the patient diagnosis report;

[0165] S7.5: Output patient diagnosis reports to the system data management module and system evaluation reports to the system interaction optimization module;

[0166] The system interaction optimization module is used to process the system evaluation report to obtain a system optimization report;

[0167] Further steps based on the system evaluation report include:

[0168] S8.1: Read the system evaluation report;

[0169] S8.2: When the system evaluation report indicates poor multimodal image quality, generate a re-acquisition command;

[0170] The reacquisition command contains a set of characters representing the image receiving preprocessing module reacquiring the specified patient dataset;

[0171] S8.3: When there is a conflict between static and dynamic feature vectors in the system evaluation report, a weight adjustment instruction is generated;

[0172] The weight adjustment instruction contains a set of characters representing the adjustment of the feature vector weights in the feature vector fusion module;

[0173] S8.4: When the system evaluation report indicates that there is a problem with the image sequence, reprocess the instruction;

[0174] The reprocessing instruction contains a set of characters representing the reprocessing of the standard dynamic image dataset by the nodule feature extraction module;

[0175] S8.5: Package the re-collection instructions, weight adjustment instructions, and reprocessing instructions to obtain a system optimization report;

[0176] The system data management module is used to process system data separately;

[0177] Furthermore, methods for processing system data separately include:

[0178] System data includes patient datasets, standard static image datasets, standard dynamic image datasets, static feature vectors, dynamic feature vectors, malignancy probability assessment values, final probability assessment values, system assessment values, patient diagnosis reports, system assessment reports, and system optimization reports;

[0179] Output the patient's diagnostic report to the doctor's visual panel receiver;

[0180] In this embodiment, the beneficial effects are achieved by receiving a patient dataset and preprocessing it to obtain a standard static image dataset. This standard image dataset is then processed to obtain static feature vectors. Continuous cross-sectional images of nodules are preprocessed to obtain a standard dynamic image dataset. This dynamic image dataset is then processed to obtain dynamic feature vectors. Based on the static and dynamic feature vectors, further processing is performed, and a probability assessment is conducted to obtain a malignancy probability assessment value. This malignancy probability assessment value is then analyzed to obtain a final probability assessment value and a system assessment value. Based on the final probability assessment value and the system assessment value, further analysis yields a patient diagnosis report and a system assessment report. Finally, the system assessment report is processed to obtain a system optimization report. By processing the system data separately, the system can effectively address the limitations of traditional methods for multimodal detection through multiple processing of the patient dataset. To address the issue of poor image data fusion, this invention further enhances continuous cross-sectional images and deeply mines dynamic image information through the establishment of image preprocessing enhancement and nodule feature extraction modules. This enables the system to effectively acquire malignant symptoms that are difficult to show in static images, greatly improving the system's sensitivity in detecting malignant nodules. Simultaneously, the model diagnosis and evaluation module further "gatekeepers" the accuracy of the system's diagnostic results. Combined with the visualization report generation module, the probability of malignant nodules and the "gatekeeper" results are clarified and traced back to their source, respectively. Finally, based on the system interaction optimization module, targeted optimization and adjustment measures are generated for the source-traced diagnostic evaluation, maximizing the system's practicality and sustainable use. Overall, this invention has significant advantages such as high efficiency in utilizing multi-source data, strong detection sensitivity, and good diagnostic process efficiency.

[0181] Example 2

[0182] Please see Figure 2 As shown, the parts not described in detail in this embodiment are described in Embodiment 1. An image processing method for detecting malignant nodules based on benign thyroid glands is provided. The method includes: S1: receiving a patient dataset and performing preprocessing to obtain a standard static image dataset;

[0183] S2: Process the standard image dataset to obtain static feature vectors;

[0184] S3: Preprocess the continuous cross-sectional images of the nodules to obtain a standard dynamic image dataset;

[0185] S4: Process the standard dynamic image dataset to obtain dynamic feature vectors;

[0186] S5: Process based on static and dynamic feature vectors, and perform probability evaluation based on the processing results to obtain the malignancy probability evaluation value;

[0187] S6: Analyze the malignancy probability assessment value to obtain the final probability assessment value and system assessment value;

[0188] S7: Based on the final probability assessment value and the system assessment value, analyze the results to obtain the patient diagnosis report and the system assessment report;

[0189] S8: Process the system evaluation report to obtain the system optimization report;

[0190] S9: Process system data separately.

[0191] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the present invention.

Claims

1. An image processing system for detecting malignant nodules based on benign thyroid glands, characterized in that, The system includes: an image receiving and preprocessing module, a feature vector fusion module, an image preprocessing enhancement module, a nodule feature extraction module, a multi-source feature calculation and evaluation module, a model diagnosis and evaluation module, a visualization report generation module, a system interaction optimization module, and a system data management module, wherein: The image receiving and preprocessing module is used to receive the patient dataset and preprocess it to obtain a standard static image dataset. The feature vector fusion module is used to process the standard image dataset to obtain static feature vectors; The image preprocessing enhancement module is used to preprocess the continuous cross-sectional images of the nodules to obtain a standard dynamic image dataset. The nodule feature extraction module is used to process a standard dynamic image dataset to obtain a dynamic feature vector; the steps for processing the standard dynamic image dataset include: S4.1: Calculate the motion velocity and direction of each pixel in the image sequence of the standard dynamic image dataset based on the optical flow method to obtain the motion vector field; S4.2: Calculate the deformation rate based on the motion vector field in step S4.1 to obtain the deformation rate; S4.3: Based on the motion vector field in step S4.1, and according to the 3D CNN model, spatiotemporal features are extracted to obtain spatiotemporal features; S4.4: Concatenate the deformation rate and spatiotemporal features to obtain the dynamic feature vector; S4.5: Output the dynamic feature vector to the multi-source feature calculation and evaluation module; The multi-source feature calculation and evaluation module is used to process static feature vectors and dynamic feature vectors, and to perform probability evaluation based on the processing results to obtain a malignancy probability evaluation value. The model diagnosis and evaluation module is used to analyze the malignancy probability evaluation value to obtain the final probability evaluation value and the system evaluation value. The visualization report generation module is used to analyze the final probability assessment value and system assessment value to obtain patient diagnosis report and system assessment report; The system interaction optimization module is used to process the system evaluation report to obtain a system optimization report; The system data management module is used to process system data separately.

2. The image processing system for detecting malignant nodules based on benign thyroid glands according to claim 1, characterized in that, The steps for receiving and preprocessing patient datasets include: S1.1: Obtain the specified patient dataset through the information management platform. The patient dataset includes B-mode, elasticity graph, and blood flow graph. S1.2: Based on the registration algorithm, different modal images of the same nodule location are registered, and the mutual information between the two images is maximized to obtain the registration dataset; S1.3: The registration dataset is standardized based on Z-score standardization to obtain a standard dataset; S1.4: Use Gaussian filtering to denoise the standard dataset to obtain a denoised dataset; S1.5: Automatically standardize and preserve regions based on nodule location, and crop the image outside the preserved regions to obtain a standard static image dataset; S1.6: Output the standard static image dataset to the feature vector fusion module.

3. The image processing system for detecting malignant nodules based on benign thyroid glands according to claim 1, characterized in that, The steps for processing a standard image dataset include: S2.1: Based on all modal images in the standard image dataset, deep feature extraction is performed using a pre-trained convolutional neural network, and the resulting feature vector sets are packaged together. S2.2: The weights of all feature vectors in the feature vector set are calculated based on the attention sub-network and then packaged to obtain the feature vector weight set; S2.3: Perform weighted summation calculation based on the feature vector weight set to obtain the static feature vector; S2.4: Output the static feature vector to the multi-source feature calculation and evaluation module.

4. The image processing system for detecting malignant nodules based on benign thyroid glands according to claim 1, characterized in that, The preprocessing steps for continuous cross-sectional images of nodules include: S3.1: Frame interval motion estimation is performed on continuously switching images using optical flow, and frame interval motion is stabilized using inverse transform to obtain a stable image dataset; S3.2: Use temporal median filtering to perform temporal neighborhood median values ​​on the pixels of all images in the stable image dataset to obtain the denoised image dataset; S3.3: Image enhancement is performed on the denoised image dataset based on the contrast-limited adaptive histogram equalization tool to obtain the enhanced image dataset; S3.4: The time intervals of the images in the enhanced image dataset are uniformly adjusted by linear interpolation to obtain a standard dynamic image dataset; S3.5: Output the standard dynamic image dataset to the nodule feature extraction module.

5. The image processing system for detecting malignant nodules based on benign thyroid glands according to claim 1, characterized in that, The steps for processing based on static and dynamic feature vectors, and then performing probability evaluation based on the processing results, include: S5.1: A joint feature vector is obtained by concatenating static and dynamic feature vectors; S5.2: Calculate the malignancy probability assessment value based on the joint feature vector and a deep neural network classifier; S5.3: Output the malignancy probability assessment value to the model diagnosis and assessment module.

6. The image processing system for detecting malignant nodules based on benign thyroid glands according to claim 1, characterized in that, The steps for analyzing the probability assessment value of malignancy include: S6.1: Based on Monte Carlo Dropout, perform N forward propagation operations on the malignancy probability assessment value to obtain N malignancy probability assessment values; S6.2: Calculate the final probability assessment value based on N malignancy probability assessment values; S6.3: Calculate the variance based on the final probability evaluation value in step S6.2 to obtain the system evaluation value; S6.4: Output the final probability assessment value and system assessment value to the visualization report generation module.

7. The image processing system for detecting malignant nodules based on benign thyroid glands according to claim 1, characterized in that, The steps for obtaining patient diagnosis reports and system evaluation reports based on the final probability assessment values ​​and system evaluation values ​​include: S7.1: Generate a probability heatmap based on the final probability evaluation value and according to the gradient-weighted class activation mapping technique; S7.2: By calculating the contribution values ​​of static and dynamic eigenvectors to the final probability evaluation value, a feature contribution map is obtained; S7.3: Based on the evaluation threshold, when the system evaluation value is greater than or equal to the evaluation threshold, a system evaluation report is generated; Factor reports include descriptions of poor multimodal image quality in the patient dataset, conflicts between static and dynamic feature vectors, and / or problems with the image sequences. S7.4: Pack the probability heatmap and feature contribution map to obtain the patient diagnosis report; S7.5: Output patient diagnosis reports to the system data management module and system evaluation reports to the system interaction optimization module.

8. The image processing system for detecting malignant nodules based on benign thyroid glands according to claim 1, characterized in that, The steps involved in processing based on the system evaluation report include: S8.1: Read the system evaluation report; S8.2: When the system evaluation report indicates poor multimodal image quality, generate a re-acquisition command; The reacquisition command contains a set of characters representing the image receiving preprocessing module reacquiring the specified patient dataset; S8.3: When there is a conflict between static and dynamic feature vectors in the system evaluation report, a weight adjustment instruction is generated; The weight adjustment instruction contains a set of characters representing the adjustment of the feature vector weights in the feature vector fusion module; S8.4: When the system evaluation report indicates that there is a problem with the image sequence, reprocess the instruction; The reprocessing instruction contains a set of characters representing the reprocessing of the standard dynamic image dataset by the nodule feature extraction module; S8.5: Package the re-collection command, weight adjustment command, and reprocessing command to obtain a system optimization report.

9. An image processing method for detecting malignant nodules based on benign thyroid glands, implemented according to any one of claims 1-8, characterized in that, The work includes the following steps: S1: Receive the patient dataset and preprocess it to obtain a standard still image dataset; S2: Process the standard image dataset to obtain static feature vectors; S3: Preprocess the continuous cross-sectional images of the nodules to obtain a standard dynamic image dataset; S4: Process the standard dynamic image dataset to obtain dynamic feature vectors; S5: Process based on static and dynamic feature vectors, and perform probability evaluation based on the processing results to obtain the malignancy probability evaluation value; S6: Analyze the malignancy probability assessment value to obtain the final probability assessment value and system assessment value; S7: Based on the final probability assessment value and the system assessment value, analyze the results to obtain the patient diagnosis report and the system assessment report; S8: Process the system evaluation report to obtain the system optimization report; S9: Process system data separately.

Images