Image lesion segmentation system based on convolutional neural network

Abstract

The application discloses an image lesion segmentation system based on a convolutional neural network, and relates to the technical field of image segmentation.The system comprises a data acquisition module, a cross-modal decoding module, a joint optimization module and a lesion segmentation module.The technical key points are as follows: the CT safe coding array and the MRI safe coding array are subjected to cross-modal decoding integration to generate a fusion lesion segmentation array;based on the correlation between the fusion lesion segmentation array and a model parameter array, the CT lesion feature array, the CT segmentation parameter array, the MRI lesion feature array and the MRI segmentation parameter array are subjected to joint optimization debugging to generate optimized segmentation model parameters;the optimized segmentation model parameters are used to generate a target lesion segmentation array of a patient set for CT images and MRI images;and based on the target lesion segmentation array, the priority marking result of the lesion area of the patient is determined.The application improves the accuracy of lesion segmentation and improves the model calculation efficiency.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of image segmentation, in particular to an image lesion segmentation system based on a convolutional neural network. BACKGROUND

[0002] Medical image lesion segmentation is one of the core technologies of a computer-aided diagnosis (CAD) system, aiming to accurately delineate lesion regions (such as tumors, hemorrhagic foci, tissue abnormalities, etc.) from CT, MRI and other images, and to provide quantitative basis for disease diagnosis, treatment planning and prognosis evaluation; early medical image segmentation technology mainly relies on manually defined features and thresholds or morphological rules, with the explosive growth of medical image data and the breakthrough of deep learning technology, automatic segmentation methods based on convolutional neural networks gradually replace traditional technologies and become a research hotspot in this field.

[0003] In the existing image lesion segmentation system, convolution operation performs excellently in medical image synthesis tasks, but convolution operation limits the expression ability of context features reflecting long-distance spatial dependence, resulting in poor medical image synthesis quality and poor display of lesion regions in the synthesized pictures; at the same time, due to the shortcomings of CT image segmentation and MRI image segmentation, such as: the limited density resolution of CT to soft tissue, the large intensity distribution difference between different scanning sequences (T1, T2, FLAIR) in MRI images, leading to poor cross-center data generalization, which cannot accurately distinguish CT images and MRI images in single modal form, hindering image registration of CT images and MRI images, and further affecting data or image fusion in cross-modal form;

[0004] In addition, in the process of image segmentation, a fixed weight allocation strategy is usually adopted, which cannot dynamically adapt to data quality changes, resulting in a decline in overall segmentation accuracy; for example, in brain glioma segmentation, CT is sensitive to calcification area, and MRI is better for edema zone, but fixed weight cannot dynamically adjust between calcification area (CT dominant) and edema zone (MRI dominant); for example, when there is serious noise (such as low-dose CT) or artifacts (such as MRI motion artifacts) in a modality, fixed weight still gives it a fixed contribution, introducing false segmentation; for lesion segmentation results, the traditional influence lesion segmentation system is not optimized with model parameters, resulting in a decline in resource calculation and insufficient real-time performance. SUMMARY

[0005] (I) Technical problems solved

[0006] In view of the deficiencies of the prior art, the present application provides an image lesion segmentation system based on a convolutional neural network, comprising a data acquisition module, a cross-modal decoding module, a joint optimization module and a lesion segmentation module, specific analysis is performed on single CT images, single MRI images and CT and MRI images by designing single modal forms and cross-modal forms, the accuracy of data analysis is improved, the accuracy of image registration of CT images and MRI images is improved, in the process of lesion segmentation, a confidence weight is designed to dynamically adapt to data quality changes, the overall segmentation accuracy is improved, the joint optimization of related parameters and segmentation model parameters is introduced into the whole system to construct a CNN network, model calculation is accelerated, and the problems in the background art are solved.

[0007] (II) Technical solutions

[0008] To achieve the above object, the present application is implemented by the following technical solutions:

[0009] The present application provides an image lesion segmentation system based on a convolutional neural network, the system comprising:

[0010] A data acquisition module acquires a CT safety coding array, a CT lesion feature array and a CT segmentation parameter array, and an MRI safety coding array, an MRI lesion feature array and an MRI segmentation parameter array;

[0011] A cross-modal decoding module performs cross-modal decoding integration on the CT safety coding array and the MRI safety coding array, enhances CT features and MRI features based on different modal forms, and obtains a fusion lesion segmentation array;

[0012] A joint optimization module performs joint optimization and debugging on the CT lesion feature array, the CT segmentation parameter array, the MRI lesion feature array and the MRI segmentation parameter array based on the correlation between the fusion lesion segmentation array and a model variable array, and generates optimized segmentation model parameters;

[0013] A lesion segmentation module generates a target lesion segmentation array for CT images and MRI images of a patient set by using the optimized segmentation model parameters, and determines a priority labeling result of a patient lesion area based on the target lesion segmentation array.

[0014] Further, the CT safety coding array is a safety coding array of a lesion segmentation array for CT images of the patient set;

[0015] The CT lesion feature array and the CT segmentation parameter array are model variable arrays of the CT lesion segmentation array;

[0016] The MRI safety coding array is a safety coding array of a lesion segmentation array for MRI images of the patient set;

[0017] The MRI lesion feature array and the MRI segmentation parameter array are model variable arrays of the MRI lesion segmentation array.

[0018] Further, the step of cross-modality decoding integration includes:

[0019] Decrypting the CT security coding array to output the CT image;

[0020] Decrypting the MRI security coding array to output the MRI image;

[0021] Judging whether it is in a cross-modality form;

[0022] When the decrypted result only has the CT image or the MRI image, it is determined to be in a single-modality form;

[0023] When the decrypted result includes the CT image and the MRI image, it is determined to be in a cross-modality form, and the CT image and the MRI image are non-rigidly registered, and the CT features and the MRI features are fused through channel concatenation and attention mechanism;

[0024] The feature variance and the modality contribution ratio of each modality form are calculated, and a corresponding segmentation branch is dynamically selected to generate a corresponding fused lesion segmentation array; wherein the segmentation branch at least includes one of a straight line segment branch, an inclined line segment branch, a curved line segment branch, a high complexity branch and a basic branch.

[0025] Further, the step of determining the single-modality form includes:

[0026] Only the CT image: constructing a CT energy spectrum curve based on the CT image, identifying a peak value region in the CT energy spectrum curve using a local maximum value method, activating a residual block structure in the peak value region, extracting corresponding energy values, intensity values and penetration depths based on the peak value region, establishing an energy value / intensity value-penetration depth / intensity value two-dimensional graph and determining a line segment type, thereby performing feature fusion in the residual block structure, and the weight of the fusion is automatically adjusted based on the proportion of the line segment type; wherein the line segment type includes a straight line segment, an inclined line segment and a curved line segment;

[0027] Only the MRI image: extracting multi-sequence MRI data based on the MRI image, including T1WI, T2WI and QSM;

[0028] Extracting an edema region based on the T2WI to generate an edema mask;

[0029] Extracting a metal deposition region based on the QSM, including at least one of a copper deposition region and an iron deposition region;

[0030] Generating a deposition mask in combination with the T1WI;

[0031] When the edema region and the metal deposition region have spatial overlap, or the proportion of the metal deposition region exceeds a preset threshold, a high complexity branch is triggered, T1WI and T2WI are taken as query vectors, and the deposition mask and the edema mask are taken as key values to obtain an attention weight matrix to enhance the MRI features; otherwise, a basic branch is triggered.

[0032] Meanwhile, the bottleneck block structure is activated to compress and expand the outputs of the high complexity branch and the basic branch.

[0033] Further, the association relationship is realized by at least one of the following ways:

[0034] Based on the Pearson correlation coefficient or the Spearman rank correlation coefficient of the fusion lesion segmentation array and the model parameter array, the correlation strength between the two is quantified.

[0035] Further, the step of joint optimization debugging includes:

[0036] Based on the CT lesion feature array, the CT segmentation parameter array, the MRI lesion feature array, the MRI segmentation parameter array, and the association relationship of the fusion lesion segmentation array, the Nth round of parameter debugging is performed;

[0037] The CT lesion feature array, the CT segmentation parameter array, the MRI lesion feature array, and the MRI segmentation parameter array after the Nth round of debugging are subjected to lesion reasoning;

[0038] Based on the Nth CT lesion reasoning array, the Nth MRI lesion reasoning array, and a preset error calculation function, an Nth segmentation error value is obtained;

[0039] When the absolute difference between the Nth segmentation error value and the Mth segmentation error value is less than a preset threshold, the segmentation model parameters after the Nth round of debugging are determined as the final optimization result; otherwise, M=N, N=N+1, and the iteration is returned;

[0040] Wherein, N is a natural number greater than 0, and the gradient descent algorithm is used to update the CT segmentation parameters or the MRI segmentation parameters in each round of debugging.

[0041] Further, the step of lesion reasoning includes:

[0042] Load the model architecture optimized in the Nth round of debugging, select the reasoning form, including single modality reasoning and cross-modality reasoning;

[0043] Based on the reasoning form, the CT lesion feature array, the CT segmentation parameter array, the MRI lesion feature array, and the MRI segmentation parameter array are taken as model condition inputs, and the predicted lesion probability is output.

[0044] Further, the algorithm model of the error calculation function is:

[0045] In the single-modal form, a Dice coefficient error function is used;

[0046] In the cross-modal form, a weighted joint Dice loss function is used.

[0047] Further, the step of priority labeling result comprises:

[0048] Loading the optimized segmentation model parameters to obtain the confidence weights of the CT lesion segmentation array and the MRI lesion segmentation array, and the confidence weights are based on at least one of the probability graph entropy or the proportion of the cross-modal overlapping area;

[0049] Based on the confidence weights, the weighted summation is obtained to obtain a score evaluation coefficient Priority;

[0050] The score evaluation coefficient is compared and analyzed with a standard score interval [DF min , DF max ];

[0051] When Priority≤DF min , Priority is marked as a, edited as a first-level character, and a and the first-level character are combined to generate a low-priority lesion area;

[0052] When DF min <Priority≤DF max , Priority is marked as b, edited as a second-level character, and b and the second-level character are combined to generate a medium-priority lesion area;

[0053] When Priority>DF max , Priority is marked as c, edited as a third-level character, and c and the third-level character are combined to generate a high-priority lesion area.

[0054] (Three) beneficial effects

[0055] The present application provides an image lesion segmentation system based on a convolutional neural network, which has the following beneficial effects:

[0056] 1、The present application directly accesses the original feature value through the decoding operation, restores the spatial properties, numerical continuity and metadata integrity of the CT image and the MRI image, supports statistical modeling and machine learning, makes it possible to align coordinates, associate features and perform model reasoning required for cross-modal fusion, facilitates the analysis of the mutual fusion of CT and MRI in the later stage, and improves the diagnostic accuracy and reliability;

[0057] 2、The application designs two different modal forms, including a single modal form and a cross-modal form, in the single modal form, for CT images, a two-dimensional graph is constructed with the energy value / intensity value as the X-axis and the penetration depth / intensity value as the Y-axis, the line segment type is determined based on the two-dimensional graph, the residual block structure is activated, and the CT features are enhanced according to different line segment types; for MRI images, the edema area and the metal deposition area are extracted, and the bottleneck block structure is activated to compress and expand the output channels of the high-complexity branch and the basic branch; in the cross-modal form, through the non-rigid registration and fusion of CT images and MRI images, the registration offset problem caused by the resolution and contrast difference of CT and MRI is solved, and the accuracy of lesion segmentation is improved;

[0058] 3、The application sets the confidence weight to dynamically adjust the contribution of each modal form, significantly improves the accuracy and clinical practicability of medical image segmentation, and adaptively enhances the result reliability, thereby providing more reliable decision support for AI-assisted diagnosis. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 is a module schematic diagram of an image lesion segmentation system according to an exemplary embodiment;

[0060] Figure 2 is a module block diagram in a single modal form according to an exemplary embodiment;

[0061] Figure 3 is a flowchart under CT and MRI non-rigid registration fusion. DETAILED DESCRIPTION

[0062] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the application.

[0063] EMBODIMENT

[0064] The embodiment of the application provides an image lesion segmentation system based on a convolutional neural network; Figure 1 is a module schematic diagram of an image lesion segmentation system according to an exemplary embodiment; Figure 2 is a module block diagram in a single modal form according to an exemplary embodiment; Figure 3 is a flowchart under CT and MRI non-rigid registration fusion. Figure 1 - Figure 3The system comprises a data acquisition module, a cross-modal decoding module, a joint optimization module, and a lesion segmentation module, and the data acquisition module, the cross-modal decoding module, the joint optimization module, and the lesion segmentation module are in communication connection;

[0065] The data acquisition module acquires a CT safe coding array, a CT lesion feature array, and a CT segmentation parameter array uploaded by a CT image platform, and an MRI safe coding array, an MRI lesion feature array, and an MRI segmentation parameter array uploaded by an MRI image platform;

[0066] The CT safe coding array is a safe coding array of a lesion segmentation array of a patient set for a CT image;

[0067] The CT lesion feature array and the CT segmentation parameter array are model variable arrays of the CT lesion segmentation array;

[0068] The MRI safe coding array is a safe coding array of a lesion segmentation array of a patient set for an MRI image;

[0069] The MRI lesion feature array and the MRI segmentation parameter array are model variable arrays of the MRI lesion segmentation array;

[0070] The following explains the terms involved:

[0071] The safe coding array is an encoding array composed of encrypted and anonymized image data;

[0072] The lesion feature array is a deep feature extracted by a convolutional neural network (CNN), i.e., a quantitative feature set extracted from a lesion segmentation result; for example, a quantitative feature set extracted from a CT lesion segmentation result;

[0073] The segmentation parameter array is a configuration parameter set required for accurate positioning of a lesion area, for example, abnormal tissue areas (such as deposition areas, edema areas) are labeled pixel by pixel / voxel in CT images or MRI images, and a binary mask or a probability process is generated, and corresponding trainable parameters or hyperparameters (such as learning rate, iteration number) are generated;

[0074] The lesion feature array and the segmentation parameter array together constitute the input or configuration parameters of the model, which are used to perform the lesion segmentation task of medical images; the cooperative optimization (such as automatic parameter adjustment) of the two is the key to improving the accuracy of medical image analysis, and needs to be clearly distinguished and reasonably applied, especially in clinical diagnosis and scientific research;

[0075] The lesion segmentation array is a lesion segmentation result output by the system in the segmentation process, and a feature set is obtained quantitatively;

[0076] The security coding array, the lesion feature array, and the segmentation parameter array are uploaded by the image platform, and the three arrays jointly construct a safe, accurate, and expandable medical image analysis system, aiming to optimize the cooperation between different modalities (for example: single modality CT, single modality MRI, cross modality CT and MRI), ensure the consistency and adaptability of the lesion segmentation result, and be suitable for tumor screening, emergency diagnosis and surgical planning scenes;

[0077] The cross-modality decoding module decodes and integrates the CT security coding array and the MRI security coding array, enhances CT features and MRI features based on different modalities, and obtains a fusion lesion segmentation array;

[0078] The steps of cross-modality decoding and integration include:

[0079] Decrypt the CT security coding array to output the CT image;

[0080] Decrypt the MRI security coding array to output the MRI image;

[0081] For example: data upload of the CT image platform, the CT image platform encrypts the collected CT image and patient data to generate a CT security coding array; for example: differential privacy processing of the pixel coordinates of the CT image, adding controllable noise or using homomorphic encryption for lesion location, the specific selection depends on security requirements and performance considerations; the encrypted array is uploaded to the computer system in binary or Base64 encoding form to ensure the security of the data during transmission and storage;

[0082] Through encryption operation, the patient set and its image data (CT image, MRI image) can be traced, and the above data is usually stored in the form of binary ciphertext or hash value in the blockchain, realizing the association of the hashed patient ID, at the same time, setting out the role-based (RBAC) control decryption permission (such as only radiologists can decrypt CT), forming a chain structure, realizing dynamic authorization; through decoding operation, the spatial attributes, numerical continuity and metadata integrity of the CT image and the MRI image are restored, supporting statistical modeling and machine learning, making it possible for cross-modality fusion to require coordinate alignment, feature association and model reasoning, facilitating the analysis of the mutual fusion of CT and MRI in the later stage, and improving the diagnostic accuracy and reliability; in addition, the decoded data needs to be re-encrypted and stored, and the access log retention time is not less than the set data life cycle, and when the preset data life cycle is exceeded, it will be automatically deleted;

[0083] Determine whether it is a cross-modality form:

[0084] When the decrypted result is only CT image or MRI image, it is determined as single modality form; otherwise, when the decrypted result includes CT image and MRI image, it is determined as cross modality form;

[0085] When it is single modality form:

[0086] Only CT image:

[0087] A CT spectral curve is constructed based on the CT image, a local maximum value method is used to identify a peak value region in the CT spectral curve, and a residual block structure is activated in the peak value region, so as to force the system to have higher segmentation accuracy in the peak value region;

[0088] Specifically, the steps of the local maximum value method include:

[0089] The CT spectral curve describes the law of the attenuation characteristics of X-rays when passing through a substance with the change of photon energy, and can be applied to the X-ray source spectrum curve for optimizing the CT scanning parameters, or the material attenuation spectrum curve; the CT spectral curve is used to analyze the material composition and distinguish different tissues or contrast agents (such as iodine and calcium);

[0090] The attenuation coefficient of each X-ray photon energy point in the CT spectral curve is scanned, and the points with a neighborhood higher than Δμ are marked as peak values; wherein Δμ represents the peak value amplitude, which is the difference between the peak point value and the baseline; the threshold filtering is used to remove the pseudo-peak caused by noise, and the effective peak point is determined according to the effective peak point; in addition, the CT spectral curve may have noise, therefore, the CT spectral curve needs to be smoothed, such as using Gaussian filtering, median filtering or other filtering methods to remove noise;

[0091] Based on the peak value region, the corresponding energy value, intensity value and penetration depth are extracted, a two-dimensional graph is drawn with the energy value / intensity value as the X-axis and the penetration depth / intensity value as the Y-axis, and the line segment type is determined based on the two-dimensional graph, including straight line segment, inclined line segment and curve segment;

[0092] The steps of determining the line segment type are as follows:

[0093] When at least one of the horizontal line or the vertical line exists, it is determined as a straight line segment, and a two-point algorithm is used to uniformly enhance the CT features: It is used to suppress redundant information and improve calculation efficiency;

[0094] In the formula, α (E, d) represents a gain function for uniformly enhancing the CT features, E represents the energy value, E0 represents the center energy, ΔE represents the energy value range, α base1 And α base2 Both represent the reference gain, and α base1 > α base2> 0, d represents the penetration depth, L0 represents the horizontal line, and L1 represents the vertical line;

[0095] Formula explanation: When it is a horizontal line, the penetration depth contrast is stretched based on the energy value range; when it is a vertical line, the penetration depth is introduced to enhance the energy response, suppress metal artifacts, and strengthen the visualization of soft tissues; the mutual combination of the horizontal line and the vertical line realizes the precise visualization of the CT image, and provides high-value image support for clinical diagnosis and treatment planning;

[0096] When at least one of the positive or negative slope straight lines exists, it is determined as a diagonal line segment, and a bilinear interpolation algorithm is used to linearly enhance the CT features, which is used to strengthen the structure boundary and suppress noise;

[0097] For example: the inflection point position is extracted in the diagonal line segment, and a weight matrix is defined at the inflection point, and the bilinear interpolation is applied to each sub-line segment to ensure smooth transition at the connection, combine the enhancement effect of each sub-line segment, and eliminate boundary mutation;

[0098] When at least one of the exponential and polynomial exists, it is determined as a curve segment, and a preset nonlinear model algorithm is used to complexly enhance the CT features, which is used to capture nonlinear features and enhance the model's discriminability for complex pathologies;

[0099] For example: in the curve segment, a Sigmoid or polynomial function is used to enhance the high-contrast region (such as calcification) and enhance the high-frequency details of the curve segment;

[0100] Wherein, the residual block structure includes a straight line segment branch, a diagonal line segment branch, and a curve segment branch, by determining the line segment type, capturing linear and nonlinear relationships, and effectively extracting deep features of CT energy spectrum; and then performing feature fusion in the residual block structure: , and the fusion weight is automatically adjusted based on the proportion of the line segment type; in the formula, F fuse represents the fused CT features, i represents the type of the line segment, including a straight line segment, a diagonal line segment, and a curve segment, represents the CT features obtained under the i-th line segment type, λ represents the weight, including uniform enhancement weight, linear enhancement weight, and nonlinear enhancement weight, and the uniform enhancement weight, the linear enhancement weight, and the nonlinear enhancement weight correspond to the straight line segment, the diagonal line segment, and the curve segment one by one, and λ i =1;

[0101] For example: if the curve segment proportion is 60%, the nonlinear enhancement weight is 0.6;

[0102] Only MRI images:

[0103] Based on the MRI images, multi-sequence MRI data is extracted, including T1WI, T2WI, and QSM;

[0104] Extract the edema region in combination with T2WI to generate an edema mask;

[0105] Specifically, the edema region can be identified by a signal intensity threshold segmentation method, because the edema region has increased water content, resulting in prolonged T2 relaxation time, and appears as a high signal on T2WI. The above steps are not described in detail;

[0106] In combination with QSM, extract the metal deposition region, at least one of the copper deposition region and the iron deposition region;

[0107] In combination with T1WI, generate a deposition mask;

[0108] When the edema region and the metal deposition region exist spatial overlap, or the metal deposition region accounts for more than a preset threshold, for example: the T1WI signal intensity in the copper deposition region is higher than the preset percentile (generally, the value is 10% of the voxel), the high complexity branch is triggered, T1WI and T2WI are used as query vectors, and the deposition mask and the edema mask are used as key values to obtain an attention weight matrix to enhance the MRI features;

[0109] Conversely, the basic branch is triggered;

[0110] Activate the bottleneck block structure to compress and expand the output of the high complexity branch and the basic branch;

[0111] Specifically, metal deposition (such as intracerebral iron deposition, calcification, or hemorrhagic residue) appears as a significant increase in local magnetization rate in QSM (quantitative susceptibility mapping), and the joint analysis of QSM and T1WI can significantly improve the specificity of the metal deposition region, especially in differentiating different pathological types such as calcification, iron deposition, and hemorrhage. Therefore, the metal deposition region is the basis for image lesion analysis, for example: the cause of Wilson's disease is ATP7B gene mutation leading to copper metabolism disorder, and copper deposition in the liver, cornea and basal ganglia; and the cause of Parkinson's disease is the degeneration of dopaminergic neurons in the substantia nigra compacta, accompanied by abnormal deposition of iron (Fe 3+ ) ions; by analyzing the metal deposition region, the image lesion segmentation is more accurate; at the same time, the edema region and the metal deposition region are considered, so that the MRI features are extracted more deeply;

[0112] When it is a cross-modality form: non-rigid registration is performed on CT images and MRI images, and CT features and MRI features are fused through channel concatenation and attention mechanism;

[0113] Specifically, the non-rigid registration includes using the Demons algorithm or B-spline free form deformation, optimizing mutual information as a similarity measure, balancing the segmentation error of CT and MRI through multiple rounds of debugging to avoid model overfitting to a single modality; based on the CT image and the MRI image to be registered, a large number of data sets are formed, and are divided into a test set, a training set and a validation set; in the training process, a double UNet network is used, and deep features of the CT image and the MRI image are extracted through a CT feature extractor and an MRI feature extractor respectively; the CT features and the MRI features are concatenated in the channel to generate concatenated features; the concatenated features are dynamically weighted and fused in the spatial and channel dimensions through an attention fusion module to generate fused features; the model weights of each epoch are saved during the training process, the validation set is inferred to generate a lesion probability map and evaluation indicators, and model evaluation on the validation set is performed after each epoch of training, for example: selecting the model weight with the highest Dice coefficient of the validation set, if the Dice is the same, preferentially selecting the model with the smaller HD95;

[0114] The feature variance of each modality form and the corresponding modality contribution ratio are calculated, and the corresponding segmentation branch is dynamically selected to generate the corresponding fusion lesion segmentation array; wherein the segmentation branch at least includes one of a straight line segment branch, an inclined line segment branch, a curved line segment branch, a high complexity branch and a basic branch;

[0115] The joint optimization module is based on the correlation between the fusion lesion segmentation array and the model parameter array, and performs joint optimization and debugging on the CT lesion feature array, the CT segmentation parameter array, the MRI lesion feature array and the MRI segmentation parameter array to generate the optimized segmentation model parameters;

[0116] The step of joint optimization and debugging includes:

[0117] Specifically, the image lesion segmentation system is based on a convolutional neural network (CNN) architecture, and at least includes a generator and a discriminator; the generator is composed of an encoder, a bottleneck layer and a decoder, and the encoder and the decoder include a plurality of residual block structures or bottleneck block structures for extracting and reconstructing metal deposition features, edema features and the like in the image;

[0118] In addition, a generative adversarial network (GAN) is also arranged on the CNN network to balance the segmentation accuracy difference between CT and MRI, wherein the discriminator network distinguishes the segmentation result features of CT and MRI, and correspondingly obtains the CT segmentation parameter array and the MRI segmentation parameter array and the corresponding fusion lesion segmentation array;

[0119] Based on the correlation between the CT lesion feature array, the CT segmentation parameter array, the MRI lesion feature array, the MRI segmentation parameter array and the corresponding fusion lesion segmentation array, the Nth round of parameter debugging is performed to obtain:

[0120] the CT lesion feature array after the Nth round of debugging, the CT segmentation parameter array after the Nth round of debugging,

[0121] the MRI lesion feature array after the Nth round of debugging, the MRI segmentation parameter array after the Nth round of debugging;

[0122] wherein N is a natural number greater than 0, and the gradient descent algorithm is used to update the CT segmentation parameter or the MRI segmentation parameter in each round of debugging, and the lesion feature distribution of CT and MRI is aligned through feature space projection;

[0123] and the association is achieved through at least one of the following ways:

[0124] The Pearson correlation coefficient or the Spearman rank correlation coefficient of the fusion lesion segmentation array and the model variable array is obtained to quantify the correlation between the two and reflect the correlation strength;

[0125] Specifically, the Pearson correlation coefficient measures the linear relationship between the two, which is suitable for normally distributed data (e.g., volume features, regular shape features); the Spearman rank correlation coefficient measures the monotonic relationship, which is suitable for non-normal or non-linear data (e.g., texture features, hyperparameters); the specific steps are not described in detail;

[0126] The CT lesion feature array after the Nth round of debugging, the CT segmentation parameter array, the MRI lesion feature array, and the MRI segmentation parameter array are subjected to lesion reasoning to obtain:

[0127] the Nth CT lesion reasoning array;

[0128] the Nth MRI lesion reasoning array;

[0129] wherein the steps of lesion reasoning include:

[0130] loading the model architecture optimized in the Nth round of debugging, and selecting a reasoning form, including single-modality reasoning and cross-modality reasoning;

[0131] Based on the reasoning form, the CT lesion feature array, the CT segmentation parameter array, the MRI lesion feature array, and the MRI segmentation parameter array are input as model conditions to output the predicted lesion probability;

[0132] Single-modality reasoning:

[0133] The CT lesion feature array and the CT segmentation parameter array are input as model conditions, for example, the model architecture selects a 3D U-Net encoder, and the CT lesion probability is output;

[0134] Input the MRI lesion feature array and the MRI segmentation parameter array as the model condition, for example: select Swin-Transformer encoder in the model architecture, output the MRI lesion probability;

[0135] Cross-modal inference: input the CT lesion feature array, CT segmentation parameter array, MRI lesion feature array, MRI segmentation parameter array, and fusion lesion segmentation array as the model condition, and output the predicted lesion probability;

[0136] Based on the Nth CT lesion inference array, the Nth MRI lesion inference array, and the preset error calculation function, obtain the Nth segmentation error value;

[0137] When the absolute difference between the Nth segmentation error value and the Mth segmentation error value is less than the preset threshold, the segmentation model parameters after the Nth iteration are determined as the final optimization result; otherwise, let M=N, N=N+1, and return to iteration;

[0138] Wherein, the Nth segmentation error value includes the Nth CT segmentation error value corresponding to the Nth CT lesion inference array and the Nth MRI segmentation error value corresponding to the Nth MRI lesion inference array; when the absolute difference between the Nth segmentation error value and the Mth segmentation error value is less than the preset threshold, the absolute value of the Nth CT segmentation error value and the Mth CT segmentation error value is less than the preset threshold, and the absolute value of the Nth MRI segmentation error value and the Mth MRI segmentation error value is less than the preset threshold;

[0139] The algorithm model of the preset error calculation function is:

[0140] Under the single-modal form, use the Dice coefficient error function;

[0141] Under the cross-modal form, use the weighted joint Dice loss function;

[0142] The following is an explanation of the terms involved:

[0143] Fusion lesion segmentation array: a comprehensive data matrix generated by the segmentation results of medical images under each modal form (e.g., single CT, single MRI, CT and MRI under the cross-modal form);

[0144] Lesion inference array: a parameter set for decision-making in the lesion area, based on high-level semantic information generated by the lesion segmentation array and cross-modal data, and a quantified feature set;

[0145] Segmentation model parameters: refer to the weights, biases, and convolution kernels inside the trained model;

[0146] The lesion segmentation module generates a target lesion segmentation array for the CT image and the MRI image of the patient set by using the optimized segmentation model parameters; and determines a priority labeling result of the patient lesion area based on the target lesion segmentation array.

[0147] The step of determining the priority labeling result comprises:

[0148] The optimized segmentation model parameters are loaded to obtain the confidence weight of the CT lesion segmentation array and the MRI lesion segmentation array, and the confidence weight is based on at least one of the probability map entropy or the proportion of the cross-modality overlapping area.

[0149] The score evaluation coefficient Priority is obtained by the formula:

[0150] In the formula, β1 and β2 are adjustment coefficients, and β1+β2=1, represents the confidence weight based on the proportion of the cross-modality overlapping area, represents the confidence weight based on the probability map entropy, represents the normalized probability map entropy, J represents the total number of modes, and takes a value of 1 or 2, when it is a single modality form, the value is 1, and when it is a cross-modality form, the value is 2.

[0151] The score evaluation coefficient is compared and analyzed with the standard score interval [DF min , DF max ].

[0152] When Priority≤DF min , Priority is marked as a, edited as a first-level character at the same time, and a and the first-level character are combined to generate a low-priority lesion area.

[0153] When DF min <Priority≤DF max , Priority is marked as b, edited as a second-level character at the same time, and b and the second-level character are combined to generate a medium-priority lesion area.

[0154] When Priority>DF max , Priority is marked as c, edited as a third-level character at the same time, and c and the third-level character are combined to generate a high-priority lesion area.

[0155] For example, for an XX patient, the CT image and the MRI image of the slope area are obtained.

[0156] ​Based on the CT image, the CT energy spectrum curve is constructed, the calcification form (spot, ring or bow) of the tumor, the bone destruction type (osteolytic, expansive, ground glass-like), and the adjacent bone changes (scallop-like indentation, hardening edge) are identified based on the enhanced CT features of each line segment type; for example: assuming only vertical lines, the soft tissue visualization degree is enhanced, and the calcification form can be more clearly identified; based on the MRI image, the edema area and the metal deposition area are extracted by the system, and the edema within 3mm around the calcification indicates the active period of the tumor; assuming that the cross-modal form is determined, the CT features and the MRI features are fused to further enhance the features and obtain the fused lesion segmentation array;

[0157] The following lists the enhanced feature descriptions, see Table 1:

[0158] Table 1 Enhanced feature description data

[0159]

[0160] The low-priority lesion area, the medium-priority lesion area, and the high-priority lesion area are transmitted to the Picture Archiving and Communication System (PACS) as priority labeling results in real time, and are displayed in the visualization interface of the doctor workstation, and the labeling results support DICOM standard format export; in addition, the priority labeling results conform to the doctor's diagnosis logic, for example: for the medium-priority lesion area, a multidisciplinary consultation is required within 72 hours to develop a surgery / radiotherapy plan; the confidence-driven weight adjusts the contribution of each modality by quantifying the single-modality uncertainty and the cross-modality consistency, solving the problem of rigid weight and dependence on prior knowledge in traditional methods, and significantly improving the accuracy and clinical practicability of medical image segmentation; its core value lies in self-adaptive data quality and enhanced result reliability, providing more reliable decision support for AI-assisted diagnosis.

[0161] In the application, the several formulas involved are calculated by taking the values after de-dimensioning, and the formula is a formula obtained by software simulation of a large amount of data to reflect the current real situation, and the parameters in the formula are set by a person skilled in the art according to the actual situation.

[0162] The above embodiments can be realized wholly or partially by software, hardware, firmware or any other combination. When realized by software, the above embodiments can be realized wholly or partially in the form of a computer program product. Those skilled in the art can realize that the units and algorithm steps of the examples described in connection with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized by hardware or software depends on the specific application and design constraints of the technical solutions.

[0163] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, and may be located in one place, or distributed on multiple network units. Part or all of the units can be selected to achieve the purpose of the embodiment of the present application according to actual needs.

[0164] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. An image lesion segmentation system based on a convolutional neural network, characterized by, The system comprises: a data acquisition module for acquiring CT safe coding arrays, CT lesion feature arrays and CT segmentation parameter arrays, and MRI safe coding arrays, MRI lesion feature arrays and MRI segmentation parameter arrays; a cross-modal decoding module for performing cross-modal decoding integration on the CT safe coding arrays and the MRI safe coding arrays, enhancing CT features and MRI features based on different modal forms, and obtaining a fused lesion segmentation array; a joint optimization module for performing joint optimization and debugging on the CT lesion feature arrays, the CT segmentation parameter arrays, the MRI lesion feature arrays and the MRI segmentation parameter arrays based on the correlation between the fused lesion segmentation array and the model parameter array, and generating optimized segmentation model parameters; a lesion segmentation module for generating target lesion segmentation arrays for CT images and MRI images of the patient set by using the optimized segmentation model parameters, and determining priority labeling results of patient lesion regions based on the target lesion segmentation arrays; wherein the cross-modal decoding integration step comprises: decrypting the CT safe coding arrays to output CT images; decrypting the MRI safe coding arrays to output MRI images; determining whether it is a cross-modal form: when the decrypted result only has CT images or MRI images, it is determined to be a single-modal form; when the decrypted result includes CT images and MRI images, it is determined to be a cross-modal form, and the CT images and the MRI images are non-rigidly registered, and CT features and MRI features are fused through channel concatenation and attention mechanism; calculating the feature variance and modal contribution ratio of each modal form, dynamically selecting the corresponding segmentation branch, and generating the corresponding fused lesion segmentation array; wherein the segmentation branch at least includes one of a straight line segment branch, an inclined line segment branch, a curved line segment branch, a high complexity branch and a basic branch.

2. The image lesion segmentation system based on the convolutional neural network according to claim 1, wherein: the CT safe coding arrays are the safe coding arrays of the lesion segmentation arrays of the CT images of the patient set; the CT lesion feature arrays and the CT segmentation parameter arrays are the model parameter arrays of the CT lesion segmentation arrays; the MRI safe coding arrays are the safe coding arrays of the lesion segmentation arrays of the MRI images of the patient set; the MRI lesion feature arrays and the MRI segmentation parameter arrays are the model parameter arrays of the MRI lesion segmentation arrays. 3.The convolutional neural network-based image lesion segmentation system of claim 1, wherein, the step of determining a single-modal form comprises: only CT images: constructing a CT energy spectrum curve based on the CT images, identifying peak regions in the CT energy spectrum curve using a local maximum method, activating a residual block structure in the peak regions, extracting corresponding energy values, intensity values and penetration depths based on the peak regions, establishing an energy value / intensity value-penetration depth / intensity value two-dimensional graph, and determining a line segment type, thereby performing feature fusion in the residual block structure, and the weight of the fusion is automatically adjusted based on the proportion of the line segment type; wherein the line segment type includes a straight line segment, an inclined line segment and a curved line segment; only MRI images: extracting multi-sequence MRI data including T1WI, T2WI and QSM based on the MRI images; extracting edema regions based on the T2WI to generate an edema mask; One kind based on QSM extracts metal deposition area, at least including copper deposition area, iron deposition area; And generate deposition mask in combination with T1WI; When the edema area and the metal deposition area exist spatial overlap, or the metal deposition area accounts for more than a preset threshold, a high complexity branch is triggered, T1WI and T2WI are used as query vectors, and the deposition mask and the edema mask are used as key values to obtain an attention weight matrix to enhance the MRI features; otherwise, a basic branch is triggered; At the same time, the bottleneck block structure is activated to compress and expand the output of the high complexity branch and the basic branch. 4.The convolutional neural network-based image lesion segmentation system of claim 1, wherein, The association relationship is realized by at least one of the following ways: Based on the Pearson correlation coefficient or the Spearman rank correlation coefficient of the fusion lesion segmentation array and the model parameter array, the correlation strength between the two is quantified. 5.The convolutional neural network-based image lesion segmentation system of claim 1, wherein: The steps of joint optimization debugging include: Based on the CT lesion feature array, the CT segmentation parameter array, the MRI lesion feature array, the MRI segmentation parameter array, and the association relationship of the fusion lesion segmentation array, the Nth round of parameter debugging is performed; The CT lesion feature array, the CT segmentation parameter array, the MRI lesion feature array, and the MRI segmentation parameter array after the Nth round of debugging are lesion inferences; Based on the Nth CT lesion inference array, the Nth MRI lesion inference array, and a preset error calculation function, an Nth segmentation error value is obtained; When the absolute difference between the Nth segmentation error value and the Mth segmentation error value is less than a preset threshold, the segmentation model parameters after the Nth round of debugging are determined as the final optimization result; otherwise, M=N and N=N+1, and return to iteration; Wherein, N is a natural number greater than 0, and the gradient descent algorithm is used to update the CT segmentation parameters or the MRI segmentation parameters in each round of debugging. 6.The convolutional neural network-based image lesion segmentation system according to claim 5, wherein, The steps of lesion inference include: Load the model architecture optimized in the Nth round of debugging, and select the inference form, including single modality inference and cross-modality inference; Based on the inference form, the CT lesion feature array, the CT segmentation parameter array, the MRI lesion feature array, and the MRI segmentation parameter array are used as model condition inputs, and the predicted lesion probability is output. 7.The convolutional neural network-based image lesion segmentation system according to claim 5, wherein, The algorithm model of the error calculation function is: Under the single modality form, the Dice coefficient error function is used; Under the cross-modality form, the weighted joint Dice loss function is used. 8.The convolutional neural network-based image lesion segmentation system of claim 1, wherein, The steps of priority labeling result include: Load the optimized segmentation model parameters to obtain the confidence weight of the CT lesion segmentation array and the MRI lesion segmentation array, and the confidence weight is based on at least one of the probability graph entropy or the cross-modality overlap area ratio; Based on the confidence weight, the score evaluation coefficient Priority is obtained by weighted summation. The score evaluation coefficient is compared with the standard score interval [DF min , DF max ] for analysis: When Priority < DF min Priority is marked as a, while editing as a first-class character, and a and the first-class character are combined to generate a low-priority lesion area; When DF min <Priority≤DF max The Priority is marked as b, while being edited as a secondary character, and b and the secondary character are combined to generate a medium priority lesion area; When Priority > DF max Priority is marked as c while editing as a tertiary character, and c and the tertiary character are combined to generate a high priority lesion region.

Images