Image segmentation method and system based on deep learning

Abstract

The application provides a kind of image segmentation method and system based on deep learning, first, obtain the medical image original data set containing different modal organ image data and the corresponding clinical anatomy annotation data containing organ contour and lesion region annotation information, then load the pre-trained medical image segmentation model, the medical image segmentation model includes anatomical structure prior information fusion module and clinical annotation rule mapping module, input medical image original data into model feature extraction layer, generate medical image feature atlas in combination with anatomical structure prior information, call clinical annotation rule mapping module to convert clinical anatomy annotation data into segmentation constraint parameters, obtain preliminary segmentation result set by processing feature atlas, finally, based on the association between preliminary segmentation result and clinical anatomy annotation data, optimize model segmentation parameters, output the final medical image segmentation result containing organ contour and lesion region segmentation result, thereby improve the accuracy of image segmentation.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and more specifically, to an image segmentation method and system based on deep learning. Background Technology

[0002] In the field of medical image analysis, image segmentation is a crucial technique that can accurately separate key structures such as organs and lesions from images, providing important information for subsequent diagnosis and treatment planning. However, existing medical image segmentation methods have many limitations.

[0003] On the one hand, traditional segmentation methods often rely solely on the pixel information of the medical images themselves, without fully considering prior knowledge of the anatomical structure of organs. Different organs have specific locations, shapes, and relationships within the human body, and this prior information is crucial for accurate organ segmentation. However, traditional methods ignore these key factors, leading to a significant decrease in the accuracy of segmentation results when faced with complex anatomical structures or poor image quality.

[0004] On the other hand, while existing segmentation models use clinical anatomical annotation data during training, they typically treat this data simply as a supervisory signal for model training, without deeply exploring the intrinsic relationship between clinical annotation rules and the segmentation process. This makes it difficult for the model to fully understand the clinician's annotation intent, and it fails to effectively constrain and optimize the segmentation process based on the annotation rules, thus affecting the reliability and practicality of the segmentation results. Summary of the Invention

[0005] In view of the aforementioned problems, and in conjunction with the first aspect of the present invention, embodiments of the present invention provide a deep learning-based image segmentation method, the method comprising:

[0006] Acquire a set of raw medical image data and corresponding clinical anatomical annotation data. The set of raw medical image data includes organ image data of different modalities, and the clinical anatomical annotation data includes organ contour annotation information and lesion area annotation information.

[0007] Load a pre-trained medical image segmentation model, which includes an anatomical structure prior information integration module and a clinical annotation rule mapping module;

[0008] The raw medical image data set is input into the feature extraction layer of the medical image segmentation model, and the organ anatomical features output by the anatomical structure prior information are combined with the anatomical structure integration module to generate a medical image feature map.

[0009] The clinical annotation rule mapping module is called to convert the clinical anatomical annotation data into segmentation constraint parameters, and the medical image feature atlas is subjected to segmentation constraint processing to obtain a preliminary segmentation result set.

[0010] Based on the association relationship between the preliminary segmentation result set and the clinical anatomical annotation data, segmentation parameters of the medical image segmentation model are optimized, and a final medical image segmentation result is output, wherein the final medical image segmentation result comprises an organ contour segmentation result and a lesion region segmentation result.

[0011] In another aspect, the embodiment of the present application further provides an image segmentation system based on deep learning, which is characterized by comprising:

[0012] A processor and a machine readable storage medium for storing machine executable instructions of the processor, wherein the processor is configured to execute the above-mentioned image segmentation method based on deep learning by executing the machine executable instructions.

[0013] In another aspect, the embodiment of the present application further provides a computer program product, which comprises machine executable instructions stored in a computer readable storage medium, and a processor of a computer device reads the machine executable instructions from the computer readable storage medium, and the processor executes the machine executable instructions, so that the computer device executes the above-mentioned image segmentation method based on deep learning.

[0014] Based on the above aspects, by acquiring a medical image original data set comprising different modal organ image data and corresponding clinical anatomical annotation data comprising organ contour and lesion region annotation information, and in the pre-trained medical image segmentation model, the anatomical structure prior information fusion module can fuse organ anatomical features into the feature extraction process, and generate a medical image feature atlas that is more consistent with the actual anatomical structure in combination with the medical image original data, effectively solving the problem of inaccurate segmentation caused by ignoring anatomical prior knowledge in traditional methods. The clinical annotation rule mapping module can convert the clinical anatomical annotation data into segmentation constraint parameters, and perform segmentation constraint processing on the feature atlas, so that the segmentation process is more consistent with the clinical annotation. Based on the association relationship between the preliminary segmentation result set and the clinical anatomical annotation data, the segmentation parameters are optimized, the segmentation performance of the model is further improved, and finally the medical image segmentation result comprising the organ contour and the lesion region segmentation result is output, which can provide more accurate and comprehensive image segmentation results for medical diagnosis and treatment. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 is an execution flow diagram of the image segmentation method based on deep learning provided by the embodiment of the present application.

[0016] Figure 2is a schematic diagram of exemplary hardware and software components of a deep learning-based image segmentation system provided by an embodiment of the present application. DETAILED DESCRIPTION

[0017] The present application will be described in detail below with reference to the accompanying drawings, Figure 1 is a flowchart of a deep learning-based image segmentation method provided by an embodiment of the present application, which will be described in detail below.

[0018] Step S110: Obtain a medical image original data set and corresponding clinical anatomical annotation data, wherein the medical image original data set contains organ image data of different modalities, and the clinical anatomical annotation data contains organ contour annotation information and lesion region annotation information.

[0019] This embodiment takes liver medical image segmentation as the only application scenario. The obtained medical image original data set covers three modalities of liver image data, including computed tomography image data, magnetic resonance imaging data, and ultrasound image data. The computed tomography image data is collected by spiral scanning, each image is composed of a pixel matrix, and the pixel value presents gradient changes according to the density difference of the liver and the surrounding tissues, which can distinguish the liver parenchyma, portal vein, hepatic vein, and surrounding fat tissue. The magnetic resonance imaging data includes T1 weighted imaging, T2 weighted imaging, and diffusion weighted imaging sequences, which respectively focus on displaying the anatomical structure of the liver, the edema area, and the cell diffusion characteristics. For example, in T1 weighted imaging, the liver parenchyma shows moderate signal, and the blood vessels show low signal; in T2 weighted imaging, the lesion area often shows high signal. The ultrasound image data includes two-dimensional gray-scale images and color Doppler images, the two-dimensional gray-scale images can show the shape of the liver, the integrity of the capsule, and the internal echo distribution, and the color Doppler image can reflect the blood flow direction and speed in the liver blood vessels.

[0020] For the privacy sensitive information in the above medical image original data, de-identification processing is adopted. Specifically, the patient's name, ID number, hospitalization number, examination date, and other personal identity information recorded in the image file header are removed, and the image data and the patient's identity information are stored separately in different encrypted databases. At the same time, the image data is encrypted using a symmetric encryption algorithm, and role-based access control permissions are set, only authorized personnel can access the data after identity verification, to prevent privacy leakage.

[0021] The corresponding clinical anatomical annotation data is completed by a qualified radiologist according to medical annotation specifications. The physician labels the liver contour on each modality image by manually outlining in a professional annotation software to form organ contour annotation information. The organ contour annotation information is stored in the form of a pixel coordinate set, including the row and column coordinates of all pixels on the contour. For lesion region annotation, the physician labels the lesion range of liver hemangioma, hepatocellular carcinoma and the like according to the morphological, signal or density characteristics of the lesions in the image. The same is stored in the form of a pixel coordinate set, and is associated with attribute information such as lesion type, boundary feature, and relative position to liver blood vessels.

[0022] Step S120: loading a pre-trained medical image segmentation model, the medical image segmentation model comprising an anatomical structure prior information fusion module and a clinical annotation rule mapping module.

[0023] The loaded pre-trained medical image segmentation model takes a U-shaped convolutional neural network as a basic network structure, which includes an encoder (feature extraction layer) and a decoder (segmentation output layer). The encoder is alternately composed of multiple convolutional blocks and down-sampling layers. Each convolutional block includes a convolutional layer, a batch normalization layer and an activation function layer, which are used to extract low-level edge, texture features and high-level organ structure features of the image. The down-sampling layer reduces the spatial size of the feature map through a max-pooling operation, and retains the key features. The decoder is alternately composed of multiple deconvolutional blocks and up-sampling layers. The feature map size is restored through up-sampling, and the corresponding level feature maps of the encoder are connected through jump connection to fuse the detail features and semantic features.

[0024] The model has built-in anatomical structure prior information fusion module and clinical annotation rule mapping module. The anatomical structure prior information fusion module is used to fuse the liver anatomical prior information into the feature extraction process, and the clinical annotation rule mapping module is used to convert the clinical annotation data into segmentation constraint parameters recognizable by the model. Pre-training uses a data set containing a large number of liver images and corresponding annotation data, and iteratively trains and optimizes the model parameters to make the model initially have liver and lesion segmentation capability.

[0025] Step S121: extracting organ anatomical structure common features corresponding to different modality organ image data in the medical image original data set, the organ anatomical structure common features including organ spatial position relationship features and organ tissue density distribution features.

[0026] For the three modal liver image data, the common features of organ anatomic structure are extracted. For the organ spatial position relationship features, an image registration algorithm based on mutual information is adopted, taking the computed tomography image as the reference, the magnetic resonance imaging and ultrasound image are registered with it, so that the liver position in the three modal images is aligned to the unified human anatomic coordinate system. On this basis, the relative position relationship between the liver and adjacent organs such as gallbladder, right kidney, diaphragm, duodenum, etc. is identified, the coordinate range of the liver in the anatomic coordinate system, the geometric center coordinate and the anatomic landmark point coordinate are recorded, and the organ spatial position relationship features are formed. Each dimension of the organ spatial position relationship features corresponds to a spatial parameter.

[0027] For the organ tissue density distribution features, different modal image characteristics are extracted. In the computed tomography image, the mean value and variance of the pixel values of the left lobe, right lobe and caudate lobe of the liver are counted, and the pixel value range of the blood vessel structure is identified; in the magnetic resonance imaging of each sequence, the mean value and ratio of the signal intensity of the liver parenchyma and the lesion are extracted; in the ultrasound image, the mean value and uniformity of the echo intensity of the liver parenchyma are analyzed. These parameters are integrated to form the organ tissue density distribution features.

[0028] Step S122: constructing an anatomic structure prior information library based on the organ anatomic structure common features, each prior information in the anatomic structure prior information library is associated with a corresponding organ modal identifier and an anatomic position identifier.

[0029] Step S1221: classifying the organ anatomic structure common features, classifying the common features belonging to the same organ type into one category to form an organ feature category set, each organ feature category is associated with a corresponding organ type name.

[0030] All the extracted organ anatomic structure common features are classified into the "liver" category to form a liver organ feature category set, and are associated with the organ type name "liver". The liver organ feature category set covers the spatial position relationship features and the tissue density distribution features of the liver.

[0031] Step S1222: for each organ feature category, the organ spatial position relationship features are extracted, which include the relative position features of the organ and adjacent organs, and the position features of the organ in the human anatomic coordinate system.

[0032] The organ spatial position relationship features are extracted from the liver organ feature category set. The relative position features with adjacent organs include the distance between the liver and the gallbladder, the overlapping range between the right lobe of the liver and the right kidney, the attachment position of the liver and the diaphragm, etc.; the position features in the anatomic coordinate system include the coordinate range of the liver in the sagittal plane, coronal plane and axial plane, and the coordinate parameters of the anatomic landmark points such as the porta hepatis and gallbladder fossa, which constitute the spatial position relationship feature vector.

[0033] Step S1223: Extract organ tissue density distribution features in each organ feature category, which include density mean value features and density variation trend features of different regions of the organ.

[0034] The organ tissue density distribution features are extracted from the liver organ feature category set. The density mean value features include the density mean values of each lobe of the liver in different modal images, such as the pixel value mean of the left lobe of the liver in the computer tomography image and the signal intensity mean of the right lobe of the liver in the magnetic resonance imaging T1 sequence. The density variation trend features include the density gradient variation of the liver from the capsule to the center and the density variation rule of the tissue around the blood vessels, etc. These parameters constitute the tissue density distribution feature vector.

[0035] Step S1224: Integrate the organ spatial position relationship features and the organ tissue density distribution features to form the anatomical structure prior information entry of the organ feature category.

[0036] After dimension matching, the liver spatial position relationship feature vector and the tissue density distribution feature vector are spliced to form a complete liver anatomical structure prior information entry, which includes the spatial attribute and density attribute information of the liver.

[0037] Step S1225: Assign an organ modality identifier to each anatomical structure prior information entry, which is determined according to the organ image data modality corresponding to the prior information, and different modalities correspond to different identifier symbols.

[0038] According to the image modality of the prior information entry source, the corresponding organ modality identifier is assigned. The entries based on the computer tomography image are assigned the identifier "CT", the entries based on the magnetic resonance imaging image are assigned the identifier "MRI", the entries based on the ultrasound image are assigned the identifier "US", and the entries based on the multi-modality fusion are assigned the identifier "MIX".

[0039] Step S1226: Assign an anatomical position identifier to each anatomical structure prior information entry, which is determined according to the anatomical position of the organ in the human body corresponding to the prior information, and different anatomical positions correspond to different identifier symbols.

[0040] According to the liver anatomical region division, an anatomical position identifier is assigned to each prior information entry. The left outer lobe of the liver corresponds to "L1", the left inner lobe of the liver corresponds to "L2", the right anterior lobe of the liver corresponds to "R1", the right posterior lobe of the liver corresponds to "R2", the caudate lobe corresponds to "C", and the entry reflecting the overall structure of the liver corresponds to "WHOLE".

[0041] Step S1227: Establishing a storage structure of the anatomical structure prior information library, the storage structure comprising a prior information entry field, an organ modality identification field, an anatomical position identification field, and a creation time field.

[0042] The anatomical structure prior information library is constructed using a relational database, and four core fields are designed. The prior information entry field stores the feature vector in binary form; the organ modality identification field and the anatomical position identification field store the corresponding identification in character type; and the creation time field stores the entry creation time in the form of a time stamp. At the same time, indexes are established on the modality identification and anatomical position identification fields to improve the retrieval efficiency.

[0043] Step S1228: Fill each anatomical structure prior information entry and its corresponding organ modality identification, anatomical position identification, and creation time information into the corresponding fields of the storage structure.

[0044] Each liver anatomical structure prior information entry and its modality identification, anatomical position identification, and creation time are filled into the database according to the field requirements of the storage structure. After completion, the integrity of the database is checked to ensure that all field information is complete and error-free.

[0045] Step S123: Building an anatomical structure prior information integration module, the anatomical structure prior information integration module comprising a prior information retrieval submodule and a feature fusion submodule, the prior information retrieval submodule being configured to match corresponding organ anatomical features from the anatomical structure prior information library according to the input organ image data modality identification and anatomical position identification, and the feature fusion submodule being configured to fuse the organ anatomical features with the basic features output by the feature extraction layer of the medical image segmentation model.

[0046] The anatomical structure prior information integration module is composed of a prior information retrieval submodule and a feature fusion submodule. The prior information retrieval submodule is connected to the anatomical structure prior information library through a database interface, and its retrieval logic unit parses the meta information of the input image to obtain the modality identification, and preliminarily locates the liver anatomical position through the feature extraction layer to obtain the anatomical position identification; the feature matching unit retrieves the corresponding organ anatomical feature vector from the library by combining precise matching with fuzzy matching according to the two identifications.

[0047] The feature fusion submodule comprises a feature dimension adjustment unit, a numerical standardization unit, and a cross-channel fusion unit. The feature dimension adjustment unit adjusts the dimension of the organ anatomical feature vector to be consistent with the dimension of the basic feature map; the numerical standardization unit adjusts the numerical range of the anatomical features to be consistent with the basic features using the Min-Max method; and the cross-channel fusion unit calculates the correlation of the anatomical features and the basic features in each channel through an attention mechanism, embeds the anatomical features in the channel with the highest correlation according to the weight, and realizes the fusion of the two.

[0048] Step S124: Collecting annotation specification data for different organ image segmentation in clinical diagnosis and treatment process, the annotation specification data including organ contour annotation accuracy requirement, lesion region annotation boundary range requirement.

[0049] By consulting authoritative medical literature and collecting clinical annotation specifications from multiple medical institutions, annotation specification data for liver image segmentation is obtained. The organ contour annotation accuracy requirement is clear, the pixel deviation of the contour annotation from the actual liver boundary needs to be controlled within the specified range, and the contour needs to be continuous and smooth, completely surrounding the liver without missing the caudate lobe and other regions.

[0050] The lesion region annotation boundary range requirement is formulated for different lesion types. Liver hemangioma annotation needs to include the entire enhanced region and the central non-enhanced part; hepatocellular carcinoma annotation needs to cover the tumor entity, the surrounding edema zone and satellite lesions; liver metastasis tumor annotation needs to surround each independent lesion, and fusion lesions need to be annotated with the overall range. In addition, the specification requires that all identifiable lesions need to be annotated to ensure completeness.

[0051] Step S125: Constructing a clinical annotation rule library based on the annotation specification data, each rule in the clinical annotation rule library being associated with a corresponding organ type identifier and lesion type identifier.

[0052] The annotation specification data is converted into structured clinical annotation rules to construct a clinical annotation rule library. Each rule includes rule content, organ type identifier and lesion type identifier. For example, the rule "liver contour annotation deviation does not exceed the specified range, and the contour is continuous and complete", the organ type identifier is "liver", and there is no lesion type identifier; the rule "liver hemangioma annotation includes enhanced region and central non-enhanced part", the organ type identifier is "liver", and the lesion type identifier is "liver hemangioma".

[0053] The clinical annotation rule library is stored in a document type database, and each rule is an independent document. Indexes are established in the organ type identifier and lesion type identifier fields to support fast retrieval. A rule updating mechanism is set up to adjust the rules regularly according to the latest clinical specifications to ensure timeliness.

[0054] Step S126: Building a clinical annotation rule mapping module, the clinical annotation rule mapping module including a rule matching submodule and a parameter conversion submodule, the rule matching submodule being used to match corresponding annotation rules from the clinical annotation rule library according to the organ type identifier and lesion type identifier of the input clinical anatomical annotation data, and the parameter conversion submodule being used to convert the annotation rules into segmentation constraint parameters recognizable by the medical image segmentation model.

[0055] The clinical annotation rule mapping module is composed of a rule matching submodule and a parameter conversion submodule. The rule matching submodule connects the clinical annotation rule library through a database interface, parses the organ type identifier and the lesion type identifier of the input clinical anatomical annotation data, and then filters the rules according to the organ type identifier and matches the corresponding rules according to the lesion type identifier. If only organ contour annotation is contained, only organ contour annotation rules are matched.

[0056] The rule matching submodule sets a priority mechanism to preferentially match rules consistent with the input image modality, selects general rules when there are no modality-specific rules, and outputs the rule set after matching. The rule analysis unit of the parameter conversion submodule extracts key constraint conditions in the rules, such as "contour deviation threshold" and "lesion range requirement"; the parameter mapping unit converts the constraint conditions into model parameters, such as contour feature response threshold and lesion feature intensity range, according to a predefined mapping relationship table; the parameter verification unit checks the parameter values and logical consistency, and outputs the segmentation constraint parameters after verification, or reanalyzes and converts if the verification fails.

[0057] Step S127: The anatomical structure prior information integration module, the clinical annotation rule mapping module, and the basic network structure of the medical image segmentation model are integrated. The basic network structure includes a feature extraction layer and a segmentation output layer, the output end of the anatomical structure prior information integration module is connected to the output end of the feature extraction layer, and the output end of the clinical annotation rule mapping module is connected to the input end of the segmentation output layer.

[0058] The three modules are integrated in a modular integration manner. The output ends of the convolution blocks at each level of the feature extraction layer are connected to the output end of the anatomical structure prior information integration module through a feature channel interface, and the basic feature atlas output by the convolution blocks is transmitted to the integration module for fusion and then returned to the next level of convolution block.

[0059] The output end of the clinical annotation rule mapping module is connected to the input end of the segmentation output layer through a parameter interface, the segmentation constraint parameters are transmitted to the parameter receiving buffer of the segmentation output layer, and then distributed to the convolution blocks at each level as needed. A data interaction protocol between modules is set to clearly indicate that the feature atlas adopts a multi-channel tensor format and the parameters adopt a key-value pair format for transmission, and a interface adaptation layer is used to solve the data format difference problem, ensuring normal data interaction between modules.

[0060] Step S128: The integrated medical image segmentation model is pre-trained using a training data set containing medical image data and corresponding clinical anatomical annotation data, and the model weight parameters are iteratively adjusted to make the deviation of the segmentation result output by the medical image segmentation model from the clinical anatomical annotation data meet the preset range, thereby obtaining a pre-trained medical image segmentation model.

[0061] The training data set is constructed, containing a large number of de-identified three modal liver images and corresponding annotation data, and is divided into training set, validation set and test set in proportion. The pre-training adopts gradient descent optimization algorithm, and the cross-entropy loss function is used as the loss index.

[0062] The training set image is input into the model, the feature extraction layer extracts the basic features, the anatomical structure prior information fusion module matches the anatomical features and fuses them, and the segmentation output layer generates the preliminary segmentation result under the guidance of the constraint parameter. The preliminary result is compared with the annotation data, the loss value is calculated, and the parameters such as the convolution kernel weight of the feature extraction layer, the correlation weight of the fusion module, and the deconvolution kernel weight of the segmentation output layer are adjusted by gradient descent in reverse.

[0063] The performance of the model is evaluated with the validation set after each iteration, and if the validation set loss value does not decrease continuously, the learning rate is adjusted or the network structure is fine-tuned. Repeat the training until the deviation of the test set segmentation result and the annotation data meets the preset range, save the model parameters and structure, and complete the pre-training.

[0064] Step S130: input the medical image original data set into the feature extraction layer of the medical image segmentation model, combine the organ anatomical features output by the anatomical structure prior information fusion module, and generate a medical image feature atlas.

[0065] The liver medical image original data is grouped according to the modality, and each group of images is input into the model feature extraction layer in turn. The convolution block of the feature extraction layer performs convolution, batch normalization and activation operation on the image, extracts low-level features, and transmits them to the next level of convolution block after processing by the down-sampling layer, extracts high-level features layer by layer, and generates a basic feature atlas.

[0066] The anatomical structure prior information fusion module analyzes the image modality identifier and the anatomical position identifier, matches the organ anatomical features from the prior information library, adjusts the dimensions and standardizes them, embeds them into the corresponding channel of the basic feature atlas through the attention mechanism, and forms a medical image feature atlas containing detailed and semantic features after the processing of the up-sampling.

[0067] Step S131: classify the different modal organ image data in the medical image original data set according to the modality identifier, and classify the different modal image data of the same organ into the same data group, and each data group is associated with the corresponding organ type identifier.

[0068] Traverse the medical image original data, parse the image file header to obtain the modality identifier, and form a temporary group according to the "CT", "MRI" and "US" classification. Extract the organ features of the images in the temporary group, add the "liver" type identifier after determining the liver, and then classify the different modal liver images of the same patient into the same data group according to the patient identifier, and each data group is assigned a unique identifier and associated with the "liver" type identifier.

[0069] Step S132: sequentially input the organ image data in each data group into the feature extraction layer of the medical image segmentation model, extract the basic features of each modality image data through multi-layer convolution operation, and generate the basic feature atlas corresponding to each modality.

[0070] Each data group inputs the image into the feature extraction layer in a preset order. Taking the computed tomography image as an example, the first convolution block generates the first level basic feature atlas through convolution, batch normalization and activation operation, and transmits to the second convolution block after down sampling, and repeatedly generates the second level basic feature atlas, and generates the multi-level basic feature atlas through layer-by-layer processing. The magnetic resonance imaging and ultrasound image adopt the same process to generate their own basic feature atlas, and the feature dimensions and types of different modality atlases remain consistent.

[0071] Step S133: For each data group, extract its associated organ type identifier, input the organ type identifier into the prior information retrieval submodule of the anatomical structure prior information integration module, and according to the organ type identifier, match the organ anatomical features corresponding to the organ type identifier from the anatomical structure prior information library, the organ anatomical features include the spatial distance features of the organ and the surrounding tissues and the density gradient features of the internal tissues of the organ.

[0072] The "liver" type identifier of each data group is extracted and input into the prior information retrieval submodule. The prior information retrieval submodule first filters out the prior information entries of the liver type, and then further matches the image modality identifier and the anatomical position identifier to extract the organ anatomical features, including the spatial distance features of the liver and the surrounding tissues and the density gradient features of the internal tissues. If there is no completely matched entry, the most similar entry is selected and the features are fine-tuned to fit the current image anatomical structure.

[0073] Step S134: input the basic feature atlas and the organ anatomical features into the feature fusion submodule of the anatomical structure prior information integration module, and adopt the information interaction mode across the feature channels to embed the organ anatomical features into the corresponding feature channels of the basic feature atlas.

[0074] Step S1341: analyze the feature channel structure of the basic feature atlas, determine the corresponding feature type of each feature channel, and the feature type includes spatial position feature type, density distribution feature type and edge contour feature type.

[0075] In this embodiment, after receiving the liver image basic feature map output by the feature extraction layer, the feature fusion sub-module first starts the channel analysis process. The basic feature map is a multi-channel tensor structure, and each channel corresponds to a specific type of feature. The channel analysis process determines the feature type of each channel by traversing the feature value distribution of each feature channel and combining the convolution kernel design logic of the feature extraction layer.

[0076] For example, for the basic feature map of the computer tomography liver image, the feature value of a certain channel presents obvious gradient change in the junction area of the liver and the gallbladder, right kidney and other surrounding organs, and this channel is determined to be a spatial position feature type; the feature value of a certain channel presents regular fluctuation with the change of liver tissue density, forming a significant distinction between liver parenchyma and blood vessel area, and this channel is determined to be a density distribution feature type; the feature value of a certain channel forms a high-intensity response in the edge area of the liver capsule, which can clearly outline the contour boundary of the liver, and this channel is determined to be an edge contour feature type. After analysis, a corresponding list of feature channel number and feature type is generated.

[0077] Step S1342: Analyze the organ anatomical features, determine the feature types contained therein, and match the organ anatomical feature subsets consistent with the feature types in the basic feature map.

[0078] The liver anatomical features extracted from the anatomical structure prior information library are structurally analyzed. The liver anatomical features contain multiple feature parameters, and the corresponding feature types are determined by analyzing the semantic attributes of each parameter. Among them, the spatial distance feature parameters of the liver and the surrounding tissues (such as the distance between the liver and the gallbladder, the distance between the liver and the diaphragm, etc.) correspond to the spatial position feature type; the density gradient feature parameters of the internal tissues of the liver (such as the density change rate of the left lobe capsule to the center, the density gradient of the tissues around the portal vein, etc.) correspond to the density distribution feature type.

[0079] According to the above analysis results, the liver anatomical features are divided into spatial position feature subsets and density distribution feature subsets. Referring to the feature type list of the basic feature map, the above two feature subsets consistent with the feature types in the basic feature map are screened out, and the feature parameters that do not match the feature types of the basic feature map are excluded, to ensure that the subsequent embedded feature data is consistent with the semantic attributes of the basic feature channel.

[0080] Step S1343: For each feature type, determine the channel number of the corresponding feature channel in the basic feature map, and establish a mapping relationship between the feature type and the channel number.

[0081] According to the feature channel number and feature type corresponding list generated in step S1341, the channel numbers corresponding to the spatial position feature type and the density distribution feature type are searched respectively. It is assumed that in the basic feature atlas, the feature types of channel 1, channel 2 and channel 3 are spatial position feature types, the feature types of channel 4, channel 5 and channel 6 are density distribution feature types, and the feature types of channel 7 and channel 8 are edge contour feature types.

[0082] For the spatial position feature type, the corresponding channel 1, channel 2 and channel 3 numbers are recorded; for the density distribution feature type, the corresponding channel 4, channel 5 and channel 6 numbers are recorded. Based on this, a mapping relationship table of feature type and channel number is established, which clearly shows that the spatial position feature type is mapped to channel 1-3 and the density distribution feature type is mapped to channel 4-6. This mapping relationship table will serve as the basis for feature data allocation.

[0083] Step S1344: According to the mapping relationship, the feature data in the organ dissection feature subset is allocated to the corresponding feature channel, and each feature channel receives organ dissection feature data matching its feature type.

[0084] The feature fusion sub-module allocates the organ dissection feature subset according to the mapping relationship table of feature type and channel number. The feature parameters such as the distance between the liver and the gallbladder, the distance between the liver and the diaphragm, etc. in the spatial position feature subset are allocated to channel 1, channel 2 and channel 3 respectively according to the parameter dimension. For example, the liver and gallbladder distance feature parameter is allocated to channel 1, the liver and diaphragm distance feature parameter is allocated to channel 2, and the liver and right kidney distance feature parameter is allocated to channel 3.

[0085] The feature parameters such as the left lobe of liver density gradient, the right lobe of liver density gradient, and the density gradient around the portal vein, etc. in the density distribution feature subset are allocated to channel 4, channel 5 and channel 6 respectively. For example, the left lobe of liver density gradient feature parameter is allocated to channel 4, the right lobe of liver density gradient feature parameter is allocated to channel 5, and the density gradient around the portal vein feature parameter is allocated to channel 6. Each feature channel only receives organ dissection feature data matching its feature type, avoiding feature type confusion.

[0086] Step S1345: The organ dissection feature data allocated to the feature channel is subjected to feature dimension adjustment and numerical standardization processing, so that the dimension and numerical range of the organ dissection feature data are consistent with the dimension and numerical range of the basic feature data in the corresponding feature channel.

[0087] For each organ anatomic feature data allocated in each feature channel, first, the feature dimension is adjusted. Analyzing the dimension information of the basic feature data in the corresponding feature channel (such as the length of the feature vector, the number of rows and columns of the feature matrix), through interpolation or dimension reduction processing, the dimension of the organ anatomic feature data is adjusted to be completely consistent with the dimension of the basic feature data. For example, if the basic feature data in channel 1 is a two-dimensional matrix and the organ anatomic feature data is a one-dimensional vector, the one-dimensional vector is converted into a two-dimensional matrix through matrix expansion to ensure dimension matching.

[0088] Subsequently, numerical standardization processing is performed. The maximum and minimum values of the basic feature data in the corresponding feature channel are calculated, and the numerical range of the organ anatomic feature data is adjusted to be the same as that of the basic feature data by using linear scaling method. For example, if the numerical range of the basic feature data in channel 1 is 0-1 and the numerical range of the organ anatomic feature data is 10-20, the numerical range of the organ anatomic feature data is scaled to the interval of 0-1 through linear conversion, eliminating the influence of the difference in numerical range on the fusion effect.

[0089] Step S1346: using the cross-feature channel information interaction algorithm, the adjusted and standardized organ anatomic feature data is fused with the basic feature data in the corresponding feature channel pixel by pixel, and the image detail information in the basic feature data and the anatomic structure information in the organ anatomic feature data are retained in the fusion process.

[0090] The cross-feature channel information interaction algorithm is started, which includes two stages of feature attention calculation and pixel-by-pixel fusion. In the feature attention calculation stage, for each feature channel, the correlation coefficient of the organ anatomic feature data and the basic feature data is calculated, and the attention weight matrix is generated according to the correlation coefficient. The higher the weight value, the stronger the importance of the organ anatomic feature at that position to the segmentation task.

[0091] In the pixel-by-pixel fusion stage, the organ anatomic feature data and the basic feature data are weighted and spliced pixel by pixel according to the attention weight matrix. For each pixel position, the pixel value of the basic feature data and the pixel value of the organ anatomic feature data are combined according to the attention weight to form the fused pixel value. In the fusion process, by setting a weight threshold, it is ensured that the pixel values reflecting the image detail information such as the edge details and texture changes of the liver in the basic feature data are retained, and at the same time, the pixel values reflecting the anatomic structure information such as the spatial position and density distribution of the liver in the organ anatomic feature data are effectively fused.

[0092] Step S1347: Perform intra-channel information enhancement processing on the fused feature channel data, strengthen the feature signals related to organ contours and lesion regions, and repeat the above processing process until the organ dissection feature data corresponding to all feature types are embedded into the corresponding feature channels of the basic feature atlas, generating a feature atlas embedded with organ dissection features.

[0093] Perform information enhancement processing on the fused feature data in each feature channel. A local contrast enhancement method is used to improve the contrast between the feature data related to the liver contour and potential lesion regions (such as pixel value mutation regions, high intensity feature value regions), and to strengthen the recognition of the feature signals. For example, in channel 7 (edge contour feature type), for the feature signal region corresponding to the liver capsule edge, the edge contour clarity is enhanced by increasing the difference between the pixel values of this region and the adjacent region.

[0094] After completing the embedding of organ dissection feature data of the current feature type, check if there are unprocessed feature types. If there are, repeat the processing process of steps S1341 to S1347 until the organ dissection feature data corresponding to all feature types such as spatial position feature type and density distribution feature type are embedded. Finally, all feature data in the feature channels are fused and enhanced to generate a feature atlas embedded with organ dissection features.

[0095] Step S135: Perform channel dimension information integration processing on the feature atlas embedded with organ dissection features to generate a single-modal fusion feature atlas.

[0096] The feature atlas embedded with organ dissection features is input into a channel integration unit, which uses a feature channel attention mechanism for information integration. First, the channel importance weight of each feature channel is calculated by analyzing the correlation between the feature data of each channel and the liver segmentation task, and assigning higher weights to channels that have a greater impact on the segmentation result (such as edge contour feature type channel and density distribution feature type channel), and lower weights to channels that have a smaller impact.

[0097] According to the channel importance weight, the feature data of all feature channels is weighted and spliced. The feature matrices of each channel are arranged and combined in order according to the weight, forming a comprehensive feature tensor. Then, the feature tensor is compressed and reconstructed in channel dimension through 1x1 convolution operation, reducing feature redundancy while preserving key feature information. Finally, the generated single-modal fusion feature atlas integrates the basic features and organ dissection prior features of the modal image, and has more rich semantic information.

[0098] Step S136: If the data set contains organ image data of multiple modalities, the single-modality fusion feature maps corresponding to each modality are subjected to inter-modality feature complementation processing, effective information related to organ contours and lesion regions is extracted from the features of each modality, the feature maps after inter-modality complementation processing are subjected to spatial dimension feature alignment processing, the spatially aligned feature maps are subjected to multi-layer feature aggregation processing, and a medical image feature map containing organ overall structure features and local lesion detail features is generated.

[0099] When the data set contains liver image data of three modalities of computed tomography, magnetic resonance imaging, and ultrasound, inter-modality feature complementation processing is first performed. Analyze the feature advantages of each single-modality fusion feature map: the feature map of the computed tomography modality has advantages in liver density distribution and blood vessel structure display; the feature map of the magnetic resonance imaging modality has advantages in lesion tissue signal differentiation and soft tissue resolution; and the feature map of the ultrasound modality has advantages in liver surface morphology and real-time dynamic structure.

[0100] For the organ contour extraction task, the liver capsule edge features are extracted from the computed tomography modality feature map, and the liver surface morphology features are extracted from the ultrasound modality feature map, and the two types of features are combined; for the lesion region extraction task, the lesion signal features are extracted from the magnetic resonance imaging modality feature map, and the lesion density features are extracted from the computed tomography modality feature map, and the two types of features are combined, realizing feature complementation between different modalities.

[0101] Subsequently, spatial dimension feature alignment processing is performed. Taking the computed tomography modality feature map as the reference, the complementary feature maps of the magnetic resonance imaging and ultrasound modalities are spatially aligned with the computed tomography modality feature map through image registration algorithms. The coordinate differences of key anatomical landmarks of the liver (such as the porta hepatis and gallbladder fossa) in the feature maps of the three modalities are calculated, and the spatial positions of the feature maps of the magnetic resonance imaging and ultrasound modalities are adjusted through affine transformation, so that the feature maps of the three modalities completely coincide in space.

[0102] Finally, multi-layer feature aggregation processing is performed. The spatially aligned feature maps of the three modalities are spliced according to the feature levels, with low-level features (such as edges and textures) at the bottom layer of the feature tensor and high-level features (such as organ structure and lesion semantics) at the high layer of the feature tensor. The spliced feature tensor is aggregated through multi-layer convolution operations to strengthen the association between different level features, and finally a medical image feature map containing liver overall structure features (such as liver morphology and blood vessel distribution) and local lesion detail features (such as lesion boundary and internal structure) is generated.

[0103] Step S140: call the clinical annotation rule mapping module, convert the clinical anatomical annotation data into segmentation constraint parameters, and perform segmentation constraint processing on the medical image feature atlas to obtain a preliminary segmentation result set.

[0104] The clinical annotation rule mapping module first parses the liver contour annotation information and lesion region annotation information (such as liver hemangioma annotation and hepatocellular carcinoma annotation) in the clinical anatomical annotation data, then matches the corresponding clinical annotation rules and converts them into segmentation constraint parameters. The segmentation constraint parameters and the medical image feature atlas are input into the segmentation output layer of the medical image segmentation model, and the feature atlas is classified at the pixel level under the guidance of the constraint parameters to distinguish organ regions, lesion regions and background regions, and finally generate a preliminary segmentation result set.

[0105] Step S141: parse the clinical anatomical annotation data, extract the organ contour annotation information and lesion region annotation information therein, determine the organ type identifier corresponding to the organ contour annotation information, and determine the lesion type identifier corresponding to the lesion region annotation information.

[0106] The clinical anatomical annotation data is parsed and structured, and the clinical anatomical annotation data is an XML format file containing two parts of annotation information header and annotation content. The annotation information header is parsed to obtain metadata such as image ID and annotation time corresponding to the annotation; the annotation content is parsed to extract organ contour annotation information and lesion region annotation information.

[0107] The organ contour annotation information is stored in the form of a pixel coordinate list, which contains the row coordinate and column coordinate of each pixel on the liver contour. According to this information, the corresponding organ type identifier is determined as "liver". The lesion region annotation information is also stored in the form of a pixel coordinate list, and each piece of lesion annotation information contains a lesion type field. According to this field, the lesion type identifier is determined, such as "liver hemangioma", "hepatocellular carcinoma", "liver metastasis", etc. For example, if the lesion type field of a piece of lesion annotation information is "liver hemangioma", the corresponding lesion type identifier is "liver hemangioma". After parsing, an association list of organ type identifiers and lesion type identifiers is generated to clearly identify the identifier information corresponding to each image.

[0108] Step S142: input the organ type identifier and the lesion type identifier into the rule matching sub-module of the clinical annotation rule mapping module, match the corresponding organ contour annotation rule from the clinical annotation rule library according to the organ type identifier, and match the corresponding lesion region annotation rule from the clinical annotation rule library according to the lesion type identifier.

[0109] The organ type identifier "liver" and the lesion type identifier (such as "liver hemangioma") are input into the rule matching submodule. The rule matching submodule first screens all the annotation rules of the organ type "liver" from the clinical annotation rule library according to the organ type identifier "liver", and extracts the organ contour annotation rule from the screened annotation rules, which contains the continuity requirement of liver contour annotation, the allowed range of contour deviation, etc.

[0110] Then, according to the lesion type identifier "liver hemangioma", the annotation rule of the lesion type "liver hemangioma" is further matched from the screened liver type annotation rules, which contains the boundary range requirement of liver hemangioma annotation (such as needing to include the lesion enhanced area and the central non-enhanced area), the differentiation standard of the lesion and the surrounding tissue, etc. If there are multiple matched rules, the latest rule consistent with the current image modality is selected as the final matching rule according to the effective time and applicable modality of the rule.

[0111] Step S143: The organ contour annotation rule and the lesion region annotation rule are input into the parameter conversion submodule of the clinical annotation rule mapping module. The organ contour annotation rule is converted into contour constraint parameters, which include the feature response threshold of the organ contour edge and the contour continuity requirement parameter. The lesion region annotation rule is converted into lesion constraint parameters, which include the feature intensity range parameter of the lesion region and the boundary differentiation parameter of the lesion and the surrounding tissue.

[0112] The matched organ contour annotation rule and the lesion region annotation rule are input into the parameter conversion submodule. For the organ contour annotation rule, the "contour deviation allowed range" clause is analyzed and converted into the feature response threshold of the contour edge, which is used to determine which pixel points in the feature map belong to the liver contour edge. The "contour continuity requirement" clause is analyzed and converted into the contour continuity requirement parameter, which specifies the maximum allowed distance between contour pixel points to ensure that the generated contour line is continuous and has no breakpoints.

[0113] For the lesion region annotation rule, the "lesion boundary range requirement" clause is analyzed and converted into the feature intensity range parameter of the lesion region, which defines the feature value interval of the lesion region in the feature map. The "differentiation standard of the lesion and the surrounding tissue" clause is analyzed and converted into the boundary differentiation parameter of the lesion and the surrounding tissue, which is used to quantify the feature difference threshold between the lesion region and the surrounding normal liver tissue. During the parameter conversion process, the natural language description of the rule clause is converted into numerical or Boolean parameters recognizable by the medical image segmentation model according to the parameter format requirements of the medical image segmentation model.

[0114] Step S144: integrate the contour constraint parameters and the lesion constraint parameters to generate a segmentation constraint parameter set, each parameter in the segmentation constraint parameter set being associated with a corresponding feature channel identifier and a spatial position identifier.

[0115] To integrate the contour constraint parameters and the lesion constraint parameters, first, assign each parameter a corresponding feature channel identifier. According to the feature type of the parameter, associate the contour constraint parameter to the channel of the edge contour feature type (such as channel 7, channel 8), and associate the lesion constraint parameter to the channel of the density distribution feature type and the signal feature type (such as channel 4, channel 5, channel 9).

[0116] Subsequently, assign each parameter a spatial position identifier, which is determined based on the liver anatomical partition, such as "left lobe of liver", "right anterior lobe of liver", "right posterior lobe of liver", etc., to clearly indicate the liver spatial area to which the parameter applies. For example, the spatial position identifier of the contour constraint parameter for the left lobe of liver is "left lobe of liver", and the spatial position identifier of the lesion constraint parameter for the hepatic hemangioma in the right anterior lobe of liver is "right anterior lobe of liver". Integrate all parameters and their associated feature channel identifiers and spatial position identifiers into a segmentation constraint parameter set in JSON format, which facilitates model parsing and calling.

[0117] Step S145: input the medical image feature atlas and the segmentation constraint parameter set into the segmentation output layer of the medical image segmentation model, and locate the corresponding feature channel in the medical image feature atlas according to the feature channel identifier in the segmentation constraint parameter set.

[0118] Input the medical image feature atlas and the segmentation constraint parameter set into the segmentation output layer (decoder part) together. The segmentation output layer first parses the feature channel identifier in the segmentation constraint parameter set, and locates the corresponding feature channel in the medical image feature atlas according to the identifier information. For example, if the feature channel identifier of a parameter is "channel 7", locate the feature channel numbered 7 in the medical image feature atlas; if the feature channel identifier of a parameter is "channel 4-6", locate the three feature channels numbered 4, 5, and 6. After positioning, establish the mapping relationship between the parameter and the corresponding feature channel to ensure that each parameter only acts on its associated feature channel.

[0119] Step S146: for the located feature channel, according to the spatial position identifier and the specific constraint parameter in the segmentation constraint parameter set, perform constraint filtering processing on the feature values in the feature channel, perform pixel-level classification processing on the filtered feature atlas, and classify each pixel into an organ region pixel, a lesion region pixel, or a background region pixel.

[0120] For each feature channel after positioning, first determine the parameter action range according to the spatial position identifier. Analyze the pixel coordinate range of the liver anatomical partition corresponding to the spatial position identifier in the feature channel, and limit the action range of the constraint parameter within the coordinate range. For example, if the spatial position identifier is "left lobe of liver", determine the pixel coordinate range of the left lobe of liver in the feature channel, and only the feature values in the range are constrained and screened.

[0121] According to the specific constraint parameter, the feature value is screened. For the feature response threshold in the contour constraint parameter, the pixel points with feature values greater than the threshold in the feature channel are screened, and these pixel points are preliminarily determined as liver contour candidate points; for the feature intensity range parameter in the lesion constraint parameter, the pixel points with feature values in the range are screened, and these pixel points are preliminarily determined as lesion candidate points.

[0122] The pixel-level classification is performed on the screened feature map. The screening results of multiple feature channels are integrated, and a voting mechanism is used to classify each pixel point: if a certain pixel point is determined as a contour candidate point in multiple edge contour feature channels, it is classified as an organ region pixel; if a certain pixel point is determined as a lesion candidate point in multiple lesion feature channels, it is classified as a lesion region pixel; if a certain pixel point is not determined as a candidate point by any feature channel and is located outside the organ region pixel and the lesion region pixel, it is classified as a background region pixel. For the pixel points with classification conflicts (such as being determined as organ region candidate points and lesion region candidate points at the same time), the priority of the constraint parameter is determined, and the priority of the lesion constraint parameter is higher than that of the organ contour constraint parameter, so the pixel point is classified as a lesion region pixel.

[0123] Step S147: generating a preliminary segmentation result according to the classification result, the preliminary segmentation result containing pixel distribution information of the organ contour and pixel distribution information of the lesion region.

[0124] Step S1471: extracting all pixel coordinates classified as organ region pixels from the classification result, arranging the pixel coordinates in row and column order to form an organ region pixel coordinate list.

[0125] Traverse the pixel-level classification result, filter out all pixel points with a classification label of "organ region pixel", and record the row coordinates and column coordinates of each pixel point. Sort the above pixel coordinates in ascending order of row coordinates and ascending order of column coordinates within the same row to form an ordered organ region pixel coordinate list. The organ region pixel coordinate list is stored in text format, and each record contains the row coordinates and column coordinates of a pixel point, which facilitates further analysis of the organ region.

[0126] Step S1472: performing edge detection processing on the pixel coordinates in the organ region pixel coordinate list, identifying the boundary pixel coordinates of the organ region, and the boundary pixel coordinates constitute the pixel distribution basis of the organ contour.

[0127] The organ region pixel coordinate list is input into an edge detection unit, and a contour tracking algorithm is used to identify the boundary pixels. The contour tracking algorithm selects a starting pixel point (usually the pixel point with the smallest row coordinate and the smallest column coordinate) from the organ region pixel coordinate list, and then sequentially searches for adjacent organ region pixel points in a predetermined neighborhood search order (such as clockwise). When there are no more untracked organ region pixel points in the neighborhood of the searched pixel point, and the starting pixel point is returned, the tracking of a contour is completed.

[0128] During the tracking process, all pixel coordinates located on the contour edge are recorded, and these pixel coordinates are the boundary pixel coordinates of the organ region. If there are holes in the organ region (such as areas formed by blood vessels passing through), the boundary pixel coordinates of the holes are also identified by the contour tracking algorithm to form an inner boundary pixel coordinate set. All outer and inner boundary pixel coordinates together constitute the pixel distribution basis of the organ contour.

[0129] Step S1473: connecting the boundary pixel coordinates in spatial position order to form continuous organ contour lines, recording the pixel coordinate sequence of each contour line, and constituting the pixel distribution information of the organ contour.

[0130] The identified organ region boundary pixel coordinates are connected in spatial position order. For the outer boundary pixel coordinates, the pixel points are connected in clockwise order to form a closed organ outer contour line; for the inner boundary pixel coordinates, the pixel points are connected in counterclockwise order to form a closed organ inner contour line. During the connection process, the distance between adjacent pixel points is ensured to meet the pixel-level continuity requirement to avoid breakpoints or intersections.

[0131] The pixel coordinate sequence of each contour line is recorded, and the coordinate sequences of the outer and inner contour lines are stored separately. For example, the coordinate sequence of the outer contour line records the row and column coordinates of each boundary pixel in connection order, and the coordinate sequence of the inner contour line is also recorded in connection order. These coordinate sequences together constitute the pixel distribution information of the organ contour, which can completely reflect the contour shape of the liver.

[0132] Step S1474: extracting all pixel coordinates classified as lesion region pixels from the classification result, arranging the pixel coordinates in row and column order to form a lesion region pixel coordinate list.

[0133] Again traverse the pixel-level classification result, filter out all the pixels with classification label "lesion region pixel", and record the row coordinate and column coordinate of each pixel. According to the same sorting rule as the organ region pixel coordinate list, that is, the row coordinate from small to large, and the column coordinate in the same row from small to large, sort the above pixel coordinates, and form a lesion region pixel coordinate list.

[0134] Step S1475: Perform region division processing on the pixel coordinates in the lesion region pixel coordinate list. If multiple pixel coordinates are adjacent in space and belong to the same lesion type, the pixel coordinates are classified into the same lesion sub-region.

[0135] The lesion region pixel coordinate list is input into the region division unit and processed using the connected region labeling algorithm. The connected region labeling algorithm first assigns an initial label to each pixel point in the lesion region pixel coordinate list, and then checks whether other pixel points in the four-neighborhood (up, down, left, right) or eight-neighborhood of each pixel point also belong to the lesion region pixel. If two pixel points are adjacent in space (i.e., there is a neighborhood) and have the same lesion type identification (e.g., both are "liver hemangioma"), their labels are unified to the same value, indicating that they belong to the same lesion sub-region.

[0136] The neighborhood checking and label unification process is repeated until all lesion region pixels are labeled. Each set of pixel points with the same label constitutes an independent lesion sub-region, and different labels correspond to different lesion sub-regions, thereby achieving the differentiation of multiple lesions.

[0137] Step S1476: Record the pixel coordinate range of each lesion sub-region, including the minimum row coordinate, maximum row coordinate, minimum column coordinate, and maximum column coordinate of the lesion sub-region, to form the pixel distribution basis of the lesion region.

[0138] For each lesion sub-region divided by the connected region labeling algorithm, traverse the coordinates of all pixel points in the sub-region to determine and record their pixel coordinate range. Specifically, find the minimum row coordinate and maximum row coordinate, as well as the minimum column coordinate and maximum column coordinate of the pixel points in the sub-region. These four coordinate values together define the rectangular boundary range of the lesion sub-region, even if the actual morphology of the lesion sub-region is not rectangular, the boundary range can roughly reflect the spatial position and size of the lesion. The coordinate range information of all lesion sub-regions constitutes the pixel distribution basis of the lesion region.

[0139] Step S1477: Perform density statistics on the pixel coordinates of each lesion sub-region, calculate the distribution density of the pixels in the lesion sub-region, and record the density value and corresponding sub-region identifier.

[0140] For each lesion sub-region, the total number of pixels contained in the sub-region is counted, and the area of the rectangular boundary range corresponding to the sub-region is calculated (i.e. (max_row - min_row + 1) multiplied by (max_col - min_col + 1)). The distribution density of the pixels in the lesion sub-region is obtained by dividing the total number of pixels by the area of the rectangular boundary range. The distribution density value of each lesion sub-region and its corresponding sub-region identifier (i.e. the label value assigned by the connected region labeling algorithm) are recorded to form a density statistics list.

[0141] Step S1478: The pixel coordinate range and distribution density value of each lesion sub-region are integrated to form the pixel distribution information of the lesion region.

[0142] The pixel coordinate range (min_row, max_row, min_col, max_col) of each lesion sub-region is integrated with the corresponding distribution density value. The above information of each lesion sub-region is stored in association with the sub-region identifier to form structured pixel distribution information of the lesion region. For example, the information of the lesion with sub-region identifier 1 includes the corresponding four coordinate values and distribution density value; the information of the lesion with sub-region identifier 2 also includes the corresponding coordinate range and density value, etc. The above structured information can clearly present the spatial range and pixel distribution characteristics of each lesion.

[0143] Step S1479: The pixel distribution information of the organ contour is associated with the pixel distribution information of the lesion region, and the pixel distribution information of the organ contour contains the spatial range corresponding to the pixel distribution information of the lesion region.

[0144] The pixel distribution information of the organ contour is associated with the pixel distribution information of the lesion region through coordinate comparison. It is checked whether the pixel coordinate range of each lesion sub-region is completely within the pixel coordinate range of the organ contour. Since the lesion is located inside the liver, the coordinate range of the lesion sub-region should normally be contained in the coordinate range of the organ contour. The identifier of each lesion sub-region is stored in association with the identifier of the organ contour to indicate that the lesion belongs to the liver organ. At the same time, an association field is added to the pixel distribution information of the organ contour to record the list of lesion sub-region identifiers contained therein, realizing the mutual association of the two.

[0145] Step S14710: A corresponding attribute identifier is added to the pixel distribution information of the organ contour and the pixel distribution information of the lesion region, and the attribute identifier includes the organ type attribute, the lesion type attribute, and the image modality attribute.

[0146] An attribute identifier is added to the pixel distribution information of the organ contour, wherein the organ type attribute is set to "liver", and the image modality attribute is set to the modality corresponding to the image (such as "computed tomography", "magnetic resonance imaging", or "ultrasound"). An attribute identifier is added to the pixel distribution information of the lesion region, wherein the organ type attribute is also set to "liver", the lesion type attribute is set to the type corresponding to the lesion (such as "liver hemangioma" or "liver cell carcinoma"), and the image modality attribute is consistent with the image modality attribute of the organ contour. The attribute identifier is added to the corresponding pixel distribution information in the form of a key-value pair, facilitating subsequent data management and retrieval.

[0147] Step S14711: The organ contour pixel distribution information and the lesion region pixel distribution information after adding the attribute identifier are integrated to generate a preliminary segmentation result.

[0148] The organ contour pixel distribution information and the lesion region pixel distribution information after adding the attribute identifier are integrated according to a preset data structure. The integrated preliminary segmentation result is a structured data object, which includes an organ contour module and a lesion region module. The organ contour module stores the pixel coordinate sequence of the organ contour, the attribute identifier, and the associated lesion sub-region identifier; the lesion region module stores the pixel coordinate range, the distribution density value, the attribute identifier, and the associated organ contour identifier of each lesion sub-region. The preliminary segmentation result completely records the segmentation information of the contour of the liver organ and the internal lesions.

[0149] Step S148: If the medical image original data set contains multiple organ image data, the above processing procedure is repeated to generate a preliminary segmentation result corresponding to each organ image data, forming a preliminary segmentation result set.

[0150] It is checked whether the medical image original data set contains multiple liver image data. If multiple image data are contained, the processing procedures of steps S141 to S14711 are repeated for each image data, i.e., the clinical anatomical annotation data analysis, the annotation rule matching, the segmentation constraint parameter conversion, the feature atlas segmentation constraint processing, and the preliminary segmentation result generation are sequentially performed.

[0151] A corresponding preliminary segmentation result is generated for each image data, and all preliminary segmentation results are sorted according to the image ID to form a preliminary segmentation result set. The preliminary segmentation result set is stored in the form of a list, and each element is a preliminary segmentation result of an image, which can comprehensively reflect the segmentation of all liver images in the medical image original data set.

[0152] Step S150: based on the association between the set of preliminary segmentation results and the clinical anatomical annotation data, optimize the segmentation parameters of the medical image segmentation model, and output the final medical image segmentation result, which includes organ contour segmentation results and lesion region segmentation results.

[0153] By using the correspondence between the set of preliminary segmentation results and the clinical anatomical annotation data, the deviation area is found by comparison, and the feature extraction layer weight parameters and the segmentation output layer constraint parameters of the medical image segmentation model are adjusted according to the deviation information. The adjusted parameters are used for re-segmentation processing until the segmentation result meets the preset accuracy requirement, and finally the segmentation result containing the liver contour and the lesion region is output.

[0154] Step S151: establish a one-to-one correspondence between the set of preliminary segmentation results and the clinical anatomical annotation data, and each preliminary segmentation result is associated with a corresponding clinical anatomical annotation data entry.

[0155] Extract the image ID attribute of each preliminary segmentation result in the set of preliminary segmentation results, and extract the image ID attribute of each annotation data in the clinical anatomical annotation data. According to the consistency of the image ID, a one-to-one correspondence between the preliminary segmentation result and the clinical anatomical annotation data is established. That is, each preliminary segmentation result is matched to the corresponding clinical anatomical annotation data entry through its image ID, so that each segmentation result can be referred to the accurate annotation data in the subsequent comparison process. The corresponding relationship is stored in the form of a mapping table, which records the association between the unique identifier of the preliminary segmentation result and the unique identifier of the clinical anatomical annotation data entry.

[0156] Step S152: for each correspondence, extract the organ contour pixel distribution information in the preliminary segmentation result, and compare it with the organ contour annotation information in the corresponding clinical anatomical annotation data entry to determine the deviation area of the organ contour segmentation.

[0157] For each correspondence in the mapping table, first extract the organ contour pixel distribution information in the preliminary segmentation result, specifically the pixel coordinate sequence of the organ contour; then extract the organ contour annotation information in the corresponding clinical anatomical annotation data entry, which is also the pixel coordinate sequence of the organ contour manually annotated by the physician.

[0158] The two coordinate sequences are compared using a pixel-level comparison algorithm. The difference between the organ contour pixels in the preliminary segmentation result and the organ contour pixels in the clinical anatomical annotation is calculated: for pixels that exist in the preliminary segmentation result but do not exist in the clinical anatomical annotation, they are determined to be over-segmented pixels; for pixels that exist in the clinical anatomical annotation but do not exist in the preliminary segmentation result, they are determined to be under-segmented pixels. The over-segmented pixels and the under-segmented pixels together constitute the deviation region of the organ contour segmentation, and the coordinates of these deviation pixels and the deviation type (over-segmented or under-segmented) are recorded.

[0159] Step S153: Extract the lesion region pixel distribution information in the preliminary segmentation result, and compare it with the lesion region annotation information in the corresponding clinical anatomical annotation data entry to determine the deviation region of the lesion region segmentation.

[0160] Similarly, for each corresponding relationship, the pixel coordinate range and the pixel coordinate list of each lesion sub-region in the preliminary segmentation result are extracted; and the pixel coordinate list of the lesion region manually annotated by the physician in the corresponding clinical anatomical annotation data entry is extracted.

[0161] A pixel-level comparison algorithm similar to the organ contour comparison is used to compare the pixel coordinate list of each lesion sub-region with the lesion region pixel coordinate list in the clinical anatomical annotation. Over-segmented pixels (pixels segmented out that do not appear in the annotation) and under-segmented pixels (pixels in the annotation that are not segmented out) of the lesion region in the preliminary segmentation result are identified, and it is checked whether there is a missed segmentation lesion (a lesion in the annotation that does not appear in the preliminary segmentation result) or a false positive segmentation lesion (a lesion in the preliminary segmentation result that does not appear in the annotation). The coordinates of the above deviation pixels, the deviation type, and the identification of the missed segmentation or false positive lesion are recorded to constitute the deviation region of the lesion region segmentation.

[0162] Step S154: Integrate the deviation region of the organ contour segmentation and the deviation region of the lesion region segmentation to generate a deviation region set, each deviation region in the deviation region set being associated with corresponding pixel position information and deviation degree information.

[0163] The organ contour segmentation deviation region determined in step S152 and the lesion region segmentation deviation region determined in step S153 are integrated. Each deviation region is assigned a unique deviation identifier, and its corresponding pixel position information (pixel coordinates), deviation type (over-segmented, under-segmented, missed segmentation, false positive), and deviation degree information are associated. The deviation degree information is determined by calculating the proportion of deviation pixels to the total pixels of the corresponding annotation region, and the higher the proportion, the more serious the deviation. All integrated deviation regions are stored according to the image ID and the deviation type to generate a deviation region set.

[0164] Step S155: inputting the deviation region set into a parameter optimization layer of the medical image segmentation model, and locating feature extraction layer weight parameters and segmentation output layer constraint parameters corresponding to the corresponding pixel position in the medical image segmentation model according to the pixel position information of the deviation region.

[0165] For example, step S1551: analyzing the pixel position information corresponding to each deviation region in the deviation region set to determine the row coordinate, column coordinate and depth coordinate of each pixel in the original medical image data.

[0166] For each deviation region entry in the deviation region set, the pixel position information is extracted to determine the row coordinate and column coordinate of each deviation pixel in the original medical image data. For three-dimensional medical image data (such as a tomographic sequence of computed tomography), the depth coordinate (i.e. the tomographic sequence number) also needs to be determined. The above coordinate information is arranged as structured data, each deviation pixel corresponding to a set of row, column and depth coordinates, ensuring accurate reflection of its spatial position in the original image.

[0167] Step S1552: converting the row coordinate, column coordinate and depth coordinate into feature mapping coordinates in the feature extraction layer of the medical image segmentation model, the conversion process being determined based on the down-sampling ratio or up-sampling ratio of the feature extraction layer.

[0168] The feature extraction layer reduces the spatial size of the feature map through down-sampling operation when processing image data, so the pixel coordinates in the original image need to be converted into the coordinates of the feature mapping in the feature extraction layer. The conversion process is calculated according to the down-sampling ratio of the feature extraction layer. If the feature extraction layer undergoes multiple down-sampling, the total down-sampling ratio is the product of the down-sampling ratios of each time, then the row coordinate of the feature mapping coordinate is equal to the original image row coordinate divided by the total down-sampling ratio, the column coordinate is equal to the original image column coordinate divided by the total down-sampling ratio, and the depth coordinate is converted according to the processing method of the feature extraction layer on the depth dimension (such as also down-sampling by ratio). Through this conversion process, the correspondence between the original image pixel coordinates and the feature mapping coordinates is established.

[0169] Step S1553: locating the convolution kernel responsible for extracting the feature corresponding to the feature mapping coordinates in the feature extraction layer according to the feature mapping coordinates, each convolution kernel being associated with a corresponding weight parameter group.

[0170] The feature extraction layer is composed of multiple convolution layers, each convolution layer containing multiple convolution kernels. According to the converted feature mapping coordinates, it is determined which layer of convolution kernel in the feature extraction layer extracts the feature corresponding to the coordinates. Specifically, the level of feature mapping corresponds to the level of convolution layer, and each position in the feature mapping is generated by sliding calculation of the corresponding level convolution kernel on the original image.

[0171] After locating the convolution kernel responsible for the feature mapping coordinates, the weight parameter group associated with the convolution kernel is extracted. The weight parameter group contains all the weight values used by the convolution kernel in the convolution calculation process, which directly affects the feature extraction effect and is the key object of parameter optimization.

[0172] Step S1554: Record the layer number, channel number, and weight parameter group index of the located convolution kernel to form the feature extraction layer weight parameter positioning information.

[0173] The convolution layer level number where the located convolution kernel is located (such as the first convolution layer, the second convolution layer, etc.), the channel number of the convolution kernel in the convolution layer, and the index address of the weight parameter group in the model parameter storage are recorded. The above information is integrated into the feature extraction layer weight parameter positioning information, and each positioning information corresponds to a weight parameter group associated with a bias pixel, ensuring that the weight parameter to be adjusted can be accurately found.

[0174] Step S1555: Analyze the bias type corresponding to each bias region in the bias region set, which includes organ contour bias type or lesion region bias type.

[0175] Each entry in the bias region set is analyzed again, and the bias type is divided into organ contour bias type and lesion region bias type according to the bias type description. The organ contour bias type includes over-segmentation and under-segmentation of the organ contour; the lesion region bias type includes over-segmentation, under-segmentation, missed segmentation, and false positive segmentation of the lesion. The bias type of each bias region is clearly attributed.

[0176] Step S1556: According to the bias type, match the corresponding constraint parameter category from the constraint parameter library of the segmentation output layer. The organ contour bias type corresponds to the contour constraint parameter category, and the lesion region bias type corresponds to the lesion constraint parameter category.

[0177] The constraint parameter library of the segmentation output layer stores different categories of constraint parameters, including contour constraint parameter category and lesion constraint parameter category. According to the bias type attribution of the bias region, the corresponding constraint parameter category is matched: if it is an organ contour bias type, the contour constraint parameter category (including contour feature response threshold, contour continuity requirement parameter, etc.) is matched; if it is a lesion region bias type, the lesion constraint parameter category (including lesion feature intensity range parameter, lesion and surrounding tissue boundary distinction parameter, etc.) is matched.

[0178] Step S1557: In the matched constraint parameter category, according to the feature mapping coordinates corresponding to the pixel position information of the bias region, locate the constraint parameter entry responsible for the region segmentation of the feature mapping coordinates.

[0179] In this embodiment, the constraint parameter category of the segmentation output layer is divided into multiple constraint parameter entries according to the spatial region of the feature mapping, each entry corresponds to a specific feature mapping coordinate range, and contains specific constraint parameters applicable in the range.

[0180] Taking the organ contour deviation type as an example, in the matched contour constraint parameter category, according to the feature mapping coordinates of the deviation region, the parameter entry range to which the coordinates belong is found, and the corresponding contour feature response threshold parameter entry and contour continuity requirement parameter entry are located. For the lesion region deviation type, in the lesion constraint parameter category, according to the feature mapping coordinates, the corresponding lesion feature intensity range parameter entry and lesion and surrounding tissue boundary distinction parameter entry are located. Each constraint parameter entry records the specific constraint value and the applicable feature mapping coordinate range, ensuring that the located parameters accurately correspond to the spatial position of the deviation region.

[0181] Step S1558: Record the category number, parameter index and applicable range description of the located constraint parameter entry to form the segmentation output layer constraint parameter positioning information.

[0182] The located constraint parameter entry is information extracted, and the constraint parameter category number to which the entry belongs (such as the contour constraint parameter category number is 1, and the lesion constraint parameter category number is 2), the storage index address of the parameter in the constraint parameter library, and the applicable range description (i.e. the corresponding feature mapping coordinate range) of the parameter entry are recorded. The above information is integrated into the segmentation output layer constraint parameter positioning information, each positioning information is associated with the corresponding deviation region identifier, and the specific deviation problem to which the constraint parameter adjustment is directed is clear.

[0183] Step S1559: Integrate the feature extraction layer weight parameter positioning information and the segmentation output layer constraint parameter positioning information to generate a parameter positioning set, and each positioning information in the parameter positioning set is associated with a corresponding deviation region identifier.

[0184] The feature extraction layer weight parameter positioning information generated in step S1554 and the segmentation output layer constraint parameter positioning information generated in step S1558 are associated and integrated according to the deviation region identifier. Each deviation region identifier corresponds to a group of weight parameter positioning information and constraint parameter positioning information, forming a complete parameter positioning record. All parameter positioning records are summarized to form a parameter positioning set, which clearly presents the model parameter position that needs to be adjusted for each deviation region.

[0185] Step S15510: transmit the parameter positioning set to a parameter adjustment unit of the parameter optimization layer, and the parameter adjustment unit calls corresponding weight parameters and constraint parameters in the medical image segmentation model according to the hierarchical number, the channel number, and the parameter index in the parameter positioning set.

[0186] The parameter positioning set is transmitted to the parameter adjustment unit of the parameter optimization layer through an internal data bus. The parameter adjustment unit analyzes each record in the parameter positioning set, locates the corresponding convolution kernel in the feature extraction layer according to the convolution layer hierarchical number and the channel number therein, and then calls the weight parameters of the convolution kernel through the weight parameter group index; at the same time, the corresponding constraint parameters are called from the constraint parameter library of the segmentation output layer according to the constraint parameter category number and the parameter index. After the calling is completed, the parameter adjustment unit loads the above-mentioned parameters into a temporary adjustment buffer, and prepares for parameter updating operation.

[0187] Step S156: adjust the located feature extraction layer weight parameters according to the deviation degree information of the deviation region, and enhance the extraction ability of the features related to the deviation region, and adjust the located segmentation output layer constraint parameters.

[0188] The parameter adjustment unit determines the parameter adjustment amplitude according to the deviation degree information of the deviation region. The more serious the deviation degree is, the greater the adjustment amplitude is. For the weight parameters of the feature extraction layer, if the deviation region is under-segmentation (i.e. the model does not sufficiently extract the features of the region), the weight values of the located convolution kernel weight parameters related to the features of the deviation region are increased, and the sensitivity of the model to the edge, density and other features of the region is enhanced; if the deviation region is over-segmentation (i.e. the model excessively extracts irrelevant features), the corresponding weight values are reduced, and the response of the model to irrelevant features is reduced.

[0189] For the constraint parameters of the segmentation output layer, if the organ contour is under-segmentation, the contour feature response threshold is appropriately reduced, so that more pixels meeting the contour feature are determined as organ regions; if there is over-segmentation, the contour feature response threshold is increased, and the contour pixels are strictly screened. If the lesion region is under-segmentation, the lesion feature intensity range parameter is expanded, and more pixels close to the lesion feature are included in the lesion region; if there is over-segmentation, the lesion feature intensity range parameter is reduced, and the lesion and the surrounding tissue boundary distinguishing parameter is increased, and the distinction degree of the lesion and the normal tissue is enhanced.

[0190] Step S157: re-segment the medical image feature atlas using the adjusted weight parameters and constraint parameters to generate an optimized segmentation result, and associate and compare the optimized segmentation result with the corresponding clinical anatomical annotation data again to determine whether the deviation degree meets the preset accuracy requirement.

[0191] The adjusted feature extraction layer weight parameters and segmentation output layer constraint parameters are updated into the medical image segmentation model, and other structures and parameters of the model remain unchanged. The previously generated medical image feature atlas is re-input into the updated model, and the model processes according to the segmentation process of steps S145 to S14711 to generate an optimized segmentation result.

[0192] Subsequently, according to the comparison method of steps S152 to S154, the optimized segmentation result is again compared with the corresponding clinical anatomical annotation data at the pixel level, and a new deviation degree is calculated. The new deviation degree is compared with the preset accuracy requirement (such as the deviation pixel ratio being lower than the set proportion), to determine whether the optimized segmentation result meets the requirement.

[0193] Step S158: If the preset accuracy requirement is met, the optimized segmentation result is determined as the final medical image segmentation result corresponding to the organ image data.

[0194] When the comparison result shows that the deviation degree of the optimized segmentation result meets the preset accuracy requirement, it indicates that the model parameter adjustment has achieved the expected effect. At this time, the optimized segmentation result is determined as the final medical image segmentation result corresponding to the organ image data, which contains complete liver contour pixel distribution information and lesion region pixel distribution information, and has high segmentation accuracy.

[0195] Step S159: If the preset accuracy requirement is not met, the above parameter adjustment and segmentation processing process is repeated until the deviation degree of the segmentation result meets the preset accuracy requirement.

[0196] If the comparison result shows that the deviation degree of the optimized segmentation result does not meet the preset accuracy requirement, return to step S155, re-input the new deviation region set into the parameter optimization layer, and again locate the model parameters that need to be adjusted. The parameter adjustment unit re-determines the adjustment range according to the new deviation degree information, adjusts the model parameters again, and then repeats the re-segmentation and comparison process of step S157. The parameter adjustment-re-segmentation-result comparison cycle process continues until the deviation degree of the optimized segmentation result meets the preset accuracy requirement, and then step S158 is executed to determine the final segmentation result, to ensure that the segmentation result output by the model meets the accuracy standard of clinical application.

[0197] Step S1510: The final medical image segmentation results corresponding to all organ image data are sorted to form a final medical image segmentation result set, which contains the organ contour segmentation result and the lesion region segmentation result of each organ image data.

[0198] All organ image data in the medical image original data set is traversed to collect the final medical image segmentation result corresponding to each image. The above results are sorted according to the order of image ID, and metadata tags such as image modality and examination time are added to each final segmentation result. All the sorted final segmentation results are summarized to form a final medical image segmentation result set, each entry of which contains the organ contour segmentation result (contour pixel coordinate sequence, attribute identifier) and the lesion region segmentation result (coordinate range, distribution density, attribute identifier) of the corresponding liver image, which can be directly used for subsequent application scenarios such as clinical diagnosis report generation and lesion quantitative analysis.

[0199] Based on the same inventive concept, please refer to Figure 2 The application provides a deep learning-based image segmentation system 100 for executing the above-mentioned inspection video stream processing method, and a structure schematic block diagram of the deep learning-based image segmentation system 100 is shown in FIG. 1. The deep learning-based image segmentation system 100 can include a communication unit 110, a machine readable storage medium 120, and a processor 130.

[0200] In this embodiment, the machine readable storage medium 120 and the processor 130 are located in the deep learning-based image segmentation system 100 and are separately arranged. However, it should be understood that the machine readable storage medium 120 can also be independent of the deep learning-based image segmentation system 100, and can be accessed by the processor 130 through a bus interface. Alternatively, the machine readable storage medium 120 can also be integrated into the processor 130, and can communicate and interact with external systems through the communication unit 110.

[0201] The processor 130 is the control center of the deep learning-based image segmentation system 100, and connects all parts of the deep learning-based image segmentation system 100 through various interfaces and lines. By running or executing software programs and / or modules stored in the machine readable storage medium 120, and calling data stored in the machine readable storage medium 120, the processor 130 performs various functions of the deep learning-based image segmentation system 100 and processes data, thereby overall monitoring the deep learning-based image segmentation system 100. Optionally, the processor 130 can include one or more processing cores; for example, the processor 130 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, the user interface and the application program, and the modem processor mainly processes the wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor. The machine readable storage medium 120 is used to store machine executable instructions for executing the scheme of the application, and the processor 130 is used to execute the machine executable instructions stored in the machine readable storage medium 120, so as to realize the inspection video stream processing method provided by the foregoing method embodiment.

[0202] It should be noted that the foregoing description of implementations of the application has been presented for the purposes of simplicity and clarity. It is not intended to be an exhaustive description of all implementations of the application.

Claims

1. A deep learning-based image segmentation method, characterized in that, The method includes: Acquire a set of raw medical image data and corresponding clinical anatomical annotation data. The set of raw medical image data includes organ image data of different modalities, and the clinical anatomical annotation data includes organ contour annotation information and lesion area annotation information. Load a pre-trained medical image segmentation model, which includes an anatomical structure prior information integration module and a clinical annotation rule mapping module; The raw medical image data set is input into the feature extraction layer of the medical image segmentation model, and the organ anatomical features output by the anatomical structure prior information are combined with the anatomical structure integration module to generate a medical image feature map. The clinical annotation rule mapping module is invoked to convert the clinical anatomical annotation data into segmentation constraint parameters, and the medical image feature map is subjected to segmentation constraint processing to obtain a preliminary segmentation result set. Based on the correlation between the preliminary segmentation result set and the clinical anatomical annotation data, the segmentation parameters of the medical image segmentation model are optimized, and the final medical image segmentation result is output. The final medical image segmentation result includes organ contour segmentation result and lesion region segmentation result.

2. The image segmentation method based on deep learning according to claim 1, characterized in that, The pre-trained medical image segmentation model includes: Extract common features of organ anatomical structures corresponding to organ image data of different modalities in the original medical image dataset. The common features of organ anatomical structures include organ spatial position relationship features and organ tissue density distribution features. An anatomical structure prior information database is constructed based on the common features of the organ anatomical structure. Each piece of prior information in the anatomical structure prior information database is associated with the corresponding organ modality identifier and anatomical location identifier. An anatomical structure prior information integration module is constructed. The anatomical structure prior information integration module includes a prior information retrieval submodule and a feature fusion submodule. The prior information retrieval submodule is used to match the corresponding organ anatomical features from the anatomical structure prior information database based on the modality identifier and anatomical location identifier of the input organ image data. The feature fusion submodule is used to fuse the organ anatomical features with the basic features output by the feature extraction layer of the medical image segmentation model. Collect standardized annotation data for image segmentation of different organs during clinical diagnosis and treatment. The standardized annotation data includes the accuracy requirements for organ contour annotation and the boundary range requirements for lesion area annotation. A clinical annotation rule library is constructed based on the annotation standard data. Each rule in the clinical annotation rule library is associated with a corresponding organ type identifier and lesion type identifier. A clinical annotation rule mapping module is constructed, which includes a rule matching submodule and a parameter conversion submodule. The rule matching submodule is used to match the corresponding annotation rules from the clinical annotation rule library based on the organ type identifier and lesion type identifier of the input clinical anatomical annotation data. The parameter conversion submodule is used to convert the annotation rules into segmentation constraint parameters that can be recognized by the medical image segmentation model. The anatomical structure prior information integration module, the clinical annotation rule mapping module, and the basic network structure of the medical image segmentation model are integrated. The basic network structure includes a feature extraction layer and a segmentation output layer. The output of the anatomical structure prior information integration module is connected to the output of the feature extraction layer, and the output of the clinical annotation rule mapping module is connected to the input of the segmentation output layer. The integrated medical image segmentation model is pre-trained using a training dataset containing medical image data and corresponding clinical anatomical annotation data. The model weight parameters are iteratively adjusted to ensure that the deviation between the segmentation results output by the medical image segmentation model and the clinical anatomical annotation data is within a preset range, thus obtaining the pre-trained medical image segmentation model.

3. The image segmentation method based on deep learning according to claim 2, characterized in that, The construction of the prior information database of anatomical structures based on the common features of the organ anatomical structures includes: The common features of the anatomical structures of the organs are classified, and the common features belonging to the same organ type are grouped into one category to form a set of organ feature categories. Each organ feature category is associated with the corresponding organ type name. For each organ feature category, the organ spatial position relationship features are extracted. The organ spatial position relationship features include the relative position features of the organ with adjacent organs and the position features of the organ in the human anatomical coordinate system. Extract organ tissue density distribution features from each organ feature category. The organ tissue density distribution features include the average density features and density change trend features of different regions of the organ. The spatial location relationship features of the organ are integrated with the tissue density distribution features of the organ to form the prior information entries of the anatomical structure of the organ feature category; Each anatomical structure prior information entry is assigned an organ modality identifier, which is determined based on the organ image data modality corresponding to the prior information. Different modalities correspond to different identifier symbols. Each anatomical structure prior information entry is assigned an anatomical location identifier, which is determined based on the anatomical location of the organ corresponding to the prior information in the human body. Different anatomical locations correspond to different identifiers. Establish a storage structure for the prior information database of anatomical structures, wherein the storage structure includes a prior information entry field, an organ modality identifier field, an anatomical location identifier field, and a creation time field; Each anatomical structure prior information entry and its corresponding organ modality identifier, anatomical location identifier, and creation time information are filled into the corresponding field of the storage structure.

4. The image segmentation method based on deep learning according to claim 1, characterized in that, The step of inputting the raw medical image data set into the feature extraction layer of the medical image segmentation model, and combining the organ anatomical features output by the anatomical structure prior information fusion module to generate a medical image feature atlas includes: Modal identification classification is performed on the organ image data of different modalities in the raw medical image data set, and the image data of different modalities of the same organ are grouped into the same data group, and each data group is associated with the corresponding organ type identifier; The organ image data in each data group are sequentially input into the feature extraction layer of the medical image segmentation model. The basic features of each modality image data are extracted through multi-layer convolution operations to generate the basic feature map corresponding to each modality. For each data group, the associated organ type identifier is extracted, and the organ type identifier is input into the prior information retrieval submodule of the anatomical structure prior information integration module. Based on the organ type identifier, the organ anatomical features corresponding to the organ type identifier are matched from the anatomical structure prior information database. The organ anatomical features include the spatial distance features between the organ and surrounding tissues and the density gradient features of the internal tissues of the organ. The basic feature map and the organ anatomical features are input into the feature fusion submodule of the anatomical structure prior information integration module. The organ anatomical features are embedded into the corresponding feature channels of the basic feature map by adopting a cross-feature channel information interaction method. The feature map embedded with organ anatomical features is processed by channel-dimensional information integration to generate a single-modal fusion feature map. If the data set contains organ image data of multiple modalities, the single-modal fusion feature maps corresponding to each modality are subjected to intermodal feature complementation processing to extract effective information related to organ contours and lesion areas from each modality feature map. The feature maps after intermodal complementation processing are then subjected to spatial dimension feature alignment processing. The spatially aligned feature maps are then subjected to multi-layer feature aggregation processing to generate a medical image feature map that includes the overall structural features of the organ and the detailed features of local lesions.

5. The image segmentation method based on deep learning according to claim 4, characterized in that, The feature fusion submodule, which integrates the basic feature map and the organ anatomical features into the prior information of the anatomical structure module, employs a cross-feature channel information interaction method to embed the organ anatomical features into the corresponding feature channels of the basic feature map, including: The feature channel structure of the basic feature map is analyzed to determine the feature type corresponding to each feature channel. The feature type includes spatial location feature type, density distribution feature type, and edge contour feature type. Analyze the organ anatomical features, determine the feature types contained therein, and match the organ anatomical feature subset that is consistent with the feature types in the basic feature atlas; For each feature type, determine the channel number of the corresponding feature channel in the basic feature map, and establish a mapping relationship between feature type and channel number; According to the mapping relationship, the feature data in the organ anatomical feature subset are assigned to the corresponding feature channels, and each feature channel receives organ anatomical feature data that matches its feature type. The organ anatomical feature data assigned to the feature channel is subjected to feature dimension adjustment and numerical standardization processing to ensure that the dimension and numerical range of the organ anatomical feature data are consistent with the dimension and numerical range of the basic feature data in the corresponding feature channel. A cross-feature channel information interaction algorithm is adopted to perform pixel-by-pixel information fusion between the adjusted and standardized organ anatomical feature data and the basic feature data in the corresponding feature channel. During the fusion process, the image detail information in the basic feature data and the anatomical structure information in the organ anatomical feature data are preserved. The fused feature channel data is subjected to in-channel information enhancement processing to strengthen the feature signals related to organ contours and lesion areas. The above processing is repeated until the organ anatomical feature data corresponding to all feature types are embedded into the corresponding feature channels of the basic feature atlas, generating a feature atlas with embedded organ anatomical features.

6. The image segmentation method based on deep learning according to claim 1, characterized in that, The clinical annotation rule mapping module is invoked to convert the clinical anatomical annotation data into segmentation constraint parameters, and the medical image feature atlas is subjected to segmentation constraint processing to obtain a preliminary segmentation result set, including: The clinical anatomical annotation data is analyzed to extract organ contour annotation information and lesion area annotation information, determine the organ type identifier corresponding to the organ contour annotation information, and determine the lesion type identifier corresponding to the lesion area annotation information; The organ type identifier and the lesion type identifier are input into the rule matching submodule of the clinical annotation rule mapping module. The corresponding organ contour annotation rule is matched from the clinical annotation rule library according to the organ type identifier, and the corresponding lesion region annotation rule is matched from the clinical annotation rule library according to the lesion type identifier. The organ contour annotation rules and the lesion region annotation rules are input into the parameter conversion submodule of the clinical annotation rule mapping module. The organ contour annotation rules are converted into contour constraint parameters, which include the feature response threshold of the organ contour edge and the contour continuity requirement parameters. The lesion region annotation rules are converted into lesion constraint parameters, which include the feature intensity range parameters of the lesion region and the boundary distinction parameters between the lesion and the surrounding tissue. The contour constraint parameters and the lesion constraint parameters are integrated to generate a segmentation constraint parameter set, wherein each parameter in the segmentation constraint parameter set is associated with a corresponding feature channel identifier and spatial location identifier; The medical image feature map and the segmentation constraint parameter set are input into the segmentation output layer of the medical image segmentation model. Based on the feature channel identifier in the segmentation constraint parameter set, the corresponding feature channel in the medical image feature map is located. For the located feature channels, the feature values ​​in the feature channels are constrained and filtered according to the spatial location identifier and specific constraint parameters in the segmentation constraint parameter set. The filtered feature maps are then classified at the pixel level, classifying each pixel into organ region pixels, lesion region pixels, or background region pixels. A preliminary segmentation result is generated based on the classification result. The preliminary segmentation result includes pixel distribution information of organ contours and pixel distribution information of lesion regions. If the original medical image dataset contains image data of multiple organs, the above processing procedure is repeated to generate preliminary segmentation results corresponding to each organ image data, forming a preliminary segmentation result set.

7. The image segmentation method based on deep learning according to claim 6, characterized in that, The step of generating preliminary segmentation results based on classification results includes: Extract the pixel coordinates of all pixels classified as organ regions from the classification results, and arrange the pixel coordinates in row and column order to form a list of organ region pixel coordinates. Edge detection processing is performed on the pixel coordinates in the list of pixel coordinates of the organ region to identify the boundary pixel coordinates of the organ region. The boundary pixel coordinates constitute the pixel distribution basis of the organ contour. The boundary pixel coordinates are connected in spatial order to form a continuous organ contour line. The pixel coordinate sequence of each contour line is recorded to form the pixel distribution information of the organ contour. Extract the pixel coordinates of all pixels classified as lesion areas from the classification results, and arrange the pixel coordinates in row and column order to form a list of lesion area pixel coordinates. The pixel coordinates in the list of pixel coordinates of the lesion area are divided into regions. If multiple pixel coordinates are spatially adjacent and belong to the same lesion type, the pixel coordinates are classified into the same lesion sub-region. Record the pixel coordinate range of each lesion sub-region, including the minimum row coordinate, maximum row coordinate, minimum column coordinate, and maximum column coordinate of the lesion sub-region, which constitutes the pixel distribution basis of the lesion region; Density statistics are performed on the pixel coordinates of each lesion sub-region. The distribution density of pixels within the lesion sub-region is calculated, and the density value and the corresponding sub-region identifier are recorded. The pixel coordinate range and distribution density value of each lesion sub-region are integrated to form the pixel distribution information of the lesion region; The pixel distribution information of the organ contour is associated with the pixel distribution information of the lesion region, and the pixel distribution information of the organ contour includes the spatial range corresponding to the pixel distribution information of the lesion region. Add corresponding attribute identifiers to the pixel distribution information of the organ contour and the pixel distribution information of the lesion region. The attribute identifiers include organ type attribute, lesion type attribute, and image modality attribute. The pixel distribution information of the organ contour after adding attribute labels is integrated with the pixel distribution information of the lesion area to generate a preliminary segmentation result.

8. The image segmentation method based on deep learning according to claim 1, characterized in that, The process of optimizing the segmentation parameters of the medical image segmentation model based on the correlation between the preliminary segmentation result set and the clinical anatomical annotation data, and outputting the final medical image segmentation result, includes: Establish a one-to-one correspondence between the preliminary segmentation result set and the clinical anatomy annotation data, with each preliminary segmentation result associated with a corresponding clinical anatomy annotation data entry; For each correspondence, the pixel distribution information of the organ contour in the preliminary segmentation result is extracted and compared with the organ contour annotation information in the corresponding clinical anatomy annotation data entry to determine the deviation area of ​​organ contour segmentation. Extract the pixel distribution information of the lesion region from the preliminary segmentation results, and compare it with the lesion region annotation information in the corresponding clinical anatomical annotation data entries to determine the deviation areas of lesion region segmentation; The deviation regions of the organ contour segmentation and the deviation regions of the lesion region segmentation are integrated to generate a set of deviation regions. Each deviation region in the set of deviation regions is associated with corresponding pixel position information and deviation degree information. The set of deviation regions is input into the parameter optimization layer of the medical image segmentation model. Based on the pixel position information of the deviation regions, the weight parameters of the feature extraction layer and the constraint parameters of the segmentation output layer corresponding to the corresponding pixel positions in the medical image segmentation model are located. Based on the degree of deviation in the deviation region, adjust the weight parameters of the located feature extraction layer to enhance the ability to extract relevant features in the deviation region, and adjust the constraint parameters of the located segmentation output layer. The medical image feature map is re-segmented using the adjusted weight parameters and constraint parameters to generate an optimized segmentation result. The optimized segmentation result is then compared with the corresponding clinical anatomical annotation data to determine whether the degree of deviation meets the preset accuracy requirements. If the preset accuracy requirements are met, the optimized segmentation result will be determined as the final medical image segmentation result corresponding to the organ image data. If the preset accuracy requirements are not met, repeat the above parameter adjustment and segmentation process until the deviation of the segmentation result meets the preset accuracy requirements. The final medical image segmentation results corresponding to all organ image data are organized to form a final medical image segmentation result set, which includes the organ contour segmentation result and lesion region segmentation result of each organ image data.

9. A deep learning-based image segmentation system, characterized in that, include: processor; A machine-readable storage medium for storing machine-executable instructions of the processor; The processor is configured to execute the deep learning-based image segmentation method according to any one of claims 1 to 8 by executing the machine-executable instructions.

10. A computer program product, characterized in that, The computer program product includes machine-executable instructions stored in a computer-readable storage medium, wherein a processor of a computer device reads the machine-executable instructions from the computer-readable storage medium and executes the machine-executable instructions, causing the computer device to perform the deep learning-based image segmentation method as described in any one of claims 1 to 8.

Images