Medical image processing method and device, computer equipment and storage medium
Through regional segmentation and lymph node detection of medical image processing methods, combined with graphic similarity calculation, the accuracy of lymph node metastasis evaluation in patients with abdominal pelvic malignant tumors was solved, and the detection accuracy and clinical practicality were improved.
Patent Information
- Application Number
- CN202410936598.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-12
- Publication Date
- 2024-11-12
AI Technical Summary
Existing medical image processing techniques are difficult to accurately evaluate lymph node metastasis in patients with abdominal pelvic malignant tumors, especially the lack of effective anatomical partitioning and clinical application models in abdominal lymph node detection, resulting in inaccurate evaluation.
A medical image processing method is proposed to generate text data through regional segmentation and lymph node detection segmentation, and use graphic and text similarity calculations to assist medical workers in assessing the nature and metastasis of lymph nodes. The method includes segmentation of the region of interest, lymph node detection and segmentation, text data generation, and graphic and text similarity calculation, and combined with deep learning technology to improve the accuracy of lymph node detection.
Accurate assessment of lymph node metastasis in patients with abdominal pelvic malignant tumors has been achieved, which reduces the working pressure of radiologists, improves the accuracy of tumor staging, and is conducive to precise treatment.
Smart Images

Figure CN118941508A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of medical image processing methods, and in particular to a medical image processing method, apparatus, computer equipment and storage medium. Background Art
[0002] The number and area of lymph node metastasis determine the N stage of patients with abdominal and pelvic malignant tumors, which is of great clinical significance. Lymph node metastasis affects patients' treatment decisions, such as the formulation of surgical plans and the need for preoperative neoadjuvant therapy and postoperative adjuvant therapy, and is an important factor in judging the prognosis of patients with abdominal and pelvic malignant tumors. Therefore, it is necessary to propose a new medical image processing method to assist medical workers in accurately evaluating lymph node metastasis in patients with abdominal and pelvic malignant tumors based on property evaluation data. Summary of the invention
[0003] The present application aims to solve at least one of the technical problems in the related art to a certain extent. To this end, the present application proposes a medical image processing method, apparatus, computer equipment and storage medium.
[0004] The present application provides a medical image processing method, comprising: performing regional segmentation on a medical image to be analyzed to obtain partition mask data corresponding to a region of interest; performing lymph node detection and lymph node segmentation based on the medical image to be analyzed to obtain position data and size data of the lymph nodes; generating first-category text data and second-category text data according to the partition mask data, and the position data and size data of the lymph nodes; and performing graphic-text similarity calculation using the first-category text data and the second-category text data and the medical image to be analyzed, respectively, to obtain property evaluation data of the lymph nodes.
[0005] In one embodiment, the region of interest includes multiple anatomical partitions; the medical image to be analyzed is segmented to obtain partition mask data corresponding to the region of interest, including: inputting the medical image to be analyzed into a first resolution segmentation network to obtain overall mask data of the region of interest; wherein the first resolution segmentation network is connected to a second resolution segmentation network; the overall mask data and the medical image to be analyzed are input into the second resolution segmentation network to obtain partition mask data of each of the multiple anatomical partitions.
[0006] In one of the embodiments, lymph node detection and lymph node segmentation are performed based on the medical image to be analyzed to obtain location data and size data of the lymph nodes, including: performing lymph node detection on the medical image to be analyzed to obtain detection frame data of the lymph nodes; extracting a lymph node region image from the medical image to be analyzed based on the lymph node detection frame data; performing lymph node segmentation on the lymph node region image to obtain a three-dimensional segmented image of the lymph nodes; determining the size data of the lymph nodes based on the three-dimensional segmented image; and determining the center point coordinates of the lymph nodes based on the lymph node detection frame data as the location data of the lymph nodes.
[0007] In one of the embodiments, determining the size data of the lymph node based on the three-dimensional segmented image includes: determining the maximum cross-section of the lymph node based on the three-dimensional segmented image; and determining the long axis and short axis of the lymph node based on the coordinates of the contour points of the maximum cross-section as the size data of the lymph node.
[0008] In one of the embodiments, lymph node detection is performed on a medical image to be analyzed to obtain detection frame data of the lymph nodes, including: inputting the medical image to be analyzed into a first detection network for lymph node detection to obtain a preliminary detection result; wherein the first detection network is connected to a second detection network; inputting the preliminary detection result into the second detection network for lymph node detection to obtain an intermediate detection result; and performing false positive filtering based on the intermediate detection result to obtain detection frame data of the lymph nodes.
[0009] In one embodiment, the region of interest includes multiple anatomical partitions; the first category of text data and the second category of text data are generated based on the partition mask data, and the location data and size data of the lymph nodes, including: determining the partition result of the lymph nodes based on the location data and the partition mask data of the lymph nodes; wherein the partition result is used to describe the anatomical partition in which the lymph nodes are located; the first category of text data is generated based on the first category of text templates and the partition result and size data of the lymph nodes; and the second category of text data is generated based on the second category of text templates and the partition result and size data of the lymph nodes.
[0010] In one of the embodiments, the first category of text data and the second category of text data are used to perform graphic and text similarity calculations with the medical image to be analyzed, respectively, to obtain the property assessment data of the lymph node, including: performing feature extraction on the first category of text data to obtain first text features, performing feature extraction on the second category of text data to obtain second text features; performing feature extraction on the medical image to be analyzed to obtain image features; performing similarity calculations with the first text features and the second text features and the image features, respectively, to obtain graphic and text similarity data; and generating the property assessment data of the lymph node based on the graphic and text similarity data.
[0011] The present application also proposes a medical image processing device, including: a region of interest segmentation module, used to perform region segmentation on the medical image to be analyzed, and obtain partition mask data corresponding to the region of interest; a lymph node detection and segmentation module, used to perform lymph node detection and lymph node segmentation based on the medical image to be analyzed, and obtain the location data and size data of the lymph nodes; a graphic data generation module, used to generate a first category of text data and a second category of text data according to the partition mask data, and the location data and size data of the lymph nodes; a lymph node qualitative module, used to use the first category of text data and the second category of text data to perform graphic and text similarity calculations with the medical image to be analyzed, respectively, to obtain the property evaluation data of the lymph nodes.
[0012] The present application also provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any of the above methods when executing the computer program.
[0013] The present application also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any of the above methods are implemented.
[0014] In the above embodiment, the medical image to be analyzed is firstly segmented into regions, and the partition mask data corresponding to the region of interest can be obtained. Lymph node detection and lymph node segmentation are performed based on the medical image to be analyzed to obtain the location data and size data of the lymph nodes. The location partition of the lymph nodes can be determined based on the location data of the lymph nodes and the partition mask data corresponding to the region of interest. The location partition and the size data of the lymph nodes and different categories of text templates are combined to generate the first category of text data and the second category of text data. The first category of text data is used to calculate the similarity between the text and the medical image to be analyzed, and the second category of text data is used to calculate the similarity between the text and the medical image to be analyzed. According to the similarity calculation results, the property evaluation data that can describe the lymph nodes is determined in the first category of text data and the second category of text data. At this point, an auxiliary means is realized to help medical workers accurately evaluate the lymph node metastasis of patients with abdominal and pelvic malignant tumors. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative examples and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0016] Figure 1 is a flowchart of a medical image processing method according to an embodiment of the present application;
[0017] Figure 2 A schematic diagram of a lymph node detection and lymph node segmentation process of a medical image processing method according to an embodiment of the present application;
[0018] Figure 3 A schematic diagram of a lymph node detection process of a medical image processing method according to an embodiment of the present application;
[0019] Figure 4 A schematic diagram of a lymph node qualitative analysis process of a medical image processing method according to an embodiment of the present application;
[0020] Figure 5 is a flowchart of a medical image processing method according to yet another embodiment of the present application;
[0021] Figure 6 is a structural block diagram of a medical image processing device according to an embodiment of the present application;
[0022] Figure 7 The figure is a diagram of the internal structure of a computer device according to one embodiment of the present application. DETAILED DESCRIPTION
[0023] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.
[0024] High-resolution enhanced CT is a recommended examination method in the diagnosis and treatment guidelines of various abdominal and pelvic tumors, such as gastric cancer and colon cancer, for tumor staging and efficacy evaluation. At present, the introduction of artificial intelligence (AI) technology to build a computer-aided diagnosis system is an emerging and promising method to solve these problems. Deep learning is an artificial intelligence algorithm with great potential. It has shown encouraging preliminary results in the application of automatic lymph node detection. It has high accuracy and can help radiologists detect lymph nodes more quickly, reduce the work pressure of radiologists, improve work efficiency, and improve the accuracy of tumor staging, which is conducive to the precise treatment of cancer patients.
[0025] Although the application value of deep learning in lymph node detection has been preliminarily confirmed, the existing models are mostly used for lymph node detection in single sites such as the neck and mediastinum. There are few clinical application models for pelvic tumor lymph node detection. Models for abdominal lymph node detection lack corresponding lymph node anatomical divisions. The number of cases used for training and verification is small, and clinical application practice is still insufficient.
[0026] Based on this, the present application embodiment proposes a medical image processing method, please refer to the attached Figure 1 As shown, the method includes:
[0027] S110 , performing region segmentation on the medical image to be analyzed to obtain partition mask data corresponding to the region of interest.
[0028] The medical image to be analyzed may be a DICOM raw data file of a thin-layer abdominal and pelvic CT scan of a patient during the venous phase. The region of interest may be the entire abdominal and pelvic cavity region, which includes abdominal lymph node anatomical partitions, pelvic lymph node anatomical partitions, left inguinal lymph node anatomical partitions, and right inguinal lymph node anatomical partitions. The region of interest may include multiple anatomical partitions, and the partition mask data corresponding to the region of interest may be the partition mask data of each anatomical partition.
[0029] Specifically, the medical image to be analyzed can be input into the region of interest segmentation model, and the region of interest segmentation model is used to perform region segmentation on the medical image to be analyzed, so as to obtain partition mask data corresponding to the region of interest. In another embodiment, the region of interest segmentation model may include two sub-models connected in series, and the first sub-model is used to extract the overall region of interest from the medical image to be analyzed, and the second sub-model is used to segment the partial image corresponding to the overall region of interest in the medical image to be analyzed, so as to obtain corresponding partition mask data.
[0030] S120 , performing lymph node detection and lymph node segmentation based on the medical image to be analyzed to obtain location data and size data of the lymph nodes.
[0031] The location data of the lymph nodes are used to characterize the location of the lymph nodes in the medical image to be analyzed. The size data of the lymph nodes are used to characterize the size of the lymph nodes. Specifically, the location data of the lymph nodes can be obtained by performing lymph node detection on the medical image to be analyzed. The size data of the lymph nodes can be measured and calculated by performing lymph node segmentation on the lymph nodes.
[0032] Furthermore, the lymph nodes can be located based on the lymph node detection results, the lymph node segmentation results and the partition mask data. Exemplarily, lymph node detection is performed on the medical image to be analyzed, and the lymph node detection frame obtained by the lymph node detection result can further determine the location information of the lymph node. Based on the lymph node detection frame, a lymph node region image is extracted from the medical image to be analyzed, and lymph node segmentation is performed on the lymph node region image. The lymph node segmentation image obtained by the lymph node segmentation can determine the size data of the lymph node based on the lymph node segmentation image. Among them, the size data of the lymph node may include information such as the area size, the long and short diameters of the lymph node.
[0033] S130 , generating first-category text data and second-category text data according to the partition mask data, and the location data and size data of the lymph nodes.
[0034] Specifically, different categories of text templates are pre-set for lymph nodes of different properties, such as a first category text template and a second category text template. The first category text data can be generated based on the partition mask data, the location data and size data of the lymph node, and the first category text template. The second category text data can be generated based on the partition mask data, the location data and size data of the lymph node, and the second category text template.
[0035] Exemplarily, the first type of text data may be text data for describing benign lymph nodes, and the second type of text data may be text data for describing malignant lymph nodes. Similarly, the second type of text data may be text data for describing benign lymph nodes, and the first type of text data may be text data for describing malignant lymph nodes.
[0036] S140, using the first category of text data and the second category of text data to perform image-text similarity calculations with the medical image to be analyzed, to obtain lymph node property assessment data.
[0037] Among them, the first category of text data and the second category of text data are used to describe lymph nodes of different properties respectively. The property evaluation data can be understood as data used to qualitatively describe the properties of lymph nodes, for example, in the form of probability values. Before qualitatively characterizing the lymph nodes, similarity calculations can be performed based on the medical image to be analyzed and the text data of different categories. The more similar text data can be used to describe the actual situation of the lymph nodes in the medical image to be analyzed. It can be understood that the property evaluation data of the lymph nodes can be regarded as reference data for the property evaluation of the lymph nodes, and can be used as reference data with at least one of the location data and size data of the lymph nodes to assist the work of medical workers.
[0038] Therefore, the first similarity data is obtained by calculating the similarity between the first type of text data and the medical image to be analyzed, and the second similarity data is obtained by calculating the similarity between the second type of text data and the medical image to be analyzed. The first similarity data and the second similarity data are compared to determine the nature of the lymph nodes in the medical image to be analyzed, and generate the lymph node nature evaluation data.
[0039] Exemplarily, the probability that the lymph node corresponds to the first category of text data can be obtained by calculating the graphic and text similarity between the medical image to be analyzed and the first category of text data; the probability that the lymph node corresponds to the second category of text data can be obtained by calculating the graphic and text similarity between the medical image to be analyzed and the second category of text data. Finally, the nature of the lymph node can be judged by comparing the graphic and text similarity probability values corresponding to the medical image to be analyzed and different text data.
[0040] Exemplarily, if the similarity probability value between the medical image to be analyzed and the first category of text data exceeds a certain standard, the lymph nodes displayed in the medical image to be analyzed may be the lymph nodes described in the first category of text data, and the larger the probability value, the greater the possibility; similarly, if the similarity probability value between the medical image to be analyzed and the second category of text data exceeds a certain standard, the lymph nodes displayed in the medical image to be analyzed may be the lymph nodes described in the second category of text data, and the larger the probability value, the greater the possibility.
[0041] In the above implementation, lymph node detection and lymph node segmentation are performed based on the medical image to be analyzed to obtain the location data and size data of the lymph nodes; based on the location data of the lymph nodes and the partition mask data corresponding to the region of interest, the location partition of the lymph nodes can be determined, and the location partition and the size data of the lymph nodes and the preset text template are combined to generate the first type of text data and the second type of text data. The first type of text data is used to calculate the similarity between the text and the medical image to be analyzed, and the second type of text data is used to calculate the similarity between the text and the medical image to be analyzed. According to the similarity calculation results, the property evaluation data that can describe the lymph nodes is determined in the first type of text data and the second type of text data. At this point, a new auxiliary means is realized to help medical workers, and the property evaluation data is used to assist medical workers in accurately evaluating the lymph node metastasis of patients with abdominal and pelvic malignant tumors, thereby improving the accuracy of the evaluation of lymph node metastasis.
[0042] It should be noted that the implementation method in this specification is applicable to the scene of lymph node detection and qualitative diagnosis on the enhanced CT images of the abdomen and pelvis of patients with abdominal and pelvic malignant tumors. The scope of application is not limited to the type of primary malignant tumors and has strong clinical practicality. In addition, the size information can be obtained through the lymph node segmentation module, and the anatomical division where the lymph node is located can be detected, and finally the property evaluation data of the lymph node can be output, which can help radiologists to judge the patient's condition more quickly, reduce the work pressure of radiologists, improve work efficiency, and improve the accuracy of tumor staging, which is conducive to the precise treatment of tumor patients.
[0043] In some embodiments, performing regional segmentation on the medical image to be analyzed to obtain partition mask data corresponding to the region of interest includes: inputting the medical image to be analyzed into a first resolution segmentation network to obtain overall mask data of the region of interest. The first resolution segmentation network is connected to a second resolution segmentation network. The overall mask data and the medical image to be analyzed are input into the second resolution segmentation network to obtain partition mask data of each of the multiple anatomical partitions.
[0044] The region of interest may include a plurality of anatomical divisions. For example, the region of interest may include an abdominal lymph node anatomical division, a pelvic lymph node anatomical division, a left inguinal lymph node anatomical division, and a right inguinal lymph node anatomical division.
[0045] Among them, the first resolution segmentation network and the second resolution segmentation network have different resolutions when resampling the medical image, and a multi-scale cascade strategy can be used to connect the first resolution segmentation network and the second resolution segmentation network together to obtain a refined segmentation result of the region of interest.
[0046] It should be noted in advance that when training segmentation network models of different resolutions, the output of the first resolution network can be used as the input of the second resolution network together with all medical image training samples.
[0047] Specifically, the first resolution segmentation network can be a coarse resolution segmentation network; the second resolution segmentation network can be a fine resolution segmentation network, wherein the coarse resolution segmentation network can resample the medical image to be analyzed to an isotropic resolution of 6 mm, and process it to obtain overall mask data; the fine resolution segmentation network can resample the resolution of the input image in all directions to 2 mm based on the medical image to be analyzed and the overall mask data, and process it to obtain partition mask data of multiple anatomical partitions.
[0048] In the above implementation, region segmentation is performed on the medical image to be analyzed to obtain partition mask data corresponding to the region of interest. By adopting a multi-scale cascade strategy to connect the region segmentation module with different resolution networks, a refined segmentation result of the region of interest can be obtained.
[0049] In some embodiments, please refer to the attached Figure 2 , performing lymph node detection and lymph node segmentation based on the medical image to be analyzed to obtain the location data and size data of the lymph nodes may include the following steps:
[0050] S210: Perform lymph node detection on the medical image to be analyzed to obtain lymph node detection frame data.
[0051] The lymph node detection frame data is data representing the location of the lymph node lesion. For example, the lymph node detection frame data may be a bounding box of the lymph node lesion frame. Specifically, the lymph node detection model is used to perform target detection on the medical image to be analyzed to obtain the lymph node detection frame data.
[0052] S220 , extracting a lymph node region image from the medical image to be analyzed based on the lymph node detection frame data.
[0053] Specifically, the process of extracting the lymph node regional image refers to obtaining the regional image of the lymph node lesion by cropping the detection frame data of the lymph node on the medical image to be analyzed. For example, the lymph node lesion can be cropped out from the medical image to be analyzed at 2.5 times the size of the detection frame.
[0054] S230 , performing lymph node segmentation on the lymph node region image to obtain a three-dimensional segmented image of the lymph node.
[0055] It should be noted that the lymph node segmentation for the lymph node region image here is based on the segmentation of the region image where the lymph node lesions have been detected, and the region image is input into the segmentation model to obtain the partition mask data of the lymph node region image containing the detection result. Among them, the segmentation model for performing lymph node segmentation on the lymph node region image can adopt the VB-Net model.
[0056] S240 , determining size data of the lymph nodes based on the three-dimensional segmented image.
[0057] The three-dimensional segmented image of the lymph node may be a three-dimensional segmented image obtained by resampling the lymph node segmentation result to the size of the original image data to be analyzed.
[0058] Specifically, firstly, a certain parameter of the lymph node is selected as a calculation standard for determining the lymph node size data, and then the lymph node size data is calculated according to the calculation standard.
[0059] For example, the size of the cross section of the lymph node can be used as a parameter standard to determine the size data of the lymph node. Based on the three-dimensional segmented image, the segmented area of each cross section is calculated; by comparing the sizes of the segmented areas, the maximum cross section of the lymph node is determined, and the long diameter and short diameter of the lymph node are determined based on the coordinates of the contour points of the maximum cross section as the size data of the lymph node.
[0060] S250 . Determine the coordinates of the center point of the lymph node based on the detection frame data of the lymph node as the position data of the lymph node.
[0061] The center point coordinates of the lymph node may be represented by the center point coordinates of the lymph node detection frame.
[0062] In the above implementation, by determining the location data and size data of the lymph nodes, a data basis is provided for the subsequent evaluation of lymph node metastasis.
[0063] In some embodiments, see Figure 3 , perform lymph node detection on the medical image to be analyzed and obtain the lymph node detection frame data, including:
[0064] S310: Input the medical image to be analyzed into the first detection network to perform lymph node detection and obtain a preliminary detection result.
[0065] For example, when training the first detection network, the threshold used for lymph node detection can be continuously adjusted, and lesion detection can be performed on all training images with lesion annotations until all lesions on the training images are correctly detected. It should be noted that the first detection network is connected to the second detection network.
[0066] S320, inputting the preliminary detection result into the second detection network to perform lymph node detection to obtain an intermediate detection result.
[0067] Among them, this detection process can be completed by a cascade detection strategy. Exemplarily, for the first detection network, all annotated lesion areas in the training image can be used as training positive samples, and other areas outside the remaining annotations can be regarded as negative samples; for the second detection network, the false positive lesion area output by the first detection network can be used as a training negative sample, and the second detection network can be trained in combination with all positive samples. Among them, the false positive lesion area refers to the area where the lesion is detected by the first detection network but is not actually a lymph node lesion.
[0068] S330 , performing false positive filtering based on the intermediate detection results to obtain detection frame data of the lymph node.
[0069] Among them, false positive filtering can be performed by removing false positive classifiers. Specifically, a false positive removal classifier can be added after the second detection network to remove the false positive output of the detector according to the detection results, and obtain the detection box data of the lymph node. Exemplarily, three classification networks can be set based on the bagging idea, and the output lesions of the cascade detector are respectively input into the three classifiers, and the obtained probability values are averaged as the output of the final classifier.
[0070] In the above implementation, lymph node detection is performed on the medical image to be analyzed to obtain the detection frame data of the lymph node. The medical image to be analyzed is passed through two detection networks for lesion detection by adopting a cascade detection strategy. At the same time, the threshold used for lymph node detection is adjusted during training to improve the detection rate of the lymph node lesion location. In addition, a false positive removal classifier is set after the two detection networks to further reduce the false positive of lymph node detection.
[0071] In some embodiments, generating the first category text data and the second category text data according to the partition mask data, and the location data and size data of the lymph node includes: determining the partition result of the lymph node according to the location data and the partition mask data of the lymph node. Generating the first category text data based on the first category text template and the partition result and size data of the lymph node; generating the second category text data based on the second category text template and the partition result and size data of the lymph node.
[0072] Among them, the partitioning result is used to describe the anatomical partition in which the lymph node is located. Specifically, the partitioning result is determined by the label on the lymph node partition segmentation mask where the lymph node location data is located. Exemplarily, the labels of different lymph node partition segmentation masks correspond to different anatomical partitions, and the types of labels may include abdominal lymph node anatomical partitions, pelvic lymph node anatomical partitions, left inguinal lymph node anatomical partitions, and right inguinal lymph node anatomical partitions. If the lymph node partition segmentation mask label where the lymph node detection result of the image to be analyzed is located is an abdominal lymph node anatomical partition, it means that the lymph node lesion detected in the image to be analyzed is located in the abdominal anatomical partition.
[0073] Among them, different categories of text templates are used to construct text data describing the properties of lymph nodes of different properties, which may include size information, location information, primary tumor information, and property description information of lymph nodes. Exemplarily, the first category of text templates may be benign property description text templates used to describe benign lymph nodes, and the first category of text data generated based on the partition results and size data of the lymph nodes may be benign text data; the second category of text templates may be malignant property description text templates used to describe malignant lymph nodes, and the second category of text data generated based on the second category of text templates combined with the partition results and size data of the lymph nodes is malignant text data.
[0074] Please refer to the attached Figure 4 In one embodiment, the first type of text data and the second type of text data are used to perform image-text similarity calculations with the medical image to be analyzed to obtain the property evaluation data of the lymph node. Including:
[0075] S410: Perform feature extraction on the first category of text data to obtain first text features, and perform feature extraction on the second category of text data to obtain second text features.
[0076] The corresponding text features are obtained based on the Text Encoder. Specifically, the Text Encoder can adopt the text transformer model commonly used in NLP.
[0077] S420, extracting features from the medical image to be analyzed to obtain image features, wherein corresponding text features are obtained based on Image Encoder.
[0078] Specifically, the FPN network can be used as the Image Encoder.
[0079] S430: Perform similarity calculations with the first text feature, the second text feature, and the image feature to obtain image-text similarity data.
[0080] Among them, the image-text similarity data can be image-text similarity, which is characterized by image-text similarity; specifically, the CLIP model can be used to perform similarity calculations, and output matching similarity data between lymph node image features and different text features. Exemplarily, the CLIP model includes an image encoder and a text encoder. The image encoder is used to extract features of the medical image to be analyzed to obtain image features. The text encoder is used to extract features of the first type of text data to obtain first text features, and the text encoder is used to extract features of the second type of text data to obtain second text features. Based on the extracted image features and the first type of text features and the second text features, image-text pair matching calculations are performed respectively to obtain their image-text similarity.
[0081] S440, generating lymph node property assessment data based on the image and text similarity data.
[0082] Specifically, by determining which type of text data the image-text similarity data is closer to, it can be determined which type of lymph node described by the property description text template the lymph node displayed by the medical image to be analyzed is more likely to be. Exemplarily, if the image-text similarity probability value obtained by similarity calculation between the features of the medical image to be analyzed and the benign text features extracted from the benign text data is closer to 1, the possibility of a metastatic lymph node is smaller, and the possibility of a benign nodule is greater.
[0083] In this embodiment, the property evaluation data of the lymph nodes is generated based on the similarity data of the lymph node images and texts, so as to make a qualitative judgment on the lymph nodes and output the metastasis probability value of the lymph nodes.
[0084] This specification also provides a medical image processing method. Figure 5 , the method comprises the following steps:
[0085] S502: Input the medical image to be analyzed into a region of interest segmentation network for region segmentation, and obtain partition mask data corresponding to the region of interest.
[0086] The region of interest includes multiple anatomical partitions. The region of interest segmentation network includes a first resolution segmentation network and a second resolution segmentation network connected in series. Specifically, the medical image to be analyzed is input into the first resolution segmentation network to obtain the overall mask data of the region of interest, and the overall mask data and the medical image to be analyzed are input into the second resolution segmentation network to obtain the partition mask data of each of the multiple anatomical partitions.
[0087] S504: Input the medical image to be analyzed into the first detection network to perform lymph node detection and obtain a preliminary detection result.
[0088] Wherein, the first detection network is connected to the second detection network.
[0089] S506: Input the preliminary detection result into the second detection network to perform lymph node detection to obtain an intermediate detection result.
[0090] S508 , performing false positive filtering based on the intermediate detection results to obtain detection frame data of the lymph node.
[0091] S510 , extracting a lymph node region image from the medical image to be analyzed based on the lymph node detection frame data.
[0092] S512: Perform lymph node segmentation on the lymph node region image to obtain a three-dimensional segmented image of the lymph node.
[0093] S514. Determine the maximum cross section of the lymph node based on the three-dimensional segmentation image.
[0094] S516. Determine the major axis and minor axis of the lymph node based on the coordinates of the contour points of the largest cross section as the size data of the lymph node.
[0095] S518. Determine the coordinates of the center point of the lymph node based on the detection frame data of the lymph node as the position data of the lymph node.
[0096] S520 : Generate first-category text data and second-category text data according to the partition mask data, and the location data and size data of the lymph nodes.
[0097] Specifically, the partition result of the lymph node is determined according to the location data of the lymph node and the partition mask data. The partition result is used to describe the anatomical partition in which the lymph node is located. The first category of text data is generated based on the first category of text templates and the partition result and size data of the lymph node; the second category of text data is generated based on the second category of text templates and the partition result and size data of the lymph node.
[0098] S522, extracting features from the first category of text data to obtain first text features; and extracting features from the second category of text data to obtain second text features.
[0099] S524: Extract features from the medical image to be analyzed to obtain image features.
[0100] S526: Use the first text feature, the second text feature and the image feature to perform similarity calculations respectively to obtain image-text similarity data.
[0101] S528. Generate lymph node property assessment data based on the image and text similarity data.
[0102] It should be noted that the lymph node property assessment data, lymph node location data and size data can be provided to clinicians, and the lymph node location data and size data can also be provided to clinicians to improve the accuracy of lymph node metastasis assessment.
[0103] It should be understood that, although the various steps in the above flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above flowchart may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0104] The present specification also provides a medical image processing device 600, such as Figure 6 As shown, it includes: a region of interest segmentation module 610, a lymph node detection segmentation module 620, a graphic data generation module 630 and a lymph node qualitative module 640, wherein:
[0105] The region of interest segmentation module 610 is used to perform region segmentation on the medical image to be analyzed and obtain partition mask data corresponding to the region of interest;
[0106] A lymph node detection and segmentation module 620 is used to perform lymph node detection and lymph node segmentation based on the medical image to be analyzed, and obtain location data and size data of the lymph nodes;
[0107] A graphic data generating module 630 is used to generate first-category text data and second-category text data according to the partition mask data and the location data and size data of the lymph nodes;
[0108] The lymph node qualitative module 640 is used to use the first type of text data and the second type of text data to perform image-text similarity calculations on the medical image to be analyzed, so as to obtain lymph node property evaluation data.
[0109] In some embodiments, the region of interest includes multiple anatomical partitions, and the region of interest segmentation module 610 is further used to input the medical image to be analyzed into the first resolution segmentation network to obtain the overall mask data of the region of interest. The first resolution segmentation network is connected to the second resolution segmentation network. The overall mask data and the medical image to be analyzed are input into the second resolution segmentation network to obtain the partition mask data of each of the multiple anatomical partitions.
[0110] In some embodiments, the lymph node detection and segmentation module 620 is further used to perform lymph node detection on the medical image to be analyzed to obtain lymph node detection frame data. A lymph node region image is extracted from the medical image to be analyzed based on the lymph node detection frame data, and lymph node segmentation is performed on the lymph node region image to obtain a three-dimensional segmented image of the lymph node, and size data of the lymph node is determined based on the three-dimensional segmented image. The coordinates of the center point of the lymph node are determined based on the lymph node detection frame data as the position data of the lymph node.
[0111] In some embodiments, the lymph node detection and segmentation module 620 is further used to determine the maximum cross-section of the lymph node based on the three-dimensional segmentation image, and determine the long axis and short axis of the lymph node based on the coordinates of the contour points of the maximum cross-section as the size data of the lymph node.
[0112] In some embodiments, the lymph node detection and segmentation module 620 is further used to input the medical image to be analyzed into the first detection network for lymph node detection to obtain a preliminary detection result. The first detection network is connected to a second detection network, and the preliminary detection result is input into the second detection network for lymph node detection to obtain an intermediate detection result. False positive filtering is performed based on the intermediate detection result to obtain the detection frame data of the lymph node.
[0113] In some embodiments, the region of interest includes multiple anatomical partitions, and the graphic data generation module 630 is further used to determine the partition result of the lymph node according to the location data of the lymph node and the partition mask data. The partition result is used to describe the anatomical partition in which the lymph node is located, and the first type of text data is generated based on the first type of text template and the partition result and size data of the lymph node; the second type of text data is generated based on the second type of text template and the partition result and size data of the lymph node.
[0114] In some embodiments, the region of interest includes multiple anatomical partitions, and the lymph node qualitative module 640 is further used to extract features from the first type of text data to obtain a first text feature, and extract features from the second type of text data to obtain a second text feature. Feature extraction is performed on the medical image to be analyzed to obtain image features. The first text feature and the second text feature are used to perform similarity calculations with the image feature to obtain image-text similarity data; and the property evaluation data of the lymph node is generated according to the image-text similarity data.
[0115] For the specific definition of a medical image processing device, please refer to the definition of a medical image processing method above, which will not be repeated here. Each module in the above medical image processing device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.
[0116] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a medical image processing method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.
[0117] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0118] The present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any of the above methods are implemented.
[0119] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.
[0120] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0121] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.
[0122] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0123] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.
Claims
1. A medical image processing method, characterized in that: The method comprises: Performing regional segmentation on the medical image to be analyzed to obtain partition mask data corresponding to the region of interest; Performing lymph node detection and lymph node segmentation based on the medical image to be analyzed to obtain position data and size data of the lymph nodes; Generate first-category text data and second-category text data according to the partition mask data, and the location data and size data of the lymph node; The first category of text data and the second category of text data are used to perform image-text similarity calculations with the medical image to be analyzed, respectively, to obtain the property assessment data of the lymph node.
2. The method according to claim 1, characterized in that The region of interest includes a plurality of anatomical subdivisions; The method of performing region segmentation on the medical image to be analyzed to obtain partition mask data corresponding to the region of interest includes: Inputting the medical image to be analyzed into a first resolution segmentation network to obtain overall mask data of the region of interest; wherein the first resolution segmentation network is connected to a second resolution segmentation network; The overall mask data and the medical image to be analyzed are input into the second resolution segmentation network to obtain the partition mask data of each of the multiple anatomical partitions.
3. The method according to claim 1, characterized in that The lymph node detection and lymph node segmentation are performed based on the medical image to be analyzed to obtain the position data and size data of the lymph nodes, including: Performing lymph node detection on the medical image to be analyzed to obtain detection frame data of the lymph nodes; Extracting a lymph node region image from the medical image to be analyzed based on the detection frame data of the lymph node; Performing lymph node segmentation on the lymph node region image to obtain a three-dimensional segmented image of the lymph node; determining size data of the lymph node based on the three-dimensional segmented image; The coordinates of the center point of the lymph node are determined based on the detection frame data of the lymph node as the position data of the lymph node.
4. The method according to claim 3, characterized in that The step of determining the size data of the lymph node based on the three-dimensional segmented image comprises: determining a maximum cross-section of the lymph node based on the three-dimensional segmented image; The major axis and the minor axis of the lymph node are determined based on the coordinates of the contour points of the maximum cross section as the size data of the lymph node.
5. The method according to claim 3, characterized in that: The performing lymph node detection on the medical image to be analyzed to obtain the detection frame data of the lymph node includes: Inputting the medical image to be analyzed into a first detection network to perform lymph node detection to obtain a preliminary detection result; wherein the first detection network is connected to a second detection network; Inputting the preliminary detection result into the second detection network to perform lymph node detection to obtain an intermediate detection result; False positive filtering is performed based on the intermediate detection result to obtain the detection frame data of the lymph node.
6. The method according to any one of claims 1 to 5, characterized in that: The region of interest includes a plurality of anatomical partitions; the first type of text data and the second type of text data are generated according to the partition mask data, and the position data and size data of the lymph node, including: Determining a partition result of the lymph node according to the location data of the lymph node and the partition mask data; wherein the partition result is used to describe the anatomical partition where the lymph node is located; Generate the first category of text data based on the first category of text template and the partition result and size data of the lymph node; The second category text data is generated based on the second category text template and the partitioning result and size data of the lymph node.
7. The method according to claim 6, characterized in that The using the first type of text data and the second type of text data to respectively perform image-text similarity calculation with the medical image to be analyzed to obtain the property evaluation data of the lymph node includes: Performing feature extraction on the first category of text data to obtain first text features, and performing feature extraction on the second category of text data to obtain second text features; Extracting features of the medical image to be analyzed to obtain image features; Using the first text feature, the second text feature and the image feature to perform similarity calculations respectively, to obtain image-text similarity data; The property evaluation data of the lymph node is generated according to the graphic and text similarity data.
8. A medical image processing device, characterized in that: The device comprises: A region of interest segmentation module is used to perform region segmentation on the medical image to be analyzed and obtain partition mask data corresponding to the region of interest; A lymph node detection and segmentation module, used to perform lymph node detection and lymph node segmentation based on the medical image to be analyzed, and obtain position data and size data of the lymph nodes; A graphic data generating module, used for generating first-category text data and second-category text data according to the partition mask data, and the location data and size data of the lymph nodes; The lymph node qualitative module is used to use the first category of text data, the second category of text data and the medical image to be analyzed to perform image-text similarity calculation respectively, so as to obtain the property evaluation data of the lymph node.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.
