Three-dimensional focus segmentation method and device, electronic equipment and storage medium

By using the preset lesion segmentation model to use non-medical imaging training data and feature extraction technology, the segmentation of three-dimensional lesions is automated, and the problem of high labor cost of three-dimensional lesion segmentation is solved, achieving efficient and accurate lesion recognition.

CN118918135AInactive Publication Date: 2024-11-08瀚依科技(杭州)有限公司 +1

Patent Information

Application Number
CN202411405790.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-10
Publication Date
2024-11-08
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The labor cost is high during the segmentation process of three-dimensional lesions, mainly because a large number of three-dimensional medical images are required for labeling and training models.

Method used

The preset lesion segmentation model is adopted. The model is trained through training data of non-medical images, and segmented using the pending 3D medical images and reference 3D medical images. Automatic segmentation is achieved through feature extraction, similarity matrix calculation and interactive segmentation model. , reduce labor costs.

Benefits of technology

The training of the model is done with medical images without a large number of three-dimensional lesions, which significantly reduces the labor cost of three-dimensional lesions segmentation and improves segmentation efficiency and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918135A_ABST
    Figure CN118918135A_ABST
Patent Text Reader

Abstract

The invention discloses a three-dimensional focus segmentation method and device, electronic equipment and a storage medium, and relates to the technical field of data processing, and the main technical scheme comprises the steps: obtaining a to-be-processed three-dimensional medical image, and obtaining a reference three-dimensional medical image; the reference three-dimensional medical image comprises a reference three-dimensional focus; respectively inputting the to-be-processed three-dimensional medical image and the reference three-dimensional medical image into a preset focus segmentation model to obtain a segmentation result of the to-be-processed three-dimensional medical image and a target similarity between the segmentation result and the reference three-dimensional focus; and if the target similarity is greater than a preset threshold, determining that the segmentation result is a target three-dimensional lesion, and storing the target three-dimensional lesion. Compared with the related technology, the embodiment of the invention does not need to use a large number of training three-dimensional medical images including the three-dimensional lesion to train the preset lesion segmentation model, so that the segmentation efficiency is improved. And the labor cost of segmentation of the three-dimensional focus is relatively low.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a three-dimensional lesion segmentation method and device, electronic equipment and storage medium. Background Art

[0002] A lesion is an abnormal part of the body where a disease occurs. The segmentation of a three-dimensional lesion refers to the process of accurately identifying and segmenting a three-dimensional lesion in a three-dimensional medical image.

[0003] In the related technologies of three-dimensional lesion segmentation, usually only three-dimensional medical images are input into the lesion segmentation model to obtain the segmentation results of the three-dimensional lesions. In order to enable the lesion segmentation model to accurately identify and segment the three-dimensional lesions, it is necessary to use a large number of training three-dimensional medical images containing three-dimensional lesions to train the lesion segmentation model. The training three-dimensional medical images need to be manually labeled with three-dimensional lesions, resulting in high manual costs for the segmentation of three-dimensional lesions. Summary of the invention

[0004] The present invention provides a three-dimensional lesion segmentation method and device, electronic equipment and storage medium, which mainly aims to solve the problem of high labor cost in three-dimensional lesion segmentation.

[0005] According to a first aspect of the present disclosure, a three-dimensional lesion segmentation method is provided, comprising: Acquire a three-dimensional medical image to be processed, and acquire a reference three-dimensional medical image; the reference three-dimensional medical image includes a reference three-dimensional lesion; Inputting the to-be-processed three-dimensional medical image and the reference three-dimensional medical image into a preset lesion segmentation model respectively, obtaining a segmentation result of the to-be-processed three-dimensional medical image and a target similarity between the segmentation result and the reference three-dimensional lesion; the preset lesion segmentation model is a model trained using training data of non-medical images, and the preset lesion segmentation model is used to segment images similar to the reference three-dimensional lesions in the to-be-processed three-dimensional medical image; If the target similarity is greater than a preset threshold, the segmentation result is determined to be a target three-dimensional lesion, and the target three-dimensional lesion is stored.

[0006] Optionally, before the to-be-processed three-dimensional medical image and the reference three-dimensional medical image are respectively input into a preset lesion segmentation model, the method further includes: Performing two-dimensional processing on the three-dimensional medical image to be processed to obtain at least one first two-dimensional medical image, and performing two-dimensional processing on the reference three-dimensional medical image to obtain at least one second two-dimensional medical image; Stacking the at least one first two-dimensional medical image to obtain a first stacked medical image, and stacking the at least one second two-dimensional medical image to obtain a second stacked medical image; The step of inputting the to-be-processed three-dimensional medical image and the reference three-dimensional medical image into a preset lesion segmentation model comprises: The first stacked medical images and the second stacked medical images are respectively input into the preset lesion segmentation model.

[0007] Optionally, the step of inputting the to-be-processed three-dimensional medical image and the reference three-dimensional medical image into a preset lesion segmentation model respectively to obtain a segmentation result of the to-be-processed three-dimensional medical image and a target similarity between the segmentation result and the reference three-dimensional lesion comprises: Inputting the first stacked medical images and the second stacked medical images into a preset feature extraction model in the preset lesion segmentation model respectively, to obtain a first feature vector of each first two-dimensional medical image in the first stacked medical images and a second feature vector of each second two-dimensional medical image in the second stacked medical images; Calling a preset similarity matrix algorithm to calculate the first eigenvector and the second eigenvector to obtain a target similarity matrix; Inputting the target similarity matrix into the preset upper and lower layer tracking models in the preset lesion segmentation model to obtain a target feature vector and the target similarity; The target feature vector is input into a preset interactive segmentation model in the preset lesion segmentation model to obtain the segmentation result.

[0008] Optionally, calling a preset similarity matrix algorithm to calculate the first eigenvector and the second eigenvector to obtain a target similarity matrix includes: Performing similarity matrix calculation on each first eigenvector and each second eigenvector respectively to obtain at least two similarity matrices; The at least two similarity matrices are concatenated to obtain the target similarity matrix.

[0009] Optionally, the step of inputting the target similarity matrix into the preset upper and lower layer tracking models in the preset lesion segmentation model to obtain the target feature vector and the target similarity comprises: Obtaining a feature position of the first feature vector in the target similarity matrix; Calculating the similarity between the first feature vector and the lesion feature vector to obtain the similarity between the first feature vector and the lesion feature vector; the lesion feature vector is the feature vector of the reference three-dimensional lesion in the second feature vector; Searching for a target feature position corresponding to a target first feature vector in the feature position; the target first feature vector is a first feature vector corresponding to a maximum value in the similarity; Searching for at least one feature position adjacent to the target feature position in the target similarity matrix, and determining a first feature vector corresponding to the at least one feature position and the target first feature vector as a target feature vector; The target similarity is obtained by calculating the average value of the similarity of the first feature vector corresponding to the at least one feature position and the similarity corresponding to the target first feature vector.

[0010] Optionally, after the to-be-processed three-dimensional medical image and the reference three-dimensional medical image are respectively input into a preset lesion segmentation model, the method further includes: If the target similarity is less than or equal to the preset threshold, it is determined that the segmentation result is not the target three-dimensional lesion, and the segmentation result is deleted.

[0011] According to a second aspect of the present disclosure, a three-dimensional lesion segmentation device is provided, comprising: An acquisition unit, used for acquiring a to-be-processed three-dimensional medical image and a reference three-dimensional medical image; the reference three-dimensional medical image includes a reference three-dimensional lesion; An input unit is used to input the three-dimensional medical image to be processed and the reference three-dimensional medical image into a preset lesion segmentation model respectively, to obtain a segmentation result of the three-dimensional medical image to be processed and a target similarity between the segmentation result and the reference three-dimensional lesion; the preset lesion segmentation model is a model trained using training data of non-medical images, and the preset lesion segmentation model is used to segment images similar to the reference three-dimensional lesions in the three-dimensional medical image to be processed; The storage unit is used to determine that the segmentation result is a target three-dimensional lesion when the target similarity is greater than a preset threshold, and store the target three-dimensional lesion.

[0012] Optionally, the device further comprises: a processing unit, configured to perform two-dimensional processing on the three-dimensional medical image to be processed to obtain at least one first two-dimensional medical image, and perform two-dimensional processing on the reference three-dimensional medical image to obtain at least one second two-dimensional medical image before respectively inputting the three-dimensional medical image to be processed and the reference three-dimensional medical image into a preset lesion segmentation model; A stacking unit, configured to stack the at least one first two-dimensional medical image to obtain a first stacked medical image, and to stack the at least one second two-dimensional medical image to obtain a second stacked medical image; The input unit is further used to input the first stacked medical image and the second stacked medical image into the preset lesion segmentation model respectively.

[0013] Optionally, the input unit includes: A first input module, used to input the first stacked medical images and the second stacked medical images into a preset feature extraction model in the preset lesion segmentation model, respectively, to obtain a first feature vector of each first two-dimensional medical image in the first stacked medical images and a second feature vector of each second two-dimensional medical image in the second stacked medical images; A calculation module, used for calling a preset similarity matrix algorithm, calculating the first eigenvector and the second eigenvector to obtain a target similarity matrix; A second input module is used to input the target similarity matrix into the preset upper and lower layer tracking models in the preset lesion segmentation model to obtain a target feature vector and the target similarity; The third input module is used to input the target feature vector into the preset interactive segmentation model in the preset lesion segmentation model to obtain the segmentation result.

[0014] Optionally, the calculation module is further used to: Performing similarity matrix calculation on each first eigenvector and each second eigenvector respectively to obtain at least two similarity matrices; The at least two similarity matrices are concatenated to obtain the target similarity matrix.

[0015] Optionally, the second input module is further used to: Obtaining a feature position of the first feature vector in the target similarity matrix; Calculating the similarity between the first feature vector and the lesion feature vector to obtain the similarity between the first feature vector and the lesion feature vector; the lesion feature vector is the feature vector of the reference three-dimensional lesion in the second feature vector; Searching for a target feature position corresponding to a target first feature vector in the feature position; the target first feature vector is a first feature vector corresponding to a maximum value in the similarity; Searching for at least one feature position adjacent to the target feature position in the target similarity matrix, and determining a first feature vector corresponding to the at least one feature position and the target first feature vector as a target feature vector; The target similarity is obtained by calculating the average value of the similarity of the first feature vector corresponding to the at least one feature position and the similarity corresponding to the target first feature vector.

[0016] Optionally, the device further comprises: A deleting unit is used to, after the to-be-processed three-dimensional medical image and the reference three-dimensional medical image are respectively input into a preset lesion segmentation model, determine that the segmentation result is not the target three-dimensional lesion when the target similarity is less than or equal to the preset threshold, and delete the segmentation result.

[0017] According to a third aspect of the present disclosure, there is provided an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.

[0018] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the first aspect.

[0019] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method as described in the first aspect above.

[0020] The present disclosure provides a method and device for segmenting three-dimensional lesions, an electronic device and a storage medium, which obtain a three-dimensional medical image to be processed and a reference three-dimensional medical image; the reference three-dimensional medical image contains a reference three-dimensional lesion; the three-dimensional medical image to be processed and the reference three-dimensional medical image are respectively input into a preset lesion segmentation model to obtain a segmentation result of the three-dimensional medical image to be processed and a target similarity between the segmentation result and the reference three-dimensional lesion; the preset lesion segmentation model is a model trained using training data of non-medical images, and the preset lesion segmentation model is used to segment images similar to the reference three-dimensional lesion in the three-dimensional medical image to be processed; if the target similarity is greater than a preset threshold, the segmentation result is determined to be a target three-dimensional lesion, and the target three-dimensional lesion is stored. Compared with the related art, the embodiment of the present disclosure does not need to use a large number of training three-dimensional medical images containing three-dimensional lesions to train the preset lesion segmentation model, so that the labor cost of segmenting the three-dimensional lesion is low.

[0021] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure. Figure 1 A schematic flow chart of a three-dimensional lesion segmentation method provided in an embodiment of the present disclosure; Figure 2 A schematic diagram of a process for converting three-dimensional data into two-dimensional data provided by an embodiment of the present disclosure; Figure 3 A schematic diagram of a processing process of a preset lesion segmentation model provided in an embodiment of the present disclosure; Figure 4 A schematic diagram of the structure of a three-dimensional lesion segmentation device provided in an embodiment of the present disclosure; Figure 5 A schematic diagram of the structure of another three-dimensional lesion segmentation device provided in an embodiment of the present disclosure; Figure 6 A schematic block diagram of an exemplary electronic device provided for an embodiment of the present disclosure. DETAILED DESCRIPTION

[0023] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0024] The following describes a three-dimensional lesion segmentation method and apparatus, electronic device, and storage medium according to embodiments of the present disclosure with reference to the accompanying drawings.

[0025] Figure 1 A schematic flow chart of a three-dimensional lesion segmentation method provided in an embodiment of the present disclosure.

[0026] like Figure 1 As shown, the method is applied in a server, and the method comprises the following steps: Step 101, obtaining a to-be-processed three-dimensional medical image and a reference three-dimensional medical image; the reference three-dimensional medical image includes a reference three-dimensional lesion.

[0027] The three-dimensional medical image to be processed is a three-dimensional medical image that needs to be processed for lesion segmentation. The three-dimensional medical image to be processed is obtained by three-dimensional modeling of the human body through electronic computed tomography (CT) or nuclear magnetic resonance imaging (NMRI). However, it should be clear that this statement is not intended to limit the three-dimensional medical image to be processed to be obtained only through CT or NMRI, and the three-dimensional medical image to be processed can also be obtained through other methods.

[0028] A reference 3D medical image refers to a 3D medical image used as a comparison or reference. A reference 3D medical image usually contains known, labeled pathological structures (such as tumors, inflammation, etc.) and is used as a reference for segmenting similar structures in the image to be processed. The pathological results in the reference 3D medical image are manually annotated.

[0029] The reference 3D lesion is a manually annotated pathological region (such as a tumor, lesion, etc.) in a reference 3D medical image.

[0030] The reference 3D lesion in the reference 3D medical image is a known pathological structure, which may be a tumor, inflammation or other lesions. By comparing it with the 3D medical image to be processed, it is easier to identify whether there is a similar lesion in the image to be processed.

[0031] Step 102, input the three-dimensional medical image to be processed and the reference three-dimensional medical image into a preset lesion segmentation model respectively, and obtain the segmentation result of the three-dimensional medical image to be processed and the target similarity between the segmentation result and the reference three-dimensional lesion; the preset lesion segmentation model is a model trained using training data of non-medical images, and the preset lesion segmentation model is used to segment images similar to the reference three-dimensional lesions in the three-dimensional medical image to be processed.

[0032] The preset lesion segmentation model is a model obtained by training using non-medical image data. The preset lesion segmentation model is used to segment images similar to reference three-dimensional lesions in the three-dimensional medical images to be processed; the training data for non-medical images are image data in non-medical fields used for training the preset lesion segmentation model. The training process of the preset lesion segmentation model is to input two partially similar training data into the initial lesion segmentation model respectively. The initial lesion segmentation model learns the two partially similar training data to obtain a loss function, wherein the loss function includes the weights of the initial lesion segmentation model. The weights are adjusted in a manner that makes the loss function approach the minimum value to obtain the preset lesion segmentation model.

[0033] The segmentation result refers to the result obtained after the preset lesion segmentation model segments the three-dimensional medical image to be processed. The preset segmentation model identifies the area in the three-dimensional medical image to be processed that is similar to the reference three-dimensional lesion and segments the area. The target similarity is a quantitative indicator used to measure the similarity between the segmentation result and the reference three-dimensional lesion. It is usually obtained by calculating the differences in features such as shape, size, and position between the two segmentation results and the reference three-dimensional lesion. The higher the target similarity, the more similar the segmentation result is to the reference three-dimensional lesion.

[0034] The preset lesion segmentation model is trained to identify areas in the 3D medical image to be processed that are similar to the reference 3D lesions. The preset lesion segmentation model can automatically complete the segmentation and identification of lesions, reduce the tediousness and subjectivity of manual operations, and improve the efficiency and accuracy of 3D lesion segmentation.

[0035] Step 103: If the target similarity is greater than a preset threshold, the segmentation result is determined to be a target three-dimensional lesion, and the target three-dimensional lesion is stored.

[0036] The segmentation result of the preset lesion segmentation model is not necessarily the target three-dimensional lesion. Only when the target similarity is greater than a preset threshold can the segmentation result be determined to be the target three-dimensional lesion. The preset threshold can be any value, and the preset threshold can be set specifically according to actual business needs. The embodiment of the present disclosure does not limit the specific value of the preset threshold.

[0037] By identifying the segmentation results with target similarity greater than a preset threshold as target three-dimensional lesions and storing them, doctors can be assisted in making more accurate diagnoses, formulating more effective treatment plans, and facilitating subsequent disease tracking, teaching, and research work.

[0038] The present invention provides a method for segmenting three-dimensional lesions, which comprises obtaining a three-dimensional medical image to be processed and obtaining a reference three-dimensional medical image; the reference three-dimensional medical image comprises a reference three-dimensional lesion; the three-dimensional medical image to be processed and the reference three-dimensional medical image are respectively input into a preset lesion segmentation model to obtain a segmentation result of the three-dimensional medical image to be processed and a target similarity between the segmentation result and the reference three-dimensional lesion; the preset lesion segmentation model is a model trained using training data of non-medical images, and the preset lesion segmentation model is used to segment images similar to the reference three-dimensional lesion in the three-dimensional medical image to be processed; if the target similarity is greater than a preset threshold, the segmentation result is determined to be a target three-dimensional lesion, and the target three-dimensional lesion is stored. Compared with the related art, the embodiment of the present invention does not need to use a large number of training three-dimensional medical images containing three-dimensional lesions to train the preset lesion segmentation model, so that the labor cost of segmenting the three-dimensional lesion is low.

[0039] In practical applications, before the to-be-processed 3D medical image and the reference 3D medical image are respectively input into the preset lesion segmentation model, the preset lesion segmentation model can only process 2D data. Since the to-be-processed 3D medical image and the reference 3D medical image are 3D data, the to-be-processed 3D medical image and the reference 3D medical image need to be processed into 2D data, which can be achieved by but not limited to the following methods: Figure 2 As shown, Figure 2 A schematic flow chart of a method for converting three-dimensional data into two-dimensional data provided by an embodiment of the present disclosure includes: Step 201 : performing two-dimensional processing on the three-dimensional medical image to be processed to obtain at least one first two-dimensional medical image, and performing two-dimensional processing on the reference three-dimensional medical image to obtain at least one second two-dimensional medical image.

[0040] The method for performing two-dimensional processing on the three-dimensional medical image to be processed or the reference three-dimensional medical image can be to slice the three-dimensional medical image to be processed or the reference three-dimensional medical image along one dimension direction, thereby obtaining a series of two-dimensional slices, i.e., a two-dimensional medical image. However, it should be clear that this statement is not intended to limit the two-dimensional processing method to only slicing, and the three-dimensional medical image to be processed or the reference three-dimensional medical image can also be processed in two dimensions by other methods.

[0041] Since the structure of two-dimensional medical images is simpler than that of three-dimensional medical images, converting the three-dimensional medical images to be processed into two-dimensional medical images can simplify the data structure and improve the processing efficiency of the preset lesion segmentation model.

[0042] Step 202: stack the at least one first two-dimensional medical image to obtain a first stacked medical image, and stack the at least one second two-dimensional medical image to obtain a second stacked medical image.

[0043] The stacked medical images are a collection of all two-dimensional medical images. Before stacking, it is necessary to determine along which dimension to stack. To stack multiple two-dimensional medical images along the first dimension of the three-dimensional medical image (the first dimension is any of the three dimensions of the three-dimensional medical image), it is necessary to consider stacking along the channel dimension, height dimension, or width dimension of the three-dimensional medical image, depending on the shape of the three-dimensional medical image and the neural network architecture used.

[0044] Stacking medical images actually increases the dimensionality and richness of medical images. By combining multiple two-dimensional medical images, more information in the three-dimensional medical images to be processed can be captured, thereby enhancing the expressive power of medical images.

[0045] Step 203, inputting the to-be-processed three-dimensional medical image and the reference three-dimensional medical image into a preset lesion segmentation model respectively comprises: The first stacked medical images and the second stacked medical images are respectively input into the preset lesion segmentation model.

[0046] Converting three-dimensional medical images to two-dimensional images and stacking them, and then inputting the stacked two-dimensional medical images into a preset lesion segmentation model, can reduce computational complexity, utilize mature two-dimensional image processing technology, improve the accuracy of lesion segmentation, and adapt to the input requirements of the preset lesion segmentation model.

[0047] As a refinement of step 102, when executing the step of inputting the three-dimensional medical image to be processed and the reference three-dimensional medical image into a preset lesion segmentation model respectively to obtain the segmentation result of the three-dimensional medical image to be processed and the target similarity between the segmentation result and the reference three-dimensional lesion, it can be implemented in but not limited to the following manner: inputting the first stacked medical image and the second stacked medical image into a preset feature extraction model in the preset lesion segmentation model respectively to obtain a first feature vector of each first two-dimensional medical image in the first stacked medical image and a second feature vector of each second two-dimensional medical image in the second stacked medical image; calling a preset similarity matrix algorithm to calculate the first feature vector and the second feature vector to obtain a target similarity matrix; inputting the target similarity matrix into a preset upper and lower layer tracking model in the preset lesion segmentation model to obtain a target feature vector and the target similarity; inputting the target feature vector into a preset interactive segmentation model in the preset lesion segmentation model to obtain the segmentation result.

[0048] Exemplarily, the preset lesion segmentation model consists of three sub-models, namely, a preset feature extraction model, a preset context tracking model and a preset interactive segmentation model, wherein the preset feature extraction model is a part of the preset lesion segmentation model, which is responsible for extracting meaningful features from the input medical image. For image analysis, a feature is usually some quantifiable attribute of the image, such as shape, texture, color, etc. The preset feature extraction model uses one or more algorithms (such as convolutional neural networks, autoencoders, etc.) to automatically extract these features from medical images and represent them as feature vectors. A feature vector is a numerical array used to represent features in an image or an image area. In image processing, each pixel or pixel block may be associated with a feature vector, which contains various feature values ​​of the pixel or pixel block.

[0049] The preset similarity matrix algorithm is an algorithm used to calculate the similarity between two sets of feature vectors. The preset similarity matrix algorithm accepts two sets of feature vectors as input and outputs a similarity matrix, each element of which represents the similarity between the two feature vectors.

[0050] The calculation formula of the preset similarity matrix algorithm can be implemented by formula (1): (1) in, is the similarity matrix between the first eigenvector and the second eigenvector, is the first eigenvector, is the transposed vector of the second eigenvector, is the cosine.

[0051] The target similarity matrix is ​​the result calculated by the preset similarity matrix algorithm, which represents the similarity between the first eigenvector set and the second eigenvector set. Each element of the matrix represents the similarity between a pair of eigenvectors, and these similarity values ​​can provide useful information for subsequent analysis or processing.

[0052] The preset upper and lower layer tracking model is used to track the position or change of lesions or regions of interest at different levels in stacked two-dimensional medical images. By using the information provided by the target similarity matrix, the preset upper and lower layer tracking model can determine the relative position or change of lesions at different levels, and may further extract higher-level features related to the target feature vector.

[0053] Preset interactive segmentation model A model or algorithm for image segmentation that may allow a certain degree of user interaction to more accurately segment the region of interest. The model uses the target feature vector as input and may combine interactive information provided by the user (such as labels, contours, etc.) to generate the final segmentation result. Interactive segmentation models can significantly improve the accuracy and efficiency of segmentation, especially when dealing with complex or ambiguous medical images.

[0054] As a refinement of the above embodiment, when executing the calling preset similarity matrix algorithm, calculating the first eigenvector and the second eigenvector to obtain the target similarity matrix, it can be implemented in the following manner but not limited to: performing similarity matrix calculation on each first eigenvector and each second eigenvector respectively to obtain at least two similarity matrices; splicing the at least two similarity matrices to obtain the target similarity matrix. By calculating and splicing similarity matrices, the correlation between different features can be found, which helps to retain the most important information in the dimensionality reduction process or to display the complex relationship between data in visualization, wherein the calculation formula of the similarity matrix can be implemented by formula (1).

[0055] As a refinement of the above embodiment, when executing the input of the target similarity matrix into the preset upper and lower layer tracking model in the preset lesion segmentation model to obtain the target feature vector and the target similarity, it can be implemented in but not limited to the following manner: obtaining the feature position of the first feature vector in the target similarity matrix; performing similarity calculation on the first feature vector and the lesion feature vector to obtain the similarity between the first feature vector and the lesion feature vector; the lesion feature vector is the feature vector of the reference three-dimensional lesion in the second feature vector; searching for the target feature position corresponding to the target first feature vector in the feature position; the target first feature vector is the first feature vector corresponding to the maximum value in the similarity; searching for at least one feature position adjacent to the target feature position in the target similarity matrix, and determining the first feature vector corresponding to the at least one feature position and the target first feature vector as the target feature vector; calculating the average value of the similarity of the first feature vector corresponding to the at least one feature position and the similarity corresponding to the target first feature vector to obtain the target similarity.

[0056] Exemplarily, the feature position refers to the position or index of the feature vector in the target similarity matrix. The feature position is used to identify the position index of the row or column corresponding to the feature vector in the matrix. The position index is used to find and operate the corresponding feature vector for similarity calculation and other related processing.

[0057] The method for calculating the similarity between the first eigenvector and the lesion eigenvector is to calculate the cosine value of the angle between the first eigenvector and the lesion eigenvector. However, it should be clear that this statement is not intended to limit the method of similarity calculation to only the cosine value of the angle, and the similarity can also be calculated by other methods.

[0058] In the target similarity matrix, by calculating the similarity of the feature vectors, a group of first feature vectors that are most similar to the lesion feature vectors are found, and their average similarity is calculated to assist in the diagnosis of the disease and the identification of the lesion.

[0059] In practical applications, after the to-be-processed three-dimensional medical image and the reference three-dimensional medical image are respectively input into the preset lesion segmentation model, the segmentation result output by the preset lesion segmentation model may not be the target three-dimensional lesion, and there is no need to diagnose and analyze the segmentation result. The segmentation result needs to be deleted, which can be achieved in the following way but not limited to: if the target similarity is less than or equal to the preset threshold, it is determined that the segmentation result is not the target three-dimensional lesion, and the segmentation result is deleted. In medical image processing, computing resources (such as time, memory, etc.) are limited. Deleting the wrong segmentation results can reduce the computing resources required for subsequent processing and improve processing efficiency.

[0060] In one possible implementation of the present disclosure, in order to better understand the processing of the preset lesion segmentation model, as shown in FIG. Figure 3 As shown, Figure 3 A schematic diagram of a processing process of a preset lesion segmentation model provided in an embodiment of the present disclosure, wherein a first stack of medical images of a three-dimensional medical image to be processed and a second stack of medical images of a reference three-dimensional medical image are respectively input into a preset feature extraction model in the preset lesion segmentation model to obtain a first eigenvector of the first stack of medical images and a second eigenvector of the second stack of medical images, a similarity matrix is ​​calculated for the first eigenvector and the second eigenvector to obtain a target similarity matrix, the target similarity matrix is ​​input into a preset upper and lower layer tracking model in the preset lesion segmentation model to obtain a target eigenvector and a target similarity, the target eigenvector is input into a preset interactive segmentation model in the preset lesion segmentation model to obtain a segmentation result.

[0061] In summary, the embodiments of the present disclosure can achieve the following effects: The disclosed embodiments do not need to use a large number of training three-dimensional medical images containing three-dimensional lesions to train the preset lesion segmentation model, so that the labor cost of three-dimensional lesion segmentation is relatively low.

[0062] Corresponding to the above-mentioned three-dimensional lesion segmentation method, the present invention also provides a three-dimensional lesion segmentation device. Since the device embodiment of the present invention corresponds to the above-mentioned method embodiment, details not disclosed in the device embodiment can be referred to the above-mentioned method embodiment, and will not be repeated in the present invention.

[0063] Figure 4 A schematic diagram of the structure of a three-dimensional lesion segmentation device provided in an embodiment of the present disclosure, such as Figure 4 As shown, the device is applied to a server and includes: An acquisition unit 31 is used to acquire a to-be-processed three-dimensional medical image and a reference three-dimensional medical image; the reference three-dimensional medical image includes a reference three-dimensional lesion; An input unit 32 is used to input the three-dimensional medical image to be processed and the reference three-dimensional medical image into a preset lesion segmentation model respectively, to obtain a segmentation result of the three-dimensional medical image to be processed and a target similarity between the segmentation result and the reference three-dimensional lesion; the preset lesion segmentation model is a model trained using training data of non-medical images, and the preset lesion segmentation model is used to segment images similar to the reference three-dimensional lesions in the three-dimensional medical image to be processed; The storage unit 33 is used to determine that the segmentation result is a target three-dimensional lesion when the target similarity is greater than a preset threshold, and store the target three-dimensional lesion.

[0064] The present invention provides a three-dimensional lesion segmentation device, which obtains a three-dimensional medical image to be processed and a reference three-dimensional medical image; the reference three-dimensional medical image contains a reference three-dimensional lesion; the three-dimensional medical image to be processed and the reference three-dimensional medical image are respectively input into a preset lesion segmentation model to obtain a segmentation result of the three-dimensional medical image to be processed and a target similarity between the segmentation result and the reference three-dimensional lesion; the preset lesion segmentation model is a model trained using training data of non-medical images, and the preset lesion segmentation model is used to segment images similar to the reference three-dimensional lesion in the three-dimensional medical image to be processed; if the target similarity is greater than a preset threshold, the segmentation result is determined to be a target three-dimensional lesion, and the target three-dimensional lesion is stored. Compared with the related art, the embodiment of the present invention does not need to use a large number of training three-dimensional medical images containing three-dimensional lesions to train the preset lesion segmentation model, so that the labor cost of segmenting the three-dimensional lesion is low.

[0065] Furthermore, in a possible implementation of this embodiment, Figure 5 As shown, the device also includes: The processing unit 34 is used to perform two-dimensional processing on the three-dimensional medical image to be processed to obtain at least one first two-dimensional medical image, and to perform two-dimensional processing on the reference three-dimensional medical image to obtain at least one second two-dimensional medical image before respectively inputting the three-dimensional medical image to be processed and the reference three-dimensional medical image into a preset lesion segmentation model; A stacking unit 35, configured to stack the at least one first two-dimensional medical image to obtain a first stacked medical image, and to stack the at least one second two-dimensional medical image to obtain a second stacked medical image; The input 32 unit is further used to input the first stacked medical image and the second stacked medical image into the preset lesion segmentation model respectively.

[0066] Furthermore, in a possible implementation of this embodiment, Figure 5As shown, the input unit 32 includes: A first input module 321, used to input the first stacked medical images and the second stacked medical images into a preset feature extraction model in the preset lesion segmentation model, respectively, to obtain a first feature vector of each first two-dimensional medical image in the first stacked medical images and a second feature vector of each second two-dimensional medical image in the second stacked medical images; A calculation module 322, configured to call a preset similarity matrix algorithm to calculate the first eigenvector and the second eigenvector to obtain a target similarity matrix; A second input module 323 is used to input the target similarity matrix into the preset upper and lower layer tracking models in the preset lesion segmentation model to obtain a target feature vector and the target similarity; The third input module 324 is used to input the target feature vector into the preset interactive segmentation model in the preset lesion segmentation model to obtain the segmentation result.

[0067] Furthermore, in a possible implementation of this embodiment, Figure 5 As shown, the calculation module 322 is also used for: Performing similarity matrix calculation on each first eigenvector and each second eigenvector respectively to obtain at least two similarity matrices; The at least two similarity matrices are concatenated to obtain the target similarity matrix.

[0068] Furthermore, in a possible implementation of this embodiment, Figure 5 As shown, the second input module 323 is also used for: Obtaining a feature position of the first feature vector in the target similarity matrix; Calculating the similarity between the first feature vector and the lesion feature vector to obtain the similarity between the first feature vector and the lesion feature vector; the lesion feature vector is the feature vector of the reference three-dimensional lesion in the second feature vector; Searching for a target feature position corresponding to a target first feature vector in the feature position; the target first feature vector is a first feature vector corresponding to a maximum value in the similarity; Searching for at least one feature position adjacent to the target feature position in the target similarity matrix, and determining a first feature vector corresponding to the at least one feature position and the target first feature vector as a target feature vector; The target similarity is obtained by calculating the average value of the similarity of the first feature vector corresponding to the at least one feature position and the similarity corresponding to the target first feature vector.

[0069] Furthermore, in a possible implementation of this embodiment, Figure 5 As shown, the device also includes: The deletion unit 36 ​​is used to determine that the segmentation result is not the target three-dimensional lesion and delete the segmentation result after the to-be-processed three-dimensional medical image and the reference three-dimensional medical image are respectively input into the preset lesion segmentation model, when the target similarity is less than or equal to the preset threshold.

[0070] It should be noted that the above explanation of the method embodiment is also applicable to the device of the embodiment of the present disclosure, and the principle is the same, which is no longer limited in the embodiment of the present disclosure.

[0071] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0072] Figure 6 A schematic block diagram of an example electronic device 400 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0073] like Figure 6 As shown, the device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 402 or a computer program loaded from a storage unit 408 to a RAM (Random Access Memory) 403. Various programs and data required for the operation of the device 400 can also be stored in the RAM 403. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An I / O (Input / Output) interface 405 is also connected to the bus 404.

[0074] A number of components in the device 400 are connected to the I / O interface 405, including: an input unit 406, such as a keyboard, a mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a disk, an optical disk, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows the device 400 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0075] The computing unit 401 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, CPU (Central Processing Unit), GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 401 performs the various methods and processes described above, such as a segmentation method for a three-dimensional lesion. For example, in some embodiments, the segmentation method for a three-dimensional lesion may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by the computing unit 401, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to execute the aforementioned three-dimensional lesion segmentation method in any other appropriate manner (for example, by means of firmware).

[0076] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a dedicated or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0077] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0078] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage device, magnetic storage device, or any suitable combination of the foregoing.

[0079] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0080] The systems and techniques described herein may be implemented in a computing system that includes a backend component (e.g., as a data server), or a computing system that includes a middleware component (e.g., an application server), or a computing system that includes a frontend component (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0081] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0082] It should be noted that artificial intelligence is a discipline that studies how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), and includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0083] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0084] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A three-dimensional lesion segmentation method, It is characterized in that include: Acquire a three-dimensional medical image to be processed, and acquire a reference three-dimensional medical image; the reference three-dimensional medical image includes a reference three-dimensional lesion; Inputting the to-be-processed three-dimensional medical image and the reference three-dimensional medical image into a preset lesion segmentation model respectively, obtaining a segmentation result of the to-be-processed three-dimensional medical image and a target similarity between the segmentation result and the reference three-dimensional lesion; the preset lesion segmentation model is a model trained using training data of non-medical images, and the preset lesion segmentation model is used to segment images similar to the reference three-dimensional lesions in the to-be-processed three-dimensional medical image; If the target similarity is greater than a preset threshold, the segmentation result is determined to be a target three-dimensional lesion, and the target three-dimensional lesion is stored.

2. The method according to claim 1, It is characterized in that Before the to-be-processed three-dimensional medical image and the reference three-dimensional medical image are respectively input into a preset lesion segmentation model, the method further includes: Performing two-dimensional processing on the three-dimensional medical image to be processed to obtain at least one first two-dimensional medical image, and performing two-dimensional processing on the reference three-dimensional medical image to obtain at least one second two-dimensional medical image; Stacking the at least one first two-dimensional medical image to obtain a first stacked medical image, and stacking the at least one second two-dimensional medical image to obtain a second stacked medical image; The step of inputting the to-be-processed three-dimensional medical image and the reference three-dimensional medical image into a preset lesion segmentation model comprises: The first stacked medical images and the second stacked medical images are respectively input into the preset lesion segmentation model.

3. The method according to claim 2, It is characterized in that The step of inputting the to-be-processed three-dimensional medical image and the reference three-dimensional medical image into a preset lesion segmentation model to obtain a segmentation result of the to-be-processed three-dimensional medical image and a target similarity between the segmentation result and the reference three-dimensional lesion comprises: Inputting the first stacked medical images and the second stacked medical images into a preset feature extraction model in the preset lesion segmentation model respectively, to obtain a first feature vector of each first two-dimensional medical image in the first stacked medical images and a second feature vector of each second two-dimensional medical image in the second stacked medical images; Calling a preset similarity matrix algorithm to calculate the first eigenvector and the second eigenvector to obtain a target similarity matrix; Inputting the target similarity matrix into the preset upper and lower layer tracking models in the preset lesion segmentation model to obtain a target feature vector and the target similarity; The target feature vector is input into a preset interactive segmentation model in the preset lesion segmentation model to obtain the segmentation result.

4. The method according to claim 3, It is characterized in that The calling of a preset similarity matrix algorithm, calculating the first eigenvector and the second eigenvector to obtain a target similarity matrix includes: Performing similarity matrix calculation on each first eigenvector and each second eigenvector respectively to obtain at least two similarity matrices; The at least two similarity matrices are concatenated to obtain the target similarity matrix.

5. The method according to claim 3, It is characterized in that The step of inputting the target similarity matrix into the preset upper and lower layer tracking models in the preset lesion segmentation model to obtain the target feature vector and the target similarity comprises: Obtaining a feature position of the first feature vector in the target similarity matrix; Calculating the similarity between the first feature vector and the lesion feature vector to obtain the similarity between the first feature vector and the lesion feature vector; the lesion feature vector is the feature vector of the reference three-dimensional lesion in the second feature vector; Searching for a target feature position corresponding to a target first feature vector in the feature position; the target first feature vector is a first feature vector corresponding to a maximum value in the similarity; Searching for at least one feature position adjacent to the target feature position in the target similarity matrix, and determining a first feature vector corresponding to the at least one feature position and the target first feature vector as a target feature vector; The target similarity is obtained by calculating the average value of the similarity of the first feature vector corresponding to the at least one feature position and the similarity corresponding to the target first feature vector.

6. The method according to claim 1, It is characterized in that After the to-be-processed three-dimensional medical image and the reference three-dimensional medical image are respectively input into a preset lesion segmentation model, the method further includes: If the target similarity is less than or equal to the preset threshold, it is determined that the segmentation result is not the target three-dimensional lesion, and the segmentation result is deleted.

7. A three-dimensional lesion segmentation device, It is characterized in that include: An acquisition unit, used for acquiring a to-be-processed three-dimensional medical image and a reference three-dimensional medical image; the reference three-dimensional medical image includes a reference three-dimensional lesion; An input unit is used to input the three-dimensional medical image to be processed and the reference three-dimensional medical image into a preset lesion segmentation model respectively, to obtain a segmentation result of the three-dimensional medical image to be processed and a target similarity between the segmentation result and the reference three-dimensional lesion; the preset lesion segmentation model is a model trained using training data of non-medical images, and the preset lesion segmentation model is used to segment images similar to the reference three-dimensional lesions in the three-dimensional medical image to be processed; The storage unit is used to determine that the segmentation result is a target three-dimensional lesion when the target similarity is greater than a preset threshold, and store the target three-dimensional lesion.

8. An electronic device, It is characterized in that include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium storing computer instructions, It is characterized in that The computer instructions are used to cause the computer to execute the method according to any one of claims 1-6.

10. A computer program product, It is characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Citation Information

Patent Citations

  • Three-dimensional medical image labeling method and device, and related product

    CN112053769A

  • Focus area segmentation method and device, electronic equipment and storage medium

    CN118570465A

  • Spatial distribution of pathological image patterns in 3D image data

    US20210090257A1

  • Follow-up case data processing method and apparatus, device, and storage medium

    WO2024041058A1

Cited By

  • Medical image auxiliary analysis method and system based on artificial intelligence

    CN119132524A