Medical image segmentation methods and devices, electronic devices, and storage media

Abstract

This application provides a medical image segmentation method, apparatus, electronic device, and storage medium, relating to the field of artificial intelligence technology. The method includes: cropping a first original slice image according to a first cropping range to obtain a first cropped slice image; performing image segmentation on the first cropped slice image to obtain a first mask image and a first initial segmented image; if the relationship between the mask boundary and the cropping boundary of the first mask image satisfies a first preset condition, cropping the first original slice image according to a second cropping range to obtain a second cropped slice image; performing image segmentation on the second cropped slice image to obtain a second mask image and a first intermediate segmented image; if the relationship between the mask boundary and the cropping boundary of the second mask image satisfies a second preset condition, obtaining the segmentation result of the first original slice image based on the first intermediate segmented image. This embodiment improves the efficiency of image segmentation.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a medical image segmentation method and apparatus, electronic device, and storage medium. Background Technology

[0002] Traditional image segmentation methods often yield inaccurate results. For example, once the parameters of a traditional image segmentation algorithm are determined, the region drawn by the user may be applicable to the current image section but not to other image sections. Related technologies require users to manually adjust the drawn region layer by layer, which is time-consuming and labor-intensive. Summary of the Invention

[0003] The main objective of this application is to propose a medical image segmentation method, apparatus, electronic device, and storage medium, thereby improving the efficiency of image segmentation.

[0004] To achieve the above objectives, a first aspect of this application proposes a medical image segmentation method, the method comprising:

[0005] Acquire medical tissue data, the medical tissue data including a first raw cross-sectional image;

[0006] The first cutting range is determined based on the preset outline area;

[0007] The first original cropped image is cropped according to the first cropping range to obtain the first cropped cropped image;

[0008] The first cropped image is input into a preset target image segmentation model for image segmentation to obtain a first mask image and a first initial segmentation image.

[0009] If the relationship between the mask boundary and the cropping boundary of the first mask image satisfies a first preset condition, then the first initial segmented image is deleted, and the first cropping range is expanded to obtain a second cropping range; wherein, the first preset condition is used to indicate that the mask boundary intersects with the cropping boundary;

[0010] The first original cropped image is cropped according to the second cropping range to obtain the second cropped image.

[0011] The second cropping image is input into the target image segmentation model for image segmentation to obtain a second mask image and a first intermediate segmentation image;

[0012] If the relationship between the mask boundary and the clipping boundary of the second mask image satisfies the second preset condition, then the segmentation result of the first original cross-sectional image is obtained based on the first intermediate segmentation image; wherein, the second preset condition is used to indicate that the mask boundary and the clipping boundary do not intersect.

[0013] In some embodiments, the first mask image includes a first pixel, and the condition that the relationship between the mask boundary and the cropping boundary of the first mask image satisfies a first preset condition includes:

[0014] Obtain the position of each first pixel point located on the mask boundary in a preset coordinate system to obtain a first coordinate set;

[0015] Obtain the position of each first pixel point located on the cropping boundary in the preset coordinate system to obtain the second coordinate set;

[0016] If the first coordinate in the first coordinate set and the second coordinate in the second coordinate set satisfy a first preset coordinate relationship, then the relationship between the mask boundary of the first mask image and the clipping boundary of the first mask image satisfies the first preset condition; wherein, the preset coordinate relationship includes: the horizontal coordinate of the first coordinate and the horizontal coordinate of the second coordinate are the same, or the vertical coordinate of the first coordinate and the vertical coordinate of the second coordinate are the same.

[0017] In some embodiments, the step of expanding the first cutting range to obtain a second cutting range includes:

[0018] The number of first coordinates that satisfy the first preset coordinate relationship is counted to obtain the number of intersection points;

[0019] The first amplification factor is determined based on the number of intersection points;

[0020] The first cutting range is magnified according to the first magnification factor to obtain the second cutting range.

[0021] In some embodiments, before cropping the first original cropped image according to the first cropping range to obtain the first cropped cropped image, the method further includes:

[0022] Obtain the initial pixel value of each pixel in the first original cross-sectional image;

[0023] If the initial pixel value is greater than a preset threshold, the first original cross-sectional image is normalized to obtain the normalized first original cross-sectional image.

[0024] The step of normalizing the first original cross-sectional image to obtain a normalized first original cross-sectional image specifically includes:

[0025] Obtain the initial pixel value of each pixel in the first original cross-sectional image;

[0026] The difference between the largest and smallest initial pixel values ​​is calculated to obtain the reference pixel value;

[0027] The difference between the initial pixel value and the smallest initial pixel value is calculated to obtain the deviation pixel value;

[0028] The standard pixel value is obtained by calculating the ratio between the deviation pixel value and the reference pixel value;

[0029] The initial pixel value is replaced with the standard pixel value to obtain the normalized first original cross-sectional image.

[0030] In some embodiments, the medical tissue data further includes a second original cross-sectional image, and after obtaining the segmentation result of the first original cross-sectional image based on the first intermediate segmentation image, the method further includes:

[0031] Obtain the pixel values ​​of the second mask image to get the mask pixel values;

[0032] If the mask pixel value is less than the preset mask pixel threshold, the second original cross-sectional image is deleted, and the target segmentation result is obtained based on the segmentation result of the first original cross-sectional image.

[0033] If the mask pixel value is greater than or equal to the preset mask pixel threshold, then image segmentation is performed on the second original cross-sectional image to obtain the segmentation result of the second original cross-sectional image; the segmentation result of the first original cross-sectional image and the segmentation result of the second original cross-sectional image are merged to obtain the target segmentation result.

[0034] In some embodiments, determining the first cutting range based on a preset outline area includes:

[0035] Based on the position of the outlined area in the first original cross-sectional image, the topmost outlined position, the bottommost outlined position, the leftmost outlined position, and the rightmost outlined position are extracted.

[0036] Based on the topmost, bottommost, leftmost, and rightmost outlined positions, a graphic is generated to obtain the circumscribed graphic region corresponding to the outlined area.

[0037] The first cutting range is obtained based on the circumscribed graphic region.

[0038] In some embodiments, before inputting the first cropped image into a preset target image segmentation model for image segmentation to obtain a first mask image and a first segmented image, the method further includes:

[0039] The target tissue type is obtained by identifying the tissue type of the first cropped image using a preset image tissue classification model.

[0040] Based on the target tissue type, a target image segmentation model is selected from a set of candidate image segmentation models.

[0041] To achieve the above objectives, a second aspect of this application provides a medical image segmentation apparatus, the apparatus comprising:

[0042] The tissue data acquisition module is used to acquire medical tissue data, which includes a first original cross-sectional image;

[0043] The first cutting range determination module is used to determine the first cutting range based on the preset outline area;

[0044] The first cropping processing module is used to crop the first original cross-sectional image according to the first cropping range to obtain the first cropped cross-sectional image;

[0045] The first image segmentation module is used to input the first cropped surface image into a preset target image segmentation model to perform image segmentation, and obtain a first mask image and a first initial segmentation image;

[0046] The second cropping range determination module is used to delete the first initial segmented image and expand the first cropping range to obtain a second cropping range if the relationship between the mask boundary of the first mask image and the cropping boundary of the first mask image satisfies a first preset condition; wherein, the first preset condition is used to indicate that the mask boundary intersects with the cropping boundary;

[0047] The second cropping processing module is used to crop the first original cropping image according to the second cropping range to obtain a second cropped cropping image;

[0048] The second image segmentation module is used to input the second cropped image into the target image segmentation model to perform image segmentation, and obtain a second mask image and a first intermediate segmentation image;

[0049] The segmentation result determination module is used to obtain the segmentation result of the first original cross-sectional image based on the first intermediate segmentation image if the relationship between the mask boundary and the clipping boundary of the second mask image satisfies a second preset condition; wherein, the second preset condition is used to indicate that the mask boundary and the clipping boundary do not intersect.

[0050] To achieve the above objectives, a third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0051] To achieve the above objectives, a fourth aspect of the present application provides a storage medium, which is a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0052] This application proposes a medical image segmentation method, apparatus, electronic device, and storage medium. The method dynamically adjusts the cropping range by checking whether the mask boundary of the mask image intersects with the cropping boundary. This allows users to select a region of interest in a certain section without manually adjusting the selected area, thereby improving the efficiency of image segmentation. Attached Figure Description

[0053] Figure 1 This is an optional flowchart of the medical image segmentation method provided in the embodiments of this application;

[0054] Figure 2 This is a visualization example of the original image provided in the embodiments of this application;

[0055] Figure 3 This is a visual example of the original image after outlining, provided in an embodiment of this application;

[0056] Figure 4 yes Figure 1 Flowchart for step 102;

[0057] Figure 5 This is a visual example of the first cut surface image provided in the embodiments of this application;

[0058] Figure 6 This is a visualization example of a segmented image provided in an embodiment of this application;

[0059] Figure 7 This is a visual example of the mask image provided in the embodiments of this application;

[0060] Figure 8 This is a schematic diagram of the structure of the image organization classification model provided in the embodiments of this application;

[0061] Figure 9 This is a schematic diagram of the structure of the image segmentation model provided in the embodiments of this application;

[0062] Figure 10This is an example diagram illustrating an application of the medical image segmentation method provided in this application embodiment;

[0063] Figure 11 This is a schematic diagram of the structure of the medical image segmentation device provided in the embodiments of this application;

[0064] Figure 12 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0066] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0067] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0068] First, let's analyze some of the terms used in this application:

[0069] Artificial intelligence (AI) is a new branch of computer science that studies, develops, and applies theories, methods, technologies, and systems to simulate, extend, and expand human intelligence. It aims to understand the essence of intelligence and produce intelligent machines that can react in a way similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thought. Furthermore, AI utilizes digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceiving the environment, acquiring knowledge, and using that knowledge to achieve optimal results.

[0070] Natural Language Processing (NLP): NLP uses computers to process, understand, and utilize human language (such as Chinese and English). It is a branch of artificial intelligence and an interdisciplinary field of computer science and linguistics, often referred to as computational linguistics. NLP includes syntactic analysis, semantic analysis, and discourse understanding. It is commonly used in machine translation, handwritten and printed character recognition, speech recognition and text-to-speech conversion, information and image processing, information extraction and filtering, text classification and clustering, sentiment analysis, and opinion mining. It involves data mining, machine learning, knowledge acquisition, knowledge engineering, artificial intelligence research, and linguistic research related to language computation.

[0071] Computed Tomography (CT): It uses precisely collimated X-ray beams, gamma rays, ultrasound, etc., together with highly sensitive detectors to scan a part of the human body one section after another. It has the characteristics of fast scanning time and clear images, and can be used to examine a variety of diseases. Depending on the type of radiation used, it can be divided into X-ray CT (X-CT) and gamma-ray CT (γ-CT), etc.

[0072] Residual Networks (RDBs) fully utilize all hierarchical features of the original logistic regression (LR) image through Residual Dense Blocks (RDBs). For a very deep network, directly extracting the output of each convolutional layer in the LR space is difficult, if not impractical. Residual Dense Blocks (RDBs) are used as building blocks for Residual Networks (RDNs). RDBs contain densely connected layers and Local Feature Fusion (LFF) with Local Residual Learning (LRL). Residual Dense Blocks also support continuous memory between RDBs. The output of one RDB can directly access the layers of the next RDB, allowing for continuous state propagation. Each convolutional layer of the RDB can access all subsequent layers, passing on the information that needs to be retained. By connecting the previous RDB with the states of all preceding layers of the current RDB, LFF extracts local dense features by adaptively preserving information. Furthermore, LFF achieves extremely high growth rates by stabilizing the training of larger networks. After extracting multiple layers of local dense features, Global Feature Fusion (GFF) is further performed to adaptively preserve hierarchical features globally. Each layer can directly access the original LR input, resulting in implicit deep supervised learning. Residual Networks are characterized by ease of optimization and the ability to improve accuracy by increasing depth considerably. Its internal residual blocks use skip connections to alleviate the vanishing gradient problem caused by increasing depth in deep neural networks. A residual network consists of a series of residual blocks. Each residual block is divided into two parts: a direct mapping part and a residual part. The residual part typically consists of two or three convolutional operations.

[0073] Backpropagation: The general principle of backpropagation is as follows: The training data is input into the input layer of the neural network, passes through the hidden layer, and finally reaches the output layer of the neural network to output the result; Since there is an error between the output result of the neural network and the actual result, the error between the estimated value and the actual value is calculated, and this error is backpropagated from the output layer to the hidden layer until it reaches the input layer; During the backpropagation process, the values ​​of various parameters are adjusted according to the error; The above process is iterated continuously until convergence.

[0074] Image segmentation is the technique and process of dividing an image into several specific regions with unique properties and extracting targets of interest. It is a key step from image processing to image analysis. Existing image segmentation methods can be mainly divided into the following categories: threshold-based segmentation methods, region-based segmentation methods, edge-based segmentation methods, and segmentation methods based on specific theories. From a mathematical perspective, image segmentation is the process of dividing a digital image into non-overlapping regions. The image segmentation process is also a labeling process, that is, assigning the same number to pixels belonging to the same region.

[0075] Three-dimensional optical imaging technology mainly consists of three core components: first, the transparency treatment of the sample; second, the microscope instrument itself capable of depth scanning, such as confocal microscopes, light-sheet microscopes, and two-photon microscopes; and third, the post-processing and visualization of the three-dimensional data. Based on this, combined with deep learning, the extremely high resolution and more complete biological information provided by 3D reconstruction have made 3D technology a popular imaging method. Different transparency treatment methods, microscopes, and data visualization algorithms can be selected according to the characteristics of different animal tissues and specific imaging requirements.

[0076] For the post-processing and visualization stages of 3D data, traditional 3D delineation and calibration software is generally used for 3D reconstruction of CT images, but not for delineating 3D pathological tissue images. Common delineation and calibration software applied to 3D reconstruction models of CT images includes MITK, ITK-SNAP, and 3DSlicer. Among these, multi-plane automatic delineation algorithms are basically traditional image segmentation algorithms based on thresholding or region growing. They select a region at a certain cross-section and set parameters such as thresholds or growth points, then apply these parameters to the remaining planes. For example, the delineation range is obtained from user interaction, and a trained model is used to identify and segment the feature sections within that range. However, different feature sections require different delineation ranges, leading to a decrease in image segmentation accuracy.

[0077] The delineation methods based on traditional image segmentation algorithms often yield inaccurate results, heavily reliant on image quality. If the image itself contains strong noise, the delineation result will deviate significantly from the ideal outcome. Secondly, once the parameters of traditional image algorithms are determined, it is difficult to adaptively adjust them for different cross-sections. Therefore, parameters that perfectly match the user's chosen cross-section may not be applicable to others, requiring manual adjustment layer by layer, which is time-consuming and laborious. Furthermore, the edges of the delineated regions produced by traditional algorithms often exhibit jagged edges and noise, affecting user experience and subsequent parameter calculations.

[0078] To address the aforementioned problems, the purpose of this application is to provide a medical image segmentation method that can segment three-dimensional pathological (medical) tissue images after (pseudo)H&E staining to obtain segmentation results. This allows for a simpler and faster way to locate and delineate feature tissues at the three-dimensional tissue level, assisting in subsequent parameter measurement and diagnosis.

[0079] The medical image segmentation method in this application can be executed by a server alone, by a terminal alone, or by both a terminal and a server. Furthermore, the medical image segmentation method provided in this application can also be software running on a server. The server can be configured as a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The software can be an application that implements the medical image segmentation method, but is not limited to the above forms.

[0080] The medical image segmentation method of this application can be used in a variety of general-purpose or special-purpose computer system environments or configurations. Examples include: server computers, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, distributed computing environments including any of the above systems or devices, etc. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can reside in local and remote computer storage media, including storage devices.

[0081] This application provides a medical image segmentation method, apparatus, electronic device, and computer-readable storage medium, which are specifically described through the following embodiments. First, the medical image segmentation method in the embodiments of this application is described.

[0082] See Figure 1 A medical image segmentation method according to one embodiment of this application includes:

[0083] Step 101: Obtain medical tissue data, which includes a first original cross-sectional image;

[0084] Step 102: Determine the first cutting range based on the preset outline area;

[0085] Step 103: Crop the first original cross-sectional image according to the first cropping range to obtain the first cropped cross-sectional image;

[0086] Step 104: Input the first cropping image into the preset target image segmentation model to perform image segmentation, and obtain the first mask image and the first initial segmentation image;

[0087] Step 105: If the relationship between the mask boundary and the cropping boundary of the first mask image satisfies the first preset condition, then delete the first initial segmented image and expand the first cropping range to obtain the second cropping range; wherein, the first preset condition is used to indicate that the mask boundary and the cropping boundary intersect.

[0088] Step 106: Crop the first original cropping image according to the second cropping range to obtain the second cropping image;

[0089] Step 107: Input the second cropping image into the target image segmentation model to perform image segmentation, and obtain the second mask image and the first intermediate segmentation image;

[0090] Step 108: If the relationship between the mask boundary and the clipping boundary of the second mask image satisfies the second preset condition, then the segmentation result of the first original cross-sectional image is obtained based on the first intermediate segmentation image; wherein, the second preset condition is used to indicate that the mask boundary and the clipping boundary do not intersect.

[0091] Through steps 101-108, this embodiment can more accurately assist users in delineating the feature surfaces of three-dimensional pathological data, avoiding repeated adjustments to the feature area range on different cross sections, improving the efficiency and accuracy of pathological diagnosis, providing support for the calculation of subsequent stereometric parameters, and making the pathological diagnosis process faster, less labor-intensive, and more accurate.

[0092] Steps 101-108 are described in detail below.

[0093] In step 101, medical tissue data is acquired. This medical tissue data can be three-dimensional medical tissue data, including images and videos.

[0094] If the medical tissue data includes a first original cross-sectional image, then the first original cross-sectional image is an image in the medical tissue image data, or the first original cross-sectional image is a frame in the medical tissue video data.

[0095] There are several ways to obtain medical tissue data. For example, images of the target tissue can be acquired using various image acquisition devices (such as CT scanners) to obtain medical tissue data. Alternatively, medical tissue data can be obtained from local or external databases, or by searching for medical tissue data on the internet.

[0096] Furthermore, the text describes how to obtain medical tissue data:

[0097] Medical data can be acquired by receiving medical data sent by the terminal through the server's network interface. For example, medical tissue data can be sent by the terminal to the server via a pre-defined link requesting a medical image segmentation service, i.e., the terminal sending a medical image segmentation request.

[0098] Another way to obtain medical tissue data is for the server to access a preset storage location and read the medical tissue data from that location. For example, the medical tissue data can be pre-stored in a preset storage location on the server. When performing medical image segmentation, the server reads the medical tissue data from the preset storage location to obtain the medical tissue data.

[0099] To reduce the uncertainty in subsequent image segmentation, the images in the medical tissue data need to be normalized after acquisition. Specifically, in one embodiment, the initial pixel value of each pixel in the first original cross-sectional image is obtained; if the initial pixel value is greater than a preset threshold, the first original cross-sectional image is normalized to obtain a normalized first original cross-sectional image.

[0100] The step of normalizing the first original cross-sectional image to obtain a normalized first original cross-sectional image specifically includes:

[0101] Obtain the initial pixel value of each pixel in the first original cross-sectional image;

[0102] The base pixel value is obtained by calculating the difference between the largest and smallest initial pixel values.

[0103] The difference between the initial pixel value and the smallest initial pixel value is calculated to obtain the deviation pixel value;

[0104] The standard pixel value is obtained by calculating the ratio between the deviation pixel value and the reference pixel value;

[0105] The initial pixel values ​​are replaced with standard pixel values ​​to obtain the normalized first original cross-sectional image.

[0106] In step 102, the first cutting range is determined according to the preset outline area.

[0107] Before explaining the delineated region, let's first explain what a region of interest (ROI) is. ROI: In machine vision and image processing, the region to be processed from the image is delineated using shapes such as rectangles, circles, ellipses, or irregular polygons; this is called the ROI.

[0108] The delineated region, similar to the region of interest, is used to indicate the selected area in the image to be processed. Please refer to [link / reference]. Figure 2 , Figure 3 . Figure 2 It can represent the original image, or it can represent the first original cross-sectional image or the second original cross-sectional image. Figure 3 This indicates that a region has been selected in the original image. After selecting the region, the first cropping area can be determined based on the selected region.

[0109] A detailed description of how to obtain the outlined area:

[0110] The outlined area can be obtained in two ways: either by receiving the outlined area sent by the terminal through the server's network interface, or by the server accessing a preset storage location and reading the outlined area from that location.

[0111] The following describes how to determine the first cropping area based on the outlined region:

[0112] One way to determine the first cropping range based on the outlined area is to define the outlined range based on the outlined area and use the outlined range as the first cropping range.

[0113] To reduce the impact of irregularities in the delineated region on image segmentation, another way to determine the first cropping range based on the delineated region is to determine the circumscribed region based on the delineated region and then determine the first cropping range based on the circumscribed region.

[0114] Reference Figure 4 In one embodiment, step 102 specifically includes:

[0115] Step 201: Based on the position of the outlined area in the first original cross-sectional image, extract the top outlined position, bottom outlined position, left outlined position and right outlined position.

[0116] Step 202: Generate a graphic based on the top, bottom, left, and right outline positions to obtain the outer graphic region corresponding to the outlined area.

[0117] Step 203: Obtain the first cutting range based on the circumscribed graphic region.

[0118] In step 201, the outlined area is located at the lower right corner of the first original cross-sectional image. A rectangular coordinate system can be established from this system to obtain the uppermost outlined position (x1, y1), the lowermost outlined position (x2, y2), the leftmost outlined position (x3, y3), and the rightmost outlined position (x4, y4).

[0119] In step 202, the circumscribed graphic region can be a rectangle, a circle, or an ellipse.

[0120] Taking the bounding region as a rectangle as an example, the first side length (y2-y1) of the rectangle is determined by the ordinate of the top and bottom lines, and the second side length (x2-x1) is determined by the x-coordinate of the left and right lines. The bounding rectangle region corresponding to the bounding region is obtained based on the first and second side lengths.

[0121] In step 203, the target outlining range is determined based on the circumscribed graphic region, and the target outlining range is used as the first cropping range.

[0122] Through the above steps, the first cutting range is obtained, and cutting will be performed according to the first cutting range.

[0123] In step 103, the first original cut surface image is cropped according to the first cropping range to obtain the first cropped cut surface image.

[0124] The first original cross-sectional image is as follows: Figure 2 As shown, the first cropped image obtained after cropping is as follows: Figure 5 As shown.

[0125] In step 104, the first cropping image is input into a preset target image segmentation model for image segmentation to obtain a first mask image and a first initial segmentation image.

[0126] The target image segmentation model is a model that segments the first cropped image to obtain the first segmentation result, such as the U-Net model. Here, the first segmentation result refers to the first mask image and the first initial segmented image.

[0127] Before explaining the first mask image, let's first explain what a mask is. From a physics perspective, in semiconductor manufacturing, many chip process steps use photolithography. The patterned "subject" used in these steps is called a mask. Its function is to cover a selected area on the silicon wafer with an opaque pattern template, so that subsequent etching or diffusion will only affect areas outside the selected area.

[0128] Similar to masking images, masking images use selected images, graphics, or objects to occlude (all or part) the image being processed, thereby controlling the area or process of image processing.

[0129] Specifically, in this embodiment, the image region of the first cropped section image includes tissue regions and irrelevant regions. After image segmentation, the tissue regions in the first cropped section image will obtain a first initial segmented image, such as... Figure 6 As shown, the first cropped image after removing the tissue region can be used to obtain the first mask image, as shown. Figure 7 As shown.

[0130] The target image segmentation model in step 104 can be a target image segmentation model sent by the terminal via the server's network interface. Alternatively, the server can access a preset storage location and read the target image segmentation model from that location.

[0131] For a target image segmentation model read from a preset storage location, it can specifically include:

[0132] The target tissue type is obtained by identifying the tissue type of the first cropped image using a preset image tissue classification model.

[0133] The target image segmentation model is selected from multiple pre-set candidate image segmentation models based on the target tissue type.

[0134] Since obtaining the target image segmentation model eliminates the need for extensive manual annotation of tissue types in medical tissue data, significant manpower costs can be saved. It should be noted that the tissue types described in this application embodiment can be used to indicate the organ in the cross-sectional image, such as the heart, kidney, lung, and liver. The tissue types described in this application embodiment can also be used to indicate the organ cells in the cross-sectional image, such as heart cells, kidney cells, lung cells, and liver cells. The specific tissue type settings can be configured according to actual needs, ensuring that different types of cross-sectional images can be matched with different image segmentation models.

[0135] It is also understandable that classifying first and then segmenting can improve the targeting of image segmentation models, thereby improving the accuracy of feature surface recognition and segmentation.

[0136] It should be noted that image segmentation models and image organization classification information can be deep learning models or traditional non-deep learning models. Non-deep learning models include decision trees, random forests, support vector machines, etc., and even image segmentation models can use simple region growing algorithms.

[0137] Next, we will describe the image organization classification model.

[0138] The image tissue classification model can be a traditional classification model, such as the ResNet network model. In this embodiment, the ResNet network model includes an input layer, multiple hidden layers, and an output layer. The input layer has multiple processing nodes, each with a weight matrix (convolution kernel). Each hidden layer also has multiple processing nodes, each with a weight matrix (convolution kernel). Each processing node in the input layer convolves the input vector of the ResNet network model with its own convolution kernel to obtain its output, which serves as the input to the processing nodes of the next hidden layer. Each processing node in the next hidden layer convolves the output of each processing node in the previous layer with its own convolutional layer, and obtains its own output based on the convolution result and the output of each processing node in the previous layer, which serves as the input to the processing nodes of the next-next hidden layer. This process continues until each processing node in the output layer convolves the output of each processing node in the last hidden layer with its own convolutional layer to obtain the target output vector, thereby determining the target tissue type.

[0139] In addition to using ResNet, this application proposes an image organization classification model that can further improve the accuracy of image organization classification. For example... Figure 8 As shown, in this embodiment, the image organization classification model includes multiple sequentially connected processing layers 301 and fully connected layers 302. It should be noted that each processing layer 301 is equivalent to a residual block. Each residual block convolves the input vector with its own convolution kernel and merges the convolution result with the input vector to obtain the output of that residual block, which serves as the input to the residual block of the next layer.

[0140] Therefore, the first processing layer 301 receives the vector corresponding to the first cropped image. Each processing layer 301 outputs the processed result and the unprocessed input together to the next processing layer 301 as input. The last processing layer 301 outputs the processed result to the fully connected layer 302, which generates the target output vector, thereby obtaining the target tissue type.

[0141] Each processing layer 301 includes multiple processing nodes 311 and a linear rectified function (ReLU) 312.

[0142] Processing node 311 is the same as the processing nodes in ResNet, performing a convolution of the input vector or the output vector of the previous processing layer 301 with its own convolution kernel.

[0143] The Rectified Linear Function (ReLU) is an activation function. The convolution operation at node 311 is essentially a linear operation. To balance computational simplicity and model flexibility, the model employs a combination of linear operations at node 311 and a non-linear transformation of the activation function. ReLU is a piecewise linear function; if the input is positive, it outputs directly; otherwise, it outputs zero. Its advantages include easier model training and generally better performance.

[0144] The fully connected layer 302 maps the output vectors of the preceding layers to a target output vector of a specified dimension. The image organization and classification model has multiple processing layers 301, each with multiple processing nodes 311, producing vectors of enormous dimensionality. However, the target output vector has a specified dimension. The fully connected layer 302 maps these enormous dimensional vectors from the preceding layers to vectors of a specified dimension. The fully connected layer 302 includes multiple fully connected layer nodes 321. Each fully connected layer node 321 performs convolution operations and also has a convolution kernel. Each fully connected layer node 321 convolves the output vector of the preceding layer with its own convolution kernel. For example... Figure 8 As shown, a pooling layer is required before entering the fully connected layer.

[0145] Next, the image segmentation model will be described in detail.

[0146] The image segmentation model can be a traditional segmentation model, such as the U-Net network model. In this embodiment, the U-Net network model includes an input layer, multiple hidden layers, and an output layer. The input layer has multiple processing nodes, each with a weight matrix (convolution kernel). Each hidden layer also has multiple processing nodes, each with a weight matrix (convolution kernel). Each processing node in the input layer convolves the input vector of the U-Net network model with its own convolution kernel to obtain the output of that processing node, which is then used as the input to the processing nodes of the next hidden layer. Each processing node in the next hidden layer convolves the output of each processing node in the previous layer with its own convolutional layer to obtain the output of that processing node, which is then used as the input to the processing nodes of the next-next hidden layer. This process continues until each processing node in the output layer convolves the output of each processing node in the last hidden layer with its own convolutional layer to obtain the target output feature matrix, thus obtaining the background image and the target image.

[0147] Then, the first initial segmentation image of this application can be obtained from the target image, and the first mask image of this application can be obtained from the background image. Alternatively, the first intermediate segmentation image of this application can be obtained from the target image, and the second mask image of this application can be obtained from the background image.

[0148] In addition to using U-Net, this application proposes an image segmentation model that can further improve the accuracy of image segmentation.

[0149] Reference Figure 9 In this embodiment, the image segmentation model includes multiple first processing layers 401, multiple second processing layers 402, and an output layer 403 connected sequentially. The first first processing layer 401 receives the vector corresponding to the first cropped image. Each first processing layer outputs its processed result to the next first processing layer as input. The last first processing layer 401 outputs its processed result to the first second processing layer 402. Each second processing layer 402 outputs its processed result to the next second processing layer 402 as input. The last second processing layer 402 outputs its processed result to the output layer 403, which generates a target output feature matrix, thereby obtaining the background image and the target image.

[0150] The first processing layer 401 downsamples the input vector. Specifically, the first processing layer includes multiple processing nodes 411, a Rectified Activation Function (ReLU) 412, and a dimensionality reduction module 413. The second processing layer 402 upsamples the input vector. Specifically, the second processing layer 402 includes multiple processing nodes 421, a Rectified Activation Function (ReLU) 422, and a dimensionality increase module 433. The output layer 403 upsamples the input vector. Specifically, the output layer 403 includes multiple processing nodes 431 and 432.

[0151] In this embodiment, each processing node 411, processing node 421, processing node 431 and processing node 432 is used to perform convolution of the input vector or the output vector of the previous processing layer with its own convolution kernel.

[0152] The dimensionality reduction module can specifically be a pooling module, which performs pooling processing on the input vector. Pooling, also known as pooling operation, is a very common operation in neural network models. Pooling layers mimic the human visual system to reduce the dimensionality of data. Pooling operations are often called subsampling or downsampling. When building convolutional neural networks, it is often used after convolutional layers to reduce the feature dimension of the convolutional layer output, effectively reducing network parameters while preventing overfitting.

[0153] The dimension-increasing module can specifically be an unpooling module, which is the reverse of the pooling module and is used to increase the feature dimension of the convolutional layer output.

[0154] It should be noted that in the output layer, two 1×1 convolutional kernels can be selected as processing nodes 432 to convert the feature channels into two. The processing of the output layer is equivalent to a binary classification operation, which divides the image into two categories: background and target, thereby achieving image segmentation.

[0155] Next, we will describe how to train the image organization classification model and the image segmentation model.

[0156] First, a brief description of the training process.

[0157] Training of the image tissue classification model:

[0158] (1) Create a dataset for training the image organization classification model based on the pre-labeled types;

[0159] (2) Normalize the RGB values ​​of the images in the dataset to the [0,1] interval, and enhance the data by means of translation, rotation, symmetry, etc. to obtain the final dataset, and divide it into training set, validation set and test set;

[0160] (3) Construct and train the model: Construct a ResNet network model for feature extraction from the preprocessed images. Input the training set and validation set into the ResNet model for training and optimization, and use the test set to validate the model results to obtain a trained pathological tissue classification model;

[0161] (4) Save the model as the classification module of the overall algorithm.

[0162] Segmentation model training module:

[0163] (1) Generate a mask image based on the outlined area, with the inside of the outlined area marked as 1 and the outside marked as 0;

[0164] (2) Normalize the RGB values ​​of the images in the dataset to the [0,1] interval, match the mask image, generate datasets of various tissue types according to the classification results, and enhance the data by translation, rotation, symmetry and other methods to obtain the final dataset, and divide it into training set, validation set and test set;

[0165] (3) Construct and train the model: Construct the U-Net network model, input the training set and validation set into the U-Net model for training and optimization, and use the test set to verify the model results to obtain the trained feature surface segmentation model;

[0166] (4) Save the model as the segmentation module of the overall algorithm.

[0167] Furthermore, the training process is described in detail.

[0168] Training an image organization classification model specifically includes:

[0169] Acquire three-dimensional tissue data;

[0170] Based on the cross-section, the three-dimensional tissue data is segmented to obtain the original cross-sectional image of the first sample, and the sample reference tissue type of the original cross-sectional image of the first sample is obtained.

[0171] The original cross-sectional image of the sample is input into the image tissue classification model for image classification to obtain the tissue type of the sample.

[0172] Based on the comparison between the generated tissue type and the reference tissue type, a first error function is constructed, and the parameters of the image tissue classification model are adjusted based on the first error function.

[0173] It should be noted that gradient descent can be used to backpropagate the first error function, feeding it back into the initial model to modify the model parameters of the image tissue classification model. This process is repeated until the first error function meets a preset iteration condition. This preset iteration condition is either reaching a preset number of iterations or the variance of the loss function being less than a preset threshold. When the first error function value meets the preset iteration condition, backpropagation can be stopped, and the final model parameters can be used as the final model parameters, ceasing updates to the image tissue classification model.

[0174] Training the candidate image segmentation model specifically includes:

[0175] Obtain the original cross-sectional image of the second sample and the sample tissue type of the original medical image of the second sample;

[0176] Based on the preset sample delineation area, a mask is generated from the original cross-sectional image of the second sample to obtain the sample mask image;

[0177] The sample cropping range is determined based on the sample outlined area, and the original cross-sectional image of the second sample is cropped according to the sample cropping range to obtain the sample cropped medical image;

[0178] The sample cropped medical image is normalized, and the normalized sample cropped medical image and the sample mask image are multiplied to obtain the sample reference segmented medical image.

[0179] The cropped medical image of the sample is input into the candidate image segmentation model matched with the tissue type of the sample for image segmentation to obtain the segmented medical image of the sample.

[0180] Based on the comparison between the sample-generated segmented medical image and the sample reference segmented medical image, a second error function is constructed, and the parameters of the candidate image segmentation model matched with the sample tissue type are adjusted based on the second error function.

[0181] It is understandable that adjusting the parameters of the candidate image segmentation model matched by the sample tissue type based on the second error function is similar to the processing of the image tissue classification model described above, and will not be repeated here.

[0182] After step 104, the first initial segmented image has been obtained, that is, the segmentation result of the first original cross-sectional image has been obtained. However, as... Figure 7 The first mask image shown often results in an unreasonable delineation area, causing the first mask image to be very close to, or even intersect with, the first cropping area. In this case, the resulting first initial segmentation image often lacks some organizational information. Therefore, it is necessary to determine whether the mask boundary of the first mask image intersects with the cropping boundary of the first mask image.

[0183] In step 105, if the relationship between the mask boundary and the clipping boundary of the first mask image satisfies a first preset condition, the first preset condition is used to indicate that the mask boundary and the clipping boundary intersect. If there is an intersection, the first initial segmented image is deleted, and the first clipping range is expanded to obtain a second clipping range, so that the first original cut surface can be re-cut according to the second clipping range.

[0184] For example, if the first clipping range is x0*y0 pixels, then if any pixel in the set of pixels at the mask boundary falls on any one of the four edges x=0, y=0, x=x0, y=y0, it is considered that the mask boundary has touched the clipping boundary, and the range needs to be expanded for re-detection.

[0185] It should be noted that, in order to reduce the expansion of the first cutting area, it is also necessary to count the number of intersection points.

[0186] Therefore, in another embodiment, step 105 specifically includes:

[0187] If the relationship between the mask boundary and the clipping boundary of the first mask image satisfies the first preset condition, then the number of first coordinates that meet the first preset coordinate relationship is counted to obtain the number of intersection points.

[0188] If the number of intersection points is greater than the preset threshold, the first initial segmented image is deleted, and the first cropping range is expanded to obtain the second cropping range.

[0189] It should be noted that the preset number of points threshold is set according to actual needs. For example, in this implementation, the preset number of points threshold can be 4.

[0190] Furthermore, even if the number of intersection points exceeds a preset threshold, if the intersection points are too scattered, it is not necessary to expand the first cutting area. Therefore, in another embodiment, step 105 specifically includes:

[0191] If the relationship between the mask boundary and the clipping boundary of the first mask image satisfies the first preset condition, then the number of first coordinates that meet the first preset coordinate relationship is counted to obtain the number of intersection points.

[0192] If the number of intersection points is greater than the preset threshold, the number of consecutive intersections is obtained based on the coordinates of the multiple intersection points.

[0193] If the number of consecutively intersecting data exceeds a preset threshold for the number of consecutively intersecting data, the first initial segmented image is deleted, and the first cropping range is expanded to obtain the second cropping range.

[0194] Next, we will describe how to determine the relationship between the mask boundary and the cropping boundary of the first mask image to satisfy the first preset condition:

[0195] In one embodiment, the first mask image includes first pixels, and the position of each first pixel on the mask boundary in a preset coordinate system is obtained to obtain a first coordinate set;

[0196] Obtain the position of each first pixel point on the cropping boundary in the preset coordinate system to obtain the second coordinate set;

[0197] The first coordinate set and the second coordinate set are compared. If the first coordinate in the first coordinate set and the second coordinate in the second coordinate set satisfy the first preset coordinate relationship, then the relationship between the mask boundary of the first mask image and the clipping boundary of the first mask image satisfies the first preset condition. The first preset coordinate relationship includes: the abscissa of the first coordinate and the abscissa of the second coordinate are the same, and the ordinate of the first coordinate and the ordinate of the second coordinate are the same.

[0198] Next, we will describe how to expand the first cutting area to obtain the second cutting area:

[0199] In one embodiment, the first cropping area is enlarged by a preset magnification ratio to obtain the second cropping area. For example, the first cropping area is enlarged by 10% to obtain the second cropping area. Another example is that the outlined area is expanded by 20px towards the enclosed boundary to enlarge the first cropping area and obtain the second cropping area.

[0200] In another embodiment, the number of first coordinates that satisfy the first preset coordinate relationship is counted to obtain the number of intersection points; a first magnification factor is determined based on the number of intersection points; and the first cutting range is magnified based on the first magnification factor to obtain a second cutting range.

[0201] In another embodiment, the number of first coordinates that satisfy the first preset coordinate relationship is counted to obtain the number of intersection points; if the number of intersection points is greater than the preset number of points threshold, the number of consecutive intersections is obtained based on the coordinates of multiple intersection points; a second amplification factor is determined based on the number of consecutive intersections; the first cutting range is amplified based on the first amplification factor and the second amplification factor to obtain the second cutting range.

[0202] It should be noted that the target magnification factor can be obtained by weighting the first magnification factor and the second magnification factor, and then the first cutting range can be magnified according to the target magnification factor.

[0203] In step 106, the first original cross-sectional image is cropped according to the second cropping range to obtain a second cropped cross-sectional image. The processing procedure is similar to step 103, but the difference is that the second cropping range is larger than the first cropping range, so the second cropped cross-sectional image will include more tissue areas compared to the first cropped cross-sectional image.

[0204] In step 107, the second cropped image is input into the target image segmentation model for image segmentation to obtain a second mask image and a first intermediate segmentation image. The processing procedure is similar to step 107, but the difference is that the first intermediate segmentation image will include more tissue regions compared to the first initial segmentation image.

[0205] It is understandable that the mask boundary and the clipping boundary of the second mask image may still intersect, and it is necessary to determine whether the mask boundary and the clipping boundary of the second mask image do not intersect.

[0206] Therefore, in step 108, if the relationship between the mask boundary and the clipping boundary of the second mask image satisfies a second preset condition, the second preset condition indicates that the mask boundary and the clipping boundary do not intersect. If the second preset condition is met, the segmentation result of the first original cross-sectional image is obtained based on the first intermediate segmentation image.

[0207] Next, we will describe how to determine the relationship between the mask boundary and the cropping boundary of the second mask image to satisfy the second preset condition:

[0208] In one embodiment, it can be first determined whether a first preset condition is met, and then whether a second preset condition is met. Step 108 further includes:

[0209] If the relationship between the mask boundary and the cropping boundary of the second mask image satisfies the first preset condition, then the second preset condition is not satisfied.

[0210] If the relationship between the mask boundary and the cropping boundary of the second mask image does not satisfy the first preset condition, then the second preset condition is not satisfied.

[0211] Alternatively, in another embodiment, it is directly determined whether the second preset condition is met. The second mask image includes second pixels, and step 108 further includes:

[0212] Obtain the position of each second pixel point located on the mask boundary in a preset coordinate system to obtain a third coordinate set;

[0213] Obtain the position of each second pixel point on the cropping boundary in the preset coordinate system to obtain the fourth coordinate set;

[0214] The third coordinate set and the fourth coordinate set are compared. If the third coordinate in the third coordinate set and the fourth coordinate in the fourth coordinate set satisfy the second preset coordinate relationship, then the relationship between the mask boundary of the second mask image and the clipping boundary of the second mask image satisfies the second preset condition. The second preset coordinate relationship includes at least one of the following: the abscissa of the first coordinate and the abscissa of the second coordinate are not the same, and the ordinate of the first coordinate and the ordinate of the second coordinate are not the same.

[0215] If the medical tissue image data also includes a second original cross-sectional image, after step 108, the medical image segmentation method of this embodiment further includes:

[0216] The second original cross-sectional image is segmented to obtain the segmentation result of the second original cross-sectional image;

[0217] The segmentation results of the first original cross-sectional image and the second original cross-sectional image are merged to obtain the target segmentation result.

[0218] It should be noted that medical tissue image data corresponds to a specific organ, and some organs are spherical. In such cases, although the medical tissue image data includes multiple cross-sectional images, if the segmentation result corresponding to a certain cross-sectional image is too small, it means that the obtained tissue region is already small enough, and there is no need to perform image segmentation on the next cross-sectional image. This reduces the number of image segmentation operations and further improves the overall image segmentation efficiency.

[0219] Therefore, in one embodiment, step 108 further includes:

[0220] Obtain the pixel values ​​of the second mask image to get the mask pixel values;

[0221] If the mask pixel value is less than the preset mask pixel threshold, the second original cross-section image is deleted, and the target segmentation result is obtained based on the segmentation result of the first original cross-section image.

[0222] If the mask pixel value is greater than or equal to the preset mask pixel threshold, then image segmentation is performed on the second original cross-sectional image to obtain the segmentation result of the second original cross-sectional image. The segmentation result of the first original cross-sectional image and the segmentation result of the second original cross-sectional image are merged to obtain the target segmentation result.

[0223] It should be noted that image segmentation of the second original cross-sectional image specifically includes:

[0224] The second original cropping image is cropped according to the second cropping range to obtain the third cropping image;

[0225] The third cropping image is input into the target image segmentation model for image segmentation to obtain the third mask image and the second initial segmentation image.

[0226] If the relationship between the mask boundary and the cropping boundary of the third mask image satisfies the first preset condition, then the second initial segmentation image is deleted, and the second cropping range is expanded to obtain the third cropping range; the second original cross-sectional image is cropped according to the third cropping range to obtain the fourth cropped cross-sectional image; the fourth cropped cross-sectional image is input into the target image segmentation model for image segmentation to obtain the fourth mask image and the second intermediate segmentation image; if the relationship between the mask boundary and the cropping boundary of the fourth mask image satisfies the second preset condition, then the segmentation result of the second original cross-sectional image is obtained according to the second intermediate segmentation image;

[0227] Alternatively, if the relationship between the mask boundary of the third mask image and the clipping boundary of the third mask image satisfies the second preset condition, then the segmentation result of the second original cross-section image is obtained based on the second initial segmentation image.

[0228] Based on the above method embodiments, refer to Figure 10 In one example, the medical image segmentation method includes:

[0229] (1) Obtain the outlined area of ​​interest through user interaction;

[0230] (2) Expand the user-drawn area of ​​interest into an outer rectangle to obtain the clipping range;

[0231] (3) Crops the original cropped image according to the cropping range to obtain the current cropped image;

[0232] (4) Image standardization processing: normalize the RGB values ​​of the current slice image to the [0,1] interval;

[0233] (5) Apply an image tissue classification model to detect the target tissue type in the current cross-sectional image;

[0234] (6) Apply the image segmentation model corresponding to the target tissue type. The purpose is to segment out the tissue structure of interest, and the segmented image and mask image can be obtained.

[0235] (6) Check if the mask pixel value of the segmented mask image is ≤4. If not, it means that the current section has detected a specific tissue. Otherwise, end the recognition and proceed to step (10).

[0236] (7) Detect whether the mask boundary of the segmented mask image intersects with the clipping boundary. If it does, it means that the clipping range is too small and does not include all the range boundaries. Then proceed to step (8). Otherwise, end the recognition of the current slice and proceed to step (9).

[0237] (8) Expand the cropping area by 20px to the enclosed boundary and return to step (3);

[0238] (9) Output the outline result after the current cross-section adjustment, input the cropping range of the current cross-section image into the next original cross-section image, and return to step (3);

[0239] (10) End recognition and output all cross-section recognition results.

[0240] Please see Figure 11 This application also provides a medical image segmentation apparatus that can implement the above-described medical image segmentation method. Figure 11The present invention provides a block diagram of a medical image segmentation device, comprising: a tissue data acquisition module 501, a first cropping range determination module 502, a first cropping processing module 503, a first image segmentation module 504, a second cropping range determination module 505, a second cropping processing module 506, a second image segmentation module 507, and a segmentation result determination module 508. The system includes: a tissue data acquisition module 501 for acquiring medical tissue data, including a first original cross-sectional image; a first cropping range determination module 502 for determining a first cropping range based on a preset delineation area; a first cropping processing module 503 for cropping the first original cross-sectional image according to the first cropping range to obtain a first cropped cross-sectional image; a first image segmentation module 504 for inputting the first cropped cross-sectional image into a preset target image segmentation model for image segmentation to obtain a first mask image and a first initial segmented image; and a second cropping range determination module 505 for deleting the first initial segmented image if the relationship between the mask boundary and the cropping boundary of the first mask image satisfies a first preset condition. The first cropping range is expanded to obtain a second cropping range; wherein, the first preset condition is used to indicate that the mask boundary and the cropping boundary intersect; the second cropping processing module 506 is used to crop the first original cross-sectional image according to the second cropping range to obtain a second cropped cross-sectional image; the second image segmentation module 507 is used to input the second cropped cross-sectional image into the target image segmentation model for image segmentation to obtain a second mask image and a first intermediate segmented image; the segmentation result determination module 508 is used to obtain the segmentation result of the first original cross-sectional image according to the first intermediate segmented image if the relationship between the mask boundary and the cropping boundary of the second mask image satisfies the second preset condition; wherein, the second preset condition is used to indicate that the mask boundary and the cropping boundary do not intersect.

[0241] It should be noted that the specific implementation of this medical image segmentation device is basically the same as the specific implementation of the above-mentioned medical image segmentation method, and will not be repeated here.

[0242] This application also provides an electronic device, which includes: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for communication between the processor and the memory. When the program is executed by the processor, it implements the aforementioned medical image segmentation method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0243] Please see Figure 12 , Figure 12 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0244] The processor 601 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0245] The memory 602 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 602 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 602 and is called and executed by the processor 601 using the medical image segmentation method of the embodiments of this application.

[0246] The input / output interface 603 is used to implement information input and output;

[0247] The communication interface 604 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0248] Bus 605 transmits information between various components of the device (e.g., processor 601, memory 602, input / output interface 603, and communication interface 604);

[0249] The processor 601, memory 602, input / output interface 603, and communication interface 604 are connected to each other within the device via bus 605.

[0250] This application embodiment also provides a storage medium, which is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more programs, which can be executed by one or more processors to implement the above-described medical image segmentation method.

[0251] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0252] The medical image segmentation method, apparatus, electronic device, and computer-readable storage medium provided in this application dynamically adjust the cropping range by whether the mask boundary and the cropping boundary of the mask image intersect. This allows the user to select the region of interest of a certain section, and the specific features can be automatically identified and applied to all other sections. This solves the problems of slow layer-by-layer annotation speed, low efficiency, and high labor costs in the prior art.

[0253] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0254] It will be understood by those skilled in the art that Figure 1 and Figure 4 The technical solutions shown do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0255] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0256] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0257] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0258] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0259] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0260] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0261] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A medical image segmentation method, characterized in that, The method includes: Acquire medical tissue data, the medical tissue data including a first raw cross-sectional image; The first cutting range is determined based on the preset outline area; The first original cropped image is cropped according to the first cropping range to obtain the first cropped cropped image; The first cropped image is input into a preset target image segmentation model for image segmentation to obtain a first mask image and a first initial segmentation image. If the relationship between the mask boundary and the cropping boundary of the first mask image satisfies a first preset condition, then the first initial segmented image is deleted, and the first cropping range is expanded to obtain a second cropping range; wherein, the first preset condition is used to indicate that the mask boundary intersects with the cropping boundary; The first original cropped image is cropped according to the second cropping range to obtain the second cropped image. The second cropping image is input into the target image segmentation model for image segmentation to obtain a second mask image and a first intermediate segmentation image; If the relationship between the mask boundary and the clipping boundary of the second mask image satisfies the second preset condition, then the segmentation result of the first original cross-sectional image is obtained based on the first intermediate segmentation image; wherein, the second preset condition is used to indicate that the mask boundary and the clipping boundary do not intersect.

2. The method according to claim 1, characterized in that, The first mask image includes a first pixel, and the condition that the relationship between the mask boundary and the cropping boundary of the first mask image satisfies a first preset condition includes: Obtain the position of each first pixel point located on the mask boundary in a preset coordinate system to obtain a first coordinate set; Obtain the position of each first pixel point located on the cropping boundary in the preset coordinate system to obtain the second coordinate set; If the first coordinate in the first coordinate set and the second coordinate in the second coordinate set satisfy a first preset coordinate relationship, then the relationship between the mask boundary of the first mask image and the clipping boundary of the first mask image satisfies the first preset condition; wherein, the preset coordinate relationship includes: the horizontal coordinate of the first coordinate and the horizontal coordinate of the second coordinate are the same, or the vertical coordinate of the first coordinate and the vertical coordinate of the second coordinate are the same.

3. The method according to claim 2, characterized in that, The process of expanding the first cutting range to obtain a second cutting range includes: The number of first coordinates that satisfy the first preset coordinate relationship is counted to obtain the number of intersection points; The first amplification factor is determined based on the number of intersection points; The first cutting range is magnified according to the first magnification factor to obtain the second cutting range.

4. The method according to claim 1, characterized in that, Before cropping the first original cropped image according to the first cropping range to obtain the first cropped cropped image, the method further includes: Obtain the initial pixel value of each pixel in the first original cross-sectional image; If the initial pixel value is greater than a preset threshold, the first original cross-sectional image is normalized to obtain the normalized first original cross-sectional image. The step of normalizing the first original cross-sectional image to obtain a normalized first original cross-sectional image specifically includes: Obtain the initial pixel value of each pixel in the first original cross-sectional image; The difference between the largest and smallest initial pixel values ​​is calculated to obtain the reference pixel value; The difference between the initial pixel value and the smallest initial pixel value is calculated to obtain the deviation pixel value; The standard pixel value is obtained by calculating the ratio between the deviation pixel value and the reference pixel value; The initial pixel value is replaced with the standard pixel value to obtain the normalized first original cross-sectional image.

5. The method according to claim 1, characterized in that, The medical tissue data also includes a second original cross-sectional image. After obtaining the segmentation result of the first original cross-sectional image based on the first intermediate segmentation image, the method further includes: Obtain the pixel values ​​of the second mask image to get the mask pixel values; If the mask pixel value is less than the preset mask pixel threshold, the second original cross-sectional image is deleted, and the target segmentation result is obtained based on the segmentation result of the first original cross-sectional image. If the mask pixel value is greater than or equal to the preset mask pixel threshold, then image segmentation is performed on the second original cross-sectional image to obtain the segmentation result of the second original cross-sectional image; the segmentation result of the first original cross-sectional image and the segmentation result of the second original cross-sectional image are merged to obtain the target segmentation result.

6. The method according to claim 1, characterized in that, The step of determining the first cutting range based on the preset outline area includes: Based on the position of the outlined area in the first original cross-sectional image, the topmost outlined position, the bottommost outlined position, the leftmost outlined position, and the rightmost outlined position are extracted. Based on the topmost, bottommost, leftmost, and rightmost outlined positions, a graphic is generated to obtain the circumscribed graphic region corresponding to the outlined area. The first cutting range is obtained based on the circumscribed graphic region.

7. The method according to claim 1, characterized in that, Before inputting the first cropped image into a preset target image segmentation model for image segmentation to obtain the first mask image and the first segmented image, the method further includes: The target tissue type is obtained by identifying the tissue type of the first cropped image using a preset image tissue classification model. Based on the target tissue type, a target image segmentation model is selected from a set of candidate image segmentation models.

8. A medical image segmentation device, characterized in that, The device includes: The tissue data acquisition module is used to acquire medical tissue data, which includes a first original cross-sectional image; The first cutting range determination module is used to determine the first cutting range based on the preset outline area; The first cropping processing module is used to crop the first original cross-sectional image according to the first cropping range to obtain the first cropped cross-sectional image; The first image segmentation module is used to input the first cropped surface image into a preset target image segmentation model to perform image segmentation, and obtain a first mask image and a first initial segmentation image; The second cropping range determination module is used to delete the first initial segmented image and expand the first cropping range to obtain a second cropping range if the relationship between the mask boundary of the first mask image and the cropping boundary of the first mask image satisfies a first preset condition; wherein, the first preset condition is used to indicate that the mask boundary intersects with the cropping boundary; The second cropping processing module is used to crop the first original cropping image according to the second cropping range to obtain a second cropped cropping image; The second image segmentation module is used to input the second cropped image into the target image segmentation model to perform image segmentation, and obtain a second mask image and a first intermediate segmentation image; The segmentation result determination module is used to obtain the segmentation result of the first original cross-sectional image based on the first intermediate segmentation image if the relationship between the mask boundary and the clipping boundary of the second mask image satisfies a second preset condition; wherein, the second preset condition is used to indicate that the mask boundary and the clipping boundary do not intersect.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images