Image segmentation method, device, equipment and storage medium based on position prior

Abstract

The present application relates to a method, device, computer equipment, storage medium and computer program product for image segmentation based on position prior. The present application can predetermine the position information of the target tissue organ on the entire medical image, extract accurate boundary information based on the position information, and improve the accuracy of segmentation. The method includes: inputting the medical image to be segmented into a first neural network, performing rough segmentation on the medical image through the first neural network, and generating a first mask image for the initial tissue organ obtained by segmentation; the first mask image includes a first extraction area; based on the first extraction area and the neighborhood of the first extraction area, a second extraction area is generated; based on the second extraction area, image extraction is performed on the medical image to obtain a position focused image; the medical image to be segmented, the position focused image and the first mask image are input into a second neural network, and the medical image is finely segmented by the second neural network to obtain the target tissue organ.

Description

Technical Field

[0001] The present application relates to the technical field of medical image processing, and in particular to a method, apparatus, computer equipment, storage medium and computer program product for image segmentation based on position prior. Background Art

[0002] Medical image segmentation refers to the process of dividing an image into several regions based on similar or different features of the regions. Medical image segmentation has a wide range of applications and research value in the field of medical research and practice. For example, for the measurement of the size, volume or volume of human organs, tissues or lesions, it is necessary to quantitatively measure and analyze relevant imaging indicators before and after treatment, which helps doctors diagnose, follow up or revise the treatment plan for patients.

[0003] In traditional medical diagnosis, doctors need to rely on their professional knowledge and experience to judge the imaging indicators of the target organ, but this method is greatly affected by the doctor's subjectivity, such as cognitive differences or fatigue, resulting in different judgments from person to person. With the continuous development of artificial intelligence, a medical image segmentation method based on neural networks has emerged, which can automatically distinguish and segment the required organ images.

[0004] However, due to the complexity of the human anatomical structure, existing intelligent segmentation methods are not applicable to various human tissues. For example, for small organs or complex organs such as lymph nodes, optic chiasm, etc., the segmentation effect is not accurate enough. Summary of the invention

[0005] Based on this, it is necessary to provide an image segmentation method, apparatus, computer device and computer-readable storage medium based on position prior to address the above technical problems.

[0006] In a first aspect, the present application provides an image segmentation method based on position prior. The method comprises:

[0007] Inputting a medical image to be segmented into a pre-trained first neural network, roughly segmenting the medical image through the pre-trained first neural network, and generating a first mask image for initial tissues and organs obtained by segmentation; the first mask image includes a first extraction region for extracting the initial tissues and organs in the medical image;

[0008] generating a second extraction region based on the first extraction region and a neighborhood of the first extraction region;

[0009] Performing image extraction on the medical image based on the second extraction area to obtain a position focused image;

[0010] The medical image to be segmented, the position focused image and the first mask image are input into a pre-trained second neural network, and the pre-trained second neural network is used to perform fine segmentation on the medical image to obtain the target tissue organ in the medical image.

[0011] In one embodiment, generating a second extraction area based on the first extraction area and a neighborhood of the first extraction area includes:

[0012] In the first mask image, obtaining the centroid of the first extraction area;

[0013] Taking the centroid as the center point, a polygonal area is formed around the center point with a radius of a preset length, and an image including the polygonal area is used as the second mask image.

[0014] In one embodiment, taking the centroid as the center point, forming a polygonal area around the center point with a radius of a preset length, and using the image containing the polygonal area as the second mask image, comprises:

[0015] In the case where there are multiple centroids, each of the centroids is taken as a center point, and multiple polygonal areas are formed around the center point by a radius of the preset length;

[0016] An intersection area of ​​the plurality of polygonal areas is obtained, and an image including the intersection area is used as the second mask image.

[0017] In one embodiment, performing image extraction on the medical image based on the second extraction area to obtain a position-focused image includes:

[0018] Setting the second extraction area to a preset pixel value to generate a second mask image;

[0019] The second mask image is multiplied by the medical image to be segmented to generate the position focused image.

[0020] In one embodiment, before inputting the medical image to be segmented into the pre-trained first neural network, the method further includes:

[0021] Acquire a training image to be segmented and a label image of the training image, wherein the label image identifies the tissue and organ to be segmented in the training image;

[0022] The training image is successively input into a first neural network to be trained and a second neural network to be trained for image segmentation to obtain a second segmented training image;

[0023] A target loss value is determined according to the difference between the second segmented training image and the label image, and network parameters of the first neural network and the second neural network are adjusted according to the target loss value until a training end condition is met, thereby obtaining the pre-trained first neural network and the pre-trained second neural network.

[0024] In one embodiment, the step of successively inputting the training image into a first neural network to be trained and a second neural network to be trained for image segmentation to obtain a second segmented training image comprises:

[0025] Inputting the training image into a first neural network to be trained, roughly segmenting the training image to be segmented by the first neural network to be trained, and generating a first mask training image for the initial tissue and organ training region obtained by segmentation; the first mask training image includes a first training extraction region for extracting the initial tissue and organ training region in the medical image;

[0026] Determine a second training extraction region based on a first training extraction region in the first mask training image and a neighborhood of the first training extraction region;

[0027] Performing image extraction on the medical image based on the second training extraction area to obtain a position focusing training image;

[0028] The training image to be segmented, the position focus training image and the first mask training image are input into a second neural network to be trained, and the training image to be segmented is finely segmented by the second neural network to be trained to obtain a second segmented training image.

[0029] In one embodiment, determining the target loss value according to the difference between the second segmented training image and the label image includes:

[0030] Determining a first loss value based on a difference between the first mask training image and the label image;

[0031] Determining a second loss value based on a difference between the second segmented training image and the label image;

[0032] The target loss value is obtained by performing a weighted summation of the first loss value and the second loss value.

[0033] In a second aspect, the present application also provides an image segmentation device based on position prior. The device comprises:

[0034] A medical image rough segmentation module, used for inputting a medical image to be segmented into a pre-trained first neural network, performing rough segmentation on the medical image through the pre-trained first neural network, and generating a first mask image for initial tissues and organs obtained by segmentation; the first mask image includes a first extraction area for extracting the initial tissues and organs in the medical image;

[0035] A second extraction region generating module, used for generating a second extraction region based on the first extraction region and a neighborhood of the first extraction region;

[0036] A position-focused image acquisition module, configured to perform image extraction on the medical image based on the second extraction area to acquire a position-focused image;

[0037] The medical image fine segmentation module is used to input the medical image to be segmented, the position focus image and the first mask image into a pre-trained second neural network, and perform fine segmentation on the medical image through the pre-trained second neural network to obtain the target tissue organ in the medical image.

[0038] In a third aspect, the present application further provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements each step in the above-mentioned position prior-based image segmentation method embodiment when executing the computer program.

[0039] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in the above-mentioned position prior-based image segmentation method embodiment are implemented.

[0040] The above-mentioned image segmentation method, device, computer equipment, storage medium and computer program product based on position prior, by inputting the medical image to be segmented into a pre-trained first neural network, the medical image is roughly segmented by the pre-trained first neural network, and a first mask image is generated for the initial tissue organ obtained by segmentation; the first mask image contains a first extraction area for extracting the initial tissue organ in the medical image; based on the first extraction area and the neighborhood of the first extraction area, a second extraction area is generated; based on the second extraction area, the medical image is image extracted to obtain a position focus image; the medical image to be segmented, the position focus image and the first mask image are input into a pre-trained second neural network, and the medical image is finely segmented by the pre-trained second neural network to obtain the target tissue organ in the medical image. The present application can predetermine the position information of the region of interest (i.e., the above-mentioned target tissue organ or target area) on the entire image, and the position information includes the region of interest itself and the neighborhood image around it. Extracting the accurate tissue boundary information of the above-mentioned target tissue organ or target area is conducive to obtaining more accurate boundary segmentation results through neural network segmentation in the subsequent process, and can especially improve the accuracy of segmentation of small organs such as lymph nodes and optic chiasm nerves. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A diagram showing an application environment of an image segmentation method based on position prior in one embodiment;

[0042] Figure 2 A schematic diagram of a flow chart of an image segmentation method based on position prior in one embodiment;

[0043] Figure 3 is a schematic diagram of a rough segmentation process in one embodiment;

[0044] Figure 4 A schematic diagram of a process flow of a fine segmentation process in an embodiment;

[0045] FIG5( a ) is a schematic diagram of a training process of a method for image segmentation based on position prior in one embodiment;

[0046] FIG5( b ) is a schematic diagram of a training process of an image segmentation method based on position prior in another embodiment;

[0047] Figure 6 is a structural block diagram of an image segmentation device based on position prior in one embodiment;

[0048] Figure 7 is an internal structure diagram of a computer device in one embodiment;

[0049] Figure 8 FIG. 4 is a diagram showing the internal structure of a computer device in another embodiment. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0051] The image segmentation method based on position prior provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 101 communicates with the server 102 through the network. The data storage system can store the data that the server 102 needs to process. The data storage system can be integrated on the server 102, or it can be placed on the cloud or other network servers. Among them, the terminal 101 can be but is not limited to various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 102 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0052] In one embodiment, Figure 2 As shown in the figure, a method for image segmentation based on position prior is provided. Figure 1 The terminal 101 in the example is used as an example to illustrate, and the following steps are included:

[0053] Step S201, inputting a medical image to be segmented into a pre-trained first neural network, performing rough segmentation on the medical image by the pre-trained first neural network, and generating a first mask image for initial tissues and organs obtained by segmentation; the first mask image includes a first extraction region for extracting the initial tissues and organs in the medical image;

[0054] Among them, the medical image to be segmented refers to the original medical image used to extract the region of interest (i.e., the target tissue organ or target area), such as CT (Computed Tomography) image or MR (Magnetic Resonance) image, etc.; in order to analyze or track the region of interest in the original medical image, it is necessary to extract or segment the region of interest therein to distinguish it from other nearby tissue organs. The first neural network can adopt the UNet3d network, whose input channel is 1, and one image block can be input at a time. The mask processing process refers to the use of a selected image, graphic or object to block the image to be processed (all or part) to control the area or processing process of image processing. Among them, the specific image used for covering is called a mask. This application mainly processes digital images. In digital image processing, the mask is a two-dimensional matrix array, sometimes also a multi-value image. In this application, the mask is called a mask image. The first extraction area refers to the corresponding position of the region of interest in the first mask image, where this position is the rough position of the region of interest.

[0055] Specifically, suppose that lymph nodes need to be segmented from a CT image or MR image. Since lymph nodes are small and have very irregular shapes, they need to be extracted separately from the entire tissue area for further analysis. Figure 3 As shown in FIG. 1 , a medical image to be segmented (such as a CT image or an MR image) is input into a pre-trained first neural network (i.e., network 1), and the medical image is roughly segmented by the first neural network, that is, the region of interest contained in the medical image is located, identified, and segmented to distinguish it from the surrounding tissue image, and a mask process is performed on the initial tissue image obtained by segmentation to obtain a first mask image (such as Figure 3 The first mask image contains the approximate position information of the above-mentioned region of interest, and the mask result 1 is a rough segmentation result for the target tissue organ.

[0056] Optionally, before the medical image to be segmented is input into the pre-trained first neural network, the image may be preprocessed, and the preprocessing includes normalizing the image, that is, normalizing the image value to between 0 and 1 according to the window position that best displays the segmented part. The preprocessing also includes cropping the original image to obtain different image blocks, and flipping or rotating the cropped image blocks to achieve data enhancement.

[0057] Step S202: generating a second extraction region based on the first extraction region and the neighborhood of the first extraction region.

[0058] Among them, the neighborhood of the first extracted area refers to the neighboring area image of the area of ​​interest in the above-mentioned first mask image that expands outward. For example, for a given area, its neighborhood can be considered as a new area relative to the original given area after the given area is expanded outward by a certain range according to certain rules. The expansion rule is generally a four-neighborhood or eight-neighborhood expansion. Four neighborhoods refer to the pixels above, below, left and right of a pixel point, and eight neighborhoods must also include four points on two diagonals. The specific operation is as follows: Assuming that the expansion radius of the given area is n pixels, the boundary of the area can be found by the rules of four neighborhoods or eight neighborhoods. For any pixel point in the boundary, the rules of four neighborhoods or eight neighborhoods can be used to expand n pixels. Therefore, the neighborhood of the area is the area obtained by subtracting the area before expansion from the area after expansion.

[0059] The range of the neighborhood refers to the polygon radius corresponding to the best segmentation effect. The range of the neighborhood can be of different sizes according to different segmentation tasks, that is, the size of the best neighborhood range is determined according to the shape of different target tissues and organs; in addition, the best neighborhood can also be the minimum circumscribed polygon of the region of interest, such as the minimum circumscribed rectangle of the region of interest. The minimum circumscribed rectangle of the region of interest can be solved by the currently commonly used minimum circumscribed rectangle algorithm.

[0060] Specifically, the second extraction area is generated by the first extraction area and the neighborhood of the first extraction area, and the second extraction area may be, for example, the minimum circumscribed polygon. The generation method may be to obtain a rectangle containing the first extraction area by obtaining a maximum circumscribed polygon or by a rectangular focusing function.

[0061] Step S203: performing image extraction on the medical image based on the second extraction region to obtain a position focused image.

[0062] The position-focused image refers to a small area image that contains at least the target organ tissue in the medical image to be segmented. Figure 3 In the present invention, the position focused image is also referred to as the second mask image.

[0063] Position focusing is also a kind of morphological focusing. Specifically, the medical image is extracted based on the second extraction area to obtain a position focused image. The range of the position focused image is consistent with the second extraction area. The pixel values ​​in the position focused image are consistent with the pixel values ​​of the medical image to be segmented at the same position. The pixel values ​​outside the area are assigned to 0 (black), such as Figure 3 As shown in the focused image.

[0064] Step S204, inputting the medical image to be segmented, the position focused image and the first mask image into a pre-trained second neural network, performing fine segmentation on the medical image through the pre-trained second neural network, and obtaining the target tissue organ in the medical image.

[0065] Among them, the network structure of the second neural network can also adopt UNet3d. The number of input channels of the second neural network is 3, and 3 image blocks can be input at the same time.

[0066] Specifically, Figure 4 As shown, the medical image to be segmented (i.e., the original medical image), the position focused image and the first mask image are spliced ​​in the layer thickness direction, and the splicing result is used as the input of the second neural network. The three image blocks are finely segmented by the second neural network to obtain the target tissue organ, which may be, for example, the lymph node.

[0067] In the above embodiment, the medical image to be segmented is input into a pre-trained first neural network, the medical image is roughly segmented by the pre-trained first neural network, and a first mask image is generated for the initial tissue and organ obtained by segmentation; the first mask image contains a first extraction area for extracting the initial tissue and organ in the medical image; based on the first extraction area and the neighborhood of the first extraction area, a second extraction area is generated; based on the second extraction area, image extraction is performed on the medical image to obtain a position focus image; the medical image to be segmented, the position focus image and the first mask image are input into a pre-trained second neural network, and the medical image is finely segmented by the pre-trained second neural network to obtain the target tissue and organ in the medical image. This embodiment can pre-determine the position information of the region of interest (i.e., the above target tissue and organ or target area) on the entire medical image, and the position information includes the region of interest itself and the neighborhood range around it, and extracts the accurate tissue boundary information of the above target tissue and organ or target area, which is conducive to obtaining a more accurate boundary segmentation result through neural network segmentation in the subsequent process, especially for small organs such as lymph nodes, optic chiasm nerves, etc., which can improve the accuracy of segmentation. Furthermore, the present application can combine three dimensions of information, namely the global image, the image of the adjacent tissue of the region of interest and the coarse segmentation mask of the region of interest, for learning and mining by the neural network, thereby achieving the focused positioning of the target area / target tissue organ and effectively improving the accuracy of the segmentation results.

[0068] In one embodiment, the above step S202 includes: obtaining the centroid of the first extraction area in the first mask image; taking the centroid as the center point, forming a polygonal area around the center point with a radius of a preset length, and using the image containing the polygonal area as the second mask image.

[0069] The centroid is also called the center of gravity of the image. The polygon can be a triangle, rectangle, pentagon, etc.

[0070] Specifically, Figure 3 As shown, after the original medical image is roughly segmented by the first neural network, a first mask image is obtained, that is, the approximate position of the region of interest is determined, that is, the first extraction region. For example, the region of interest is a lymph node, as shown in FIG. Figure 3 As shown in the white area in the first mask image in , there can be multiple first extraction areas. The centroid of each first extraction area is calculated to obtain the centroid of each first extraction area. For each first extraction area, the centroid is used as the center point, and a polygonal area is constructed around the center point with a radius of a preset length, and the image containing the polygonal area is used as the second mask image. For example, the position of the first extraction area can be extracted by a rectangular focusing function to obtain the first extraction area and its surrounding adjacent areas, thereby determining the position of the rectangle and obtaining the second mask image, wherein the rectangular focusing function can be to use the centroid as a fixed point, determine the long side and the short side according to actual needs, and thus extract the rectangle, and use the centroid as the center point to determine the second mask image by combining multiple rectangles.

[0071] In the above embodiment, the second mask image is obtained by further regional positioning of the mask image obtained by the coarse segmentation of the first neural network. Obviously, the second mask image includes the surrounding adjacent position area of ​​the target tissue organ. This is beneficial for the subsequent learning of more boundary information by the second neural network based on the position information of the region of interest, avoiding missing information, and thus extracting more accurate target tissue organs.

[0072] In one embodiment, the above-mentioned method of taking the centroid as the center point, forming a polygonal area around the center point with a radius of a preset length, and using the image containing the polygonal area as the second mask image includes: in the case of multiple centroids, taking each centroid as the center point, and forming multiple polygonal areas around the center point with a radius of a preset length; obtaining the intersection area of ​​the multiple polygonal areas, and using the image containing the intersection area as the second mask image.

[0073] Specifically, when there are multiple centroids, each centroid is used as the center point, and a polygon is constructed around the center point with a radius of a preset length. For example, a rectangle is constructed by a rectangular focusing function, and a length R is used as the radius to construct rectangles around different centroids, so that each rectangle can become the largest circumscribed rectangle containing the target tissue organ. If multiple focusing rectangles overlap, their intersection is taken. Among them, the length R is different in different segmentation tasks and can be flexibly adjusted according to actual conditions. For example, for small organ segmentation tasks, R is usually taken as 10 pixel units, and for large organ tasks, R is usually taken as 5 pixel units.

[0074] The above embodiment considers how to construct a polygon when there are multiple centroids so that the polygon can both encompass the target organ tissue to be segmented and ensure that too many non-target tissue organs are not mistaken for the target object to be segmented, thereby improving the accuracy of segmentation.

[0075] In one embodiment, the step S203 includes: setting the second extraction area to a preset pixel value to generate a second mask image; and multiplying the second mask image with the medical image to be segmented to generate a position-focused image.

[0076] The position-focused image refers to an image that is formed by focusing on the position information of the target tissue or organ in the original image and can emphasize the position information.

[0077] Specifically, since the second extraction area is the area that needs to be focused, the pixel value of the second extraction area is set to a preset pixel value, for example, set to a pixel value of 1 (1 represents white), and the pixel values ​​of other areas are set to 0 (black) to generate a second mask image. The second mask image is multiplied with the medical image to be segmented (i.e., the original medical image). Since the pixel value of the second extraction area is 1, after multiplication, the original pixel value at the corresponding position in the original medical image is multiplied by 1 and remains the original pixel value, that is, the original pixel value in the second extraction area is retained, while other areas become 0 after multiplication by 0. The image generated in this way retains the pixel value of the second extraction area, and the final image generated is a position focused image.

[0078] In the above embodiment, a second mask image is generated by focusing on the position information of the region of interest, i.e., the second extraction region, and the region of interest in the original medical image is extracted through the second mask image, which is beneficial for obtaining more accurate segmentation results when segmenting the image subsequently.

[0079] In one embodiment, as shown in FIG5 , before step S201, the following steps are further included:

[0080] Step S501, obtaining a training image to be segmented and a label image of the training image; the label image identifies the tissue organ to be segmented in the training image;

[0081] Among them, the training image to be segmented refers to the learning sample image used to build the model; the label image of the training image refers to the image in which the tissue organs to be segmented are marked in advance in the above-mentioned training image to be segmented, and the label image can be used as the gold standard for training the network model.

[0082] Specifically, firstly, training images to be segmented are obtained, such as CT images or MR images. The tissues and organs to be segmented in these training images to be segmented are labeled, generally by an experienced clinician. Furthermore, these training images to be segmented can be randomly divided into a training set and a test set, and the ratio can be 8:2, wherein the training set is used to build the model, and the test set is used to verify the segmentation effect of the built model.

[0083] Step S502, successively inputting the training image into the first neural network to be trained and the second neural network to be trained to perform image segmentation, thereby obtaining a second segmented training image;

[0084] The first neural network to be trained may adopt a UNet3d network structure, whose input channel is 1, and one image block may be input at a time; the network structure of the second neural network may adopt UNet3d;

[0085] The training image to be segmented is first input into the first neural network for rough segmentation to obtain a rough segmentation result, i.e., a first mask training image; and the rough segmentation result and the training image to be segmented are input into the second neural network for fine segmentation to obtain a second segmented training image.

[0086] Step S503, determining a target loss value based on the difference between the second segmented training image and the above-mentioned label image, and adjusting the network parameters of the first neural network and the network parameters of the second neural network based on the target loss value until the training end condition is met, thereby obtaining a pre-trained first neural network and the pre-trained second neural network.

[0087] Specifically, the second segmented training image is compared with the above-mentioned label image that has been marked in advance, and the target loss value is determined according to the difference between the two. The target loss value can be calculated using the Dice loss function. The network parameters of the first neural network and the network parameters of the second neural network are adjusted to guide the loss value to reach the minimum or meet other training end conditions, such as the target loss value fluctuates stably within a preset range. At this time, a pre-trained first neural network and a pre-trained second neural network are obtained.

[0088] Furthermore, after obtaining the pre-trained first neural network and the pre-trained second neural network, the pre-trained first neural network and the pre-trained second neural network may be further verified using the test set to ensure accurate segmentation effect.

[0089] In the above embodiment, the first neural network and the second neural network are trained by using pre-labeled training set images to obtain a trained first neural network and a trained second neural network, thereby providing a data basis for subsequent image segmentation.

[0090] In one embodiment, the above step S502 further includes:

[0091] Step S5021, input the above-mentioned training image into the first neural network to be trained, roughly segment the above-mentioned training image to be segmented through the first neural network to be trained, and generate a first mask training image for the initial tissue organ training area obtained by segmentation; the first mask training image contains a first training extraction area for extracting the initial tissue organ training area in the above-mentioned training image to be segmented.

[0092] The first neural network to be trained may adopt a UNet3d network structure, whose input channel is 1, and one image block may be input at a time. The initial tissue and organ training region refers to the region obtained after the target tissue and organ in the training image is roughly segmented.

[0093] The first mask training image is a mask image obtained after mask processing is performed on the initial tissue and organ training region.

[0094] Specifically, the above-mentioned training image or the image in the above-mentioned training set is input into the first neural network to be trained, and the above-mentioned training image is roughly segmented by the first neural network, and the initial tissue and organ training area is roughly segmented and marked on the training image. At the same time, the first neural network also performs mask processing on the initial tissue and organ training area to obtain a first mask training image.

[0095] Step S5022, determining a second training extraction region based on the first training extraction region in the first mask training image and the neighborhood of the first training extraction region;

[0096] The first training extraction region refers to a rough segmentation result of the target tissue organ obtained after rough segmentation.

[0097] Specifically, the second training extraction area is generated by the first training extraction area and the neighborhood of the first training extraction area, and the second training extraction area may be, for example, a minimum circumscribed polygon. The generation method may be to obtain a rectangle containing the first training extraction area by obtaining a maximum circumscribed polygon or by a rectangular focusing function.

[0098] Step S5023: performing image extraction on the training image to be segmented based on the second training extraction area to obtain a position-focused training image.

[0099] The position-focused training image refers to an image obtained by focusing on the position information of the target tissue organ in the training image.

[0100] Specifically, image extraction is performed on the medical image based on the second training extraction region to obtain a position-focused training image, the range of which is consistent with the second extraction region. The pixel values ​​in the position-focused training image are consistent with the pixel values ​​of the medical image to be segmented at the same position, and the pixel values ​​outside the region are assigned to 0 (black).

[0101] Step S5024, inputting the training image to be segmented, the position focus training image and the first mask training image into the second neural network to be trained, and performing fine segmentation on the training image to be segmented by the second neural network to be trained to obtain a second segmented training image;

[0102] The network structure of the second neural network can adopt UNet3d, and the number of input channels is 3, that is, 3 image blocks can be input simultaneously. The structure of the second neural network is not particularly limited, and it can be a two-dimensional structure or a three-dimensional structure, as long as the accuracy of each input channel is ensured.

[0103] Specifically, the training image to be segmented, the position focus training image and the first mask training image obtained after the rough segmentation are used as inputs of the second neural network, and the training image to be segmented is finely segmented by the second neural network to obtain a second segmented training image.

[0104] In the above embodiment, a location-focused training image is obtained by extracting a neighborhood and performing image extraction on the training image to be segmented based on the neighborhood, which is conducive to obtaining more accurate location information of the target tissue organ later.

[0105] In one embodiment, determining the target loss value based on the difference between the second segmented training image and the label image in the above step S503 includes: determining the first loss value based on the difference between the first mask training image and the label image; determining the second loss value based on the difference between the second segmented training image and the label image; and performing weighted summation of the first loss value and the second loss value to obtain the target loss value.

[0106] Specifically, a first loss value between the output of the first neural network (ie, the first mask training image) and the label image is calculated. For example, the first loss value can be calculated using a Dice loss function. The Dice loss function is as follows:

[0107]

[0108] Among them, s1 represents the first loss value, X represents the label image, and Y1 represents the output result of the first neural network (that is, the first mask training image mentioned above).

[0109] Then, the second loss value between the output of the second neural network (i.e., the second segmented training image) and the label image is calculated. For example, the second loss value can be calculated using the Dice loss function. The Dice loss function is as follows:

[0110]

[0111] Among them, s2 represents the second loss value, X represents the label image, and Y2 represents the output result of the second neural network (that is, the above-mentioned second segmentation training image).

[0112] Finally, the first loss value s1 and the second loss value s2 are weighted (for example, the weights are set to 0.4 and 0.6 respectively) to obtain the target loss value (i.e., the total loss), and the total loss is back-propagated to synchronously update the parameters in the first neural network and the second neural network. After completing multiple training cycles, when the calculated total loss value meets the training end condition, it can be considered that the training of the first neural network and the second neural network has been completed.

[0113] In the above embodiment, the total loss is obtained by weighting the first loss value and the second loss value, and the total loss is adjusted to complete the training of the first neural network and the second neural network, which is beneficial to subsequently obtaining the trained first neural network and the second neural network.

[0114] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0115] Based on the same inventive concept, the embodiment of the present application also provides a position prior-based image segmentation device for implementing the position prior-based image segmentation method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the position prior-based image segmentation device provided below can refer to the limitations of the position prior-based image segmentation method above, and will not be repeated here.

[0116] In one embodiment, Figure 6As shown, a position prior-based image segmentation device 600 is provided, comprising: a medical image coarse segmentation module 601, a second extraction region generation module 602, a position focused image acquisition module 603 and a medical image fine segmentation module 604, wherein:

[0117] The medical image rough segmentation module 601 is used to input the medical image to be segmented into a pre-trained first neural network, perform rough segmentation on the medical image through the pre-trained first neural network, and generate a first mask image for the initial tissue and organ obtained by segmentation; the first mask image includes a first extraction area for extracting the initial tissue and organ in the medical image;

[0118] A second extraction region generating module 602, configured to generate a second extraction region based on the first extraction region and a neighborhood of the first extraction region;

[0119] A position focused image acquisition module 603, configured to perform image extraction on the medical image based on the second extraction area to acquire a position focused image;

[0120] The medical image fine segmentation module 604 is used to input the medical image to be segmented, the position focus image and the first mask image into a pre-trained second neural network, and perform fine segmentation on the medical image through the pre-trained second neural network to obtain the target tissue organ in the medical image.

[0121] In one embodiment, the second extraction region generating module 602 is further used for:

[0122] In the first mask image, the centroid of the first extraction area is obtained; taking the centroid as the center point, a polygonal area is formed around the center point with a radius of a preset length, and an image containing the polygonal area is used as the second mask image.

[0123] In one embodiment, the second extraction region generating module 602 is further used for:

[0124] In the case where there are multiple centroids, each centroid is used as a center point, and multiple polygonal areas are formed around the center point with a radius of the preset length; the intersection area of ​​the multiple polygonal areas is obtained, and the image containing the intersection area is used as the second mask image.

[0125] In one embodiment, the position focused image acquisition module 603 is further used to:

[0126] The second extraction area is set to a preset pixel value to generate a second mask image; and the second mask image is multiplied with the medical image to be segmented to generate the position focused image.

[0127] In one embodiment, a model training unit is further included, which is used to:

[0128] A training image to be segmented and a label image of the training image are obtained, wherein the label image identifies the tissue and organ to be segmented in the training image; the training image is input into a first neural network to be trained, the training image to be segmented is roughly segmented by the first neural network to be trained, and a first mask training image is generated for the initial tissue and organ training area obtained by segmentation; the first mask training image contains a first training extraction area for extracting the initial tissue and organ training area in the training image to be segmented; a second training extraction area is determined based on the first training extraction area in the first mask training image and the neighborhood of the first training extraction area; and a second training extraction area is determined based on the second training extraction area. The training extraction area performs image extraction on the training image to be segmented to obtain a position-focused training image; the training image to be segmented, the position-focused training image and the first mask training image are input into a second neural network to be trained, and the training image to be segmented is finely segmented by the second neural network to be trained to obtain a second segmented training image; a target loss value is determined according to the difference between the second segmented training image and the label image, and network parameters of the first neural network and network parameters of the second neural network are adjusted according to the target loss value until a training end condition is met, thereby obtaining the pre-trained first neural network and the pre-trained second neural network.

[0129] In one embodiment, the model training unit is further used to:

[0130] Based on the first mask training image and the label image, a first loss value is determined; based on the difference between the second segmented training image and the label image, a second loss value is determined; and the first loss value and the second loss value are weightedly summed to obtain the target loss value.

[0131] Each module in the above-mentioned image segmentation device based on position prior can be implemented in whole or in part by software, hardware and their combination. Each module can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0132] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as medical images and image segmentation results. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for image segmentation based on position prior is implemented.

[0133] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 8 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for image segmentation based on position prior is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse, etc.

[0134] Those skilled in the art will understand that Figures 7 to 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0135] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above-mentioned position prior-based image segmentation method embodiment when executing the computer program.

[0136] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, each step in the above-mentioned position prior-based image segmentation method embodiment is implemented.

[0137] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above-mentioned position prior-based image segmentation method embodiment when executed by a processor.

[0138] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0139] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0140] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0141] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for image segmentation based on position prior, characterized in that: The method comprises: Inputting a medical image to be segmented into a pre-trained first neural network, roughly segmenting the medical image through the pre-trained first neural network, and generating a first mask image for initial tissues and organs obtained by segmentation; the first mask image includes a first extraction region for extracting the initial tissues and organs in the medical image; generating a second extraction region based on the first extraction region and a neighborhood of the first extraction region; Setting the second extraction area to a preset pixel value to generate a second mask image; performing a multiplication operation on the second mask image and the medical image to be segmented to generate a position focused image; The medical image to be segmented, the position focused image and the first mask image are input into a pre-trained second neural network, and the pre-trained second neural network is used to perform fine segmentation on the medical image to obtain the target tissue organ in the medical image.

2. The method according to claim 1, characterized in that: The step of generating a second extraction area based on the first extraction area and a neighborhood of the first extraction area comprises: In the first mask image, obtaining the centroid of the first extraction area; Taking the centroid as the center point, a polygonal area is formed around the center point with a radius of a preset length, and an image including the polygonal area is used as a second mask image.

3. The method according to claim 2, characterized in that The method of forming a polygonal area around the center point with a radius of a preset length and taking the image containing the polygonal area as the second mask image includes: In the case where there are multiple centroids, each of the centroids is taken as a center point, and multiple polygonal areas are formed around the center point by a radius of the preset length; An intersection area of ​​a plurality of polygonal areas is obtained, and an image including the intersection area is used as the second mask image.

4. The method according to claim 1, characterized in that: Before inputting the medical image to be segmented into the pre-trained first neural network, the method further includes: Acquire a training image to be segmented and a label image of the training image; the label image identifies the tissue and organ to be segmented in the training image; The training image is successively input into a first neural network to be trained and a second neural network to be trained for image segmentation to obtain a second segmented training image; A target loss value is determined according to the difference between the second segmented training image and the label image, and network parameters of the first neural network and the second neural network are adjusted according to the target loss value until a training end condition is met, thereby obtaining the pre-trained first neural network and the pre-trained second neural network.

5. The method according to claim 4, characterized in that The step of successively inputting the training image into a first neural network to be trained and a second neural network to be trained for image segmentation to obtain a second segmented training image comprises: Inputting the training image into a first neural network to be trained, roughly segmenting the training image to be segmented by the first neural network to be trained, and generating a first mask training image for the initial tissue and organ training region obtained by segmentation; the first mask training image includes a first training extraction region for extracting the initial tissue and organ training region in the training image to be segmented; Determine a second training extraction region based on a first training extraction region in the first mask training image and a neighborhood of the first training extraction region; Performing image extraction on the training image to be segmented based on the second training extraction area to obtain a position-focused training image; The training image to be segmented, the position focus training image and the first mask training image are input into a second neural network to be trained, and the training image to be segmented is finely segmented by the second neural network to be trained to obtain a second segmented training image.

6. The method according to claim 5, characterized in that The determining the target loss value according to the difference between the second segmented training image and the label image comprises: Determining a first loss value based on a difference between the first mask training image and the label image; Determining a second loss value based on a difference between the second segmented training image and the label image; The target loss value is obtained by performing a weighted summation of the first loss value and the second loss value.

7. An image segmentation device based on position prior, characterized in that: The device comprises: A medical image rough segmentation module, used for inputting a medical image to be segmented into a pre-trained first neural network, performing rough segmentation on the medical image through the pre-trained first neural network, and generating a first mask image for initial tissues and organs obtained by segmentation; the first mask image includes a first extraction area for extracting the initial tissues and organs in the medical image; A second extraction region generating module, used for generating a second extraction region based on the first extraction region and a neighborhood of the first extraction region; A position focused image acquisition module, used for setting the second extraction area to a preset pixel value, generating a second mask image, and performing a multiplication operation on the second mask image and the medical image to be segmented to generate a position focused image; The medical image fine segmentation module is used to input the medical image to be segmented, the position focus image and the first mask image into a pre-trained second neural network, and perform fine segmentation on the medical image through the pre-trained second neural network to obtain the target tissue organ in the medical image.

8. The device according to claim 7, characterized in that The second extraction region generating module is further used for: In the first mask image, obtaining the centroid of the first extraction area; Taking the centroid as the center point, a polygonal area is formed around the center point with a radius of a preset length, and an image including the polygonal area is used as a second mask image.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images