Image segmentation method and device, electronic equipment and storage medium

Abstract

The application discloses an image segmentation method and device, electronic equipment and storage medium. The method comprises the following steps: acquiring an original image, obtaining an initial segmentation image and an extended segmentation image of the original image based on an initial segmentation model and an extended segmentation model which are trained in advance; performing restoration processing on the extended segmentation image to obtain a restored image; and obtaining a target segmentation image of the original image based on the initial segmentation image, the extended segmentation image and the restored image. Through the technical scheme disclosed by the application, the problem that the segmentation result obtained based on the existing segmentation method has the phenomenon of region interruption, thereby leading to poor accuracy of the segmentation result, is solved, the completeness of the segmentation result is improved, and the accuracy of the segmentation result is improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image segmentation method, apparatus, electronic device, and storage medium. Background Technology

[0002] In the field of modern medical technology, automated coronary artery reconstruction technology has significant clinical value and practical implications for physicians. The first step in implementing automated coronary artery reconstruction is to address the issue of automated coronary artery segmentation. Unlike traditional coronary artery segmentation methods, due to the numerous advantages of neural networks, there is an increasing preference for using neural networks to perform coronary artery segmentation.

[0003] In the process of coronary artery segmentation based on neural networks, this invention found that there are regional interruptions in the segmentation results, which leads to poor accuracy of the segmentation results. Summary of the Invention

[0004] This invention provides an image segmentation method, apparatus, electronic device, and storage medium to solve the problem of region interruption in the segmentation results obtained based on existing segmentation methods, which leads to poor segmentation accuracy. The invention aims to improve the completeness of the segmentation results, thereby improving the accuracy of the segmentation results.

[0005] In a first aspect, embodiments of the present invention provide an image segmentation method, the method comprising:

[0006] The original image is obtained, and based on the pre-trained initial segmentation model and extended segmentation model, the initial segmentation image and extended segmentation image of the original image are obtained respectively.

[0007] The expanded segmented image is then restored to obtain the restored image.

[0008] Based on the initial segmented image, the expanded segmented image, and the restored image, the target segmented image of the original image is obtained.

[0009] Optionally, obtaining the target segmentation image of the original image based on the initial segmentation image, the expanded segmentation image, and the restored image includes:

[0010] The initial segmented image and the expanded segmented image are compared and overlapped to generate a first processed image;

[0011] The first processed image and the restored image are stitched together to generate a second processed image;

[0012] The initial segmented image and the second processed image are stitched together to generate the target segmented image of the original image.

[0013] Optionally, the step of stitching the initial segmented image and the second processed image to generate the target segmented image of the original image includes:

[0014] The initial segmented image and the second processed image are stitched together to obtain a fused image;

[0015] Remove the unconnected regions from the fused image to obtain the target segmentation image of the original image.

[0016] Optionally, the step of stitching the initial segmented image and the second processed image to generate the target segmented image of the original image includes:

[0017] Based on the initial segmented image and the second processed image, candidate regions are determined;

[0018] Based on the region boundary of the candidate region and the region boundary of the second processed image, at least one connected region in the candidate region is determined;

[0019] The target segmentation image of the original image is obtained by fusing the connected region with the second processed image.

[0020] Optionally, the step of restoring the expanded segmented image to obtain the restored image includes:

[0021] Obtain preset restoration parameters, and perform restoration processing on the extended segmented image based on the restoration parameters to obtain the restored image of the extended segmented image.

[0022] Optionally, the process of obtaining the initial segmentation image and the expanded segmentation image of the original image based on the pre-trained initial segmentation model and expanded segmentation model includes:

[0023] The original image is input into the initial segmentation model to obtain the initial segmented image output by the initial segmentation model;

[0024] The original image is input into the extended segmentation model to obtain the extended segmented image output by the extended segmentation model.

[0025] Optionally, the training method for the extended segmentation model includes:

[0026] Obtain sample images for training the extended segmentation model, annotate the target regions in the sample images, and extend the target regions to obtain extended labels for the target regions;

[0027] The sample image is input into the extended segmentation model to be trained to obtain the segmentation prediction result output by the extended segmentation model;

[0028] The extended segmentation model is trained based on the extended labels of the target region and the segmentation prediction results to obtain the extended segmentation model.

[0029] Secondly, embodiments of the present invention also provide an image segmentation apparatus, the apparatus comprising:

[0030] The segmented image acquisition module is used to acquire the original image and obtain the initial segmented image and the extended segmented image of the original image based on the pre-trained initial segmentation model and extended segmentation model, respectively.

[0031] The restored image acquisition module is used to perform restoration processing on the extended segmented image to obtain the restored image;

[0032] The target segmentation image acquisition module is used to obtain the target segmentation image of the original image based on the initial segmentation image, the extended segmentation image, and the restored image.

[0033] Thirdly, embodiments of the present invention also provide an electronic device, comprising:

[0034] At least one processor; and

[0035] A memory communicatively connected to the at least one processor; wherein,

[0036] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the image segmentation method according to any embodiment of the present invention.

[0037] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions that are used to cause a processor to execute the image segmentation method described in any embodiment of the present invention.

[0038] The technical solution of this invention is based on an initial segmentation model to obtain an initial segmentation image, acquire branch detail information of the original image, and simultaneously based on an extended segmentation model to obtain an extended segmentation image, acquire branch connectivity information of the original image. The extended segmentation image is then restored to ensure that the lumen shape of the subsequently obtained target segmentation result is consistent with the lumen shape in the original image. Furthermore, based on the branch detail information of the initial segmentation image, the branch connectivity information of the extended segmentation image, and the lumen shape information of the restored image, the target segmentation result of the generated original image is made closer to the original image, thereby improving the accuracy of the segmentation result.

[0039] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a flowchart of an image segmentation method provided in Embodiment 1 of the present invention;

[0042] Figure 2 This is a flowchart of an image segmentation method provided in Embodiment 2 of the present invention;

[0043] Figure 3 This is a schematic diagram illustrating the principle of generating the first processed image according to Embodiment 2 of the present invention;

[0044] Figure 4 This is a schematic diagram illustrating the principle of generating the second processed image according to Embodiment 2 of the present invention;

[0045] Figure 5 This is a schematic diagram of the principle of determining candidate regions involved in Embodiment 2 of the present invention;

[0046] Figure 6 This is a schematic diagram illustrating the principle of generating a target segmentation image according to Embodiment 2 of the present invention;

[0047] Figure 7 This is a schematic diagram of another method for generating a target segmentation image according to Embodiment 2 of the present invention;

[0048] Figure 8 This is a schematic diagram of the structure of an image segmentation device provided in Embodiment 3 of the present invention;

[0049] Figure 9 This is a schematic diagram of the structure of an electronic device that implements the image segmentation method of this invention. Detailed Implementation

[0050] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0051] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in sequences other than those illustrated or described herein.

[0052] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0053] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0054] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0055] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0056] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0057] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0058] Example 1

[0059] Figure 1 The flowchart of an image segmentation method provided in Embodiment 1 of the present invention is applicable to the situation of image segmentation. The method can be executed by an image segmentation device, which can be implemented in hardware and / or software and can be configured in a smart terminal and a cloud server.

[0060] In some other embodiments, when segmenting an image containing numerous branches, existing segmentation methods can obtain segmentation results for each branch, but interruptions can easily occur between branches, leading to inaccurate segmentation results. To address these technical problems, this embodiment proposes an image segmentation algorithm. Specifically, it obtains an initial segmented image based on an initial segmentation model, acquiring branch detail information of the original image. Simultaneously, it obtains an extended segmented image based on an extended segmentation model, acquiring branch connectivity information of the original image. Furthermore, it performs restoration processing on the extended segmentation to ensure that the lumen shape of the subsequently obtained target segmentation result is consistent with the lumen shape in the original image. Based on the branch detail information of the initial segmentation image, the branch connectivity information of the extended segmentation image, and the lumen shape information of the restored image, the generated target segmentation result of the original image is made closer to the original image, thereby improving the accuracy of the segmentation result.

[0061] like Figure 1 As shown, the method specifically includes:

[0062] S110. Obtain the original image. Based on the pre-trained initial segmentation model and extended segmentation model, obtain the initial segmentation image and extended segmentation image of the original image, respectively.

[0063] In this embodiment of the invention, the original image can be understood as an image containing numerous branches, such as an image containing coronary arteries. Optionally, it can also be other images with branches, such as flower branch images, etc. This embodiment does not limit the image content.

[0064] Specifically, if the original image contains coronary arteries, the acquisition method may include scanning the target object with medical imaging equipment to obtain the original image, or reading image data from a local database or a cloud server database to obtain the original image; optionally, if the original image contains flower branches, the acquisition method may include taking a picture with an imaging device to obtain the original image, or scanning with a preset scanning device to obtain the original image; different acquisition methods may be used depending on the type of the original image in this embodiment, and this embodiment does not limit this.

[0065] Optionally, given the original image, image segmentation processing is performed on the original image. Specifically, a pre-trained segmentation model can be used to process the original image to obtain the corresponding segmented image. Optionally, obtaining the initial segmented image of the original image based on the pre-trained initial segmentation model includes: inputting the original image into the initial segmentation model to obtain the initial segmented image output by the initial segmentation model; optionally, obtaining the extended segmented image of the original image based on the pre-trained extended segmentation model includes: inputting the original image into the extended segmentation model to obtain the extended segmented image output by the extended segmentation model.

[0066] It should be noted that this embodiment does not limit the order in which the initial segmentation image and the extended segmentation image are obtained. They can be obtained simultaneously or sequentially. This embodiment does not limit this.

[0067] Based on the above embodiments, the technical solution of this embodiment pre-trains the initial segmentation model before inputting the original image into the initial segmentation model to obtain a trained segmentation model. Specifically, the training process of the initial segmentation model includes: acquiring coronary artery images for model training and manually annotating the coronary artery images; inputting the annotated coronary artery images into a pre-determined deep neural network to obtain segmentation prediction results; comparing the segmentation prediction results of each coronary artery image with the manually annotated images, generating a loss function for the model based on the comparison results, and updating the network structure parameters of the deep neural network based on the loss function to make it evolve in the direction of reducing prediction errors; furthermore, using a large number of coronary artery images as training samples, repeating the above iterative process thousands of times, and finally stopping training when the comparison results between the segmentation prediction results and the manual standard reach a preset threshold to obtain the trained initial segmentation model.

[0068] In this embodiment, the initial segmented image obtained based on the initial segmentation model can yield segmentation results with rich branch details. It should be noted that during the image segmentation process of obtaining sample images containing coronary vessels, due to the diversity of the original images, some samples may have coronary artery stenosis caused by atherosclerosis or other reasons, which may lead to the interruption of branches in the segmentation results of the original image. The segmentation results of the extended segmentation model used in this embodiment can avoid the above-mentioned interruption phenomenon.

[0069] Based on the above implementation, this embodiment pre-trains the extended segmentation model before obtaining the extended segmentation result, thereby ensuring the accuracy of the segmentation result. Optionally, the training process of the extended segmentation model includes: acquiring sample images for training the extended segmentation model; annotating the target regions in the sample images and expanding the target regions to obtain expanded labels; inputting the sample images into the extended segmentation model to be trained to obtain the segmentation prediction result output by the extended segmentation model; and training the extended segmentation model based on the expanded labels of the target regions and the segmentation prediction result to obtain the extended segmentation model.

[0070] Specifically, sample images for training the extended segmentation model are acquired. These sample images include those showing normal coronary artery lumens and those showing stenosis. Based on the acquired images of labeled coronary artery lumens, each labeled coronary artery lumen is expanded to obtain the expanded coronary artery lumen for training the extended segmentation model. Optionally, the expansion method in this embodiment may include, but is not limited to, local expansion, global expansion, and gradual expansion. Specifically, local expansion can be understood as expanding the stenotic area to the normal size while keeping the normal areas of the lumen unchanged; global expansion can be understood as expanding the entire coronary artery lumen to a uniform size, ensuring that the diameter of the lumen at the stenosis point is reduced to the normal size; gradual expansion can be understood as expanding the coronary artery lumen to a preset size, for example, the lumen shape gradually decreases from the proximal to the distal end. Of course, the expansion methods described above are only optional expansion methods, and other expansion methods can also be used in this embodiment, which will not be listed here.

[0071] It should be noted that the effect of using the expanded samples for training in this embodiment is to enhance the connectivity of the narrow part of the lumen, thereby improving the learning ability of the expanded segmentation model for expanded segmentation.

[0072] S120. Perform restoration processing on the extended segmented image to obtain the restored image.

[0073] In this embodiment of the invention, since the lumen shape of the coronary artery in the obtained extended segmentation image is inconsistent with the lumen shape in the original image, during the process of obtaining the target segmentation image based on the extended segmentation image, in order to ensure that the lumen shape in the subsequently obtained target segmentation image is consistent with the lumen shape in the original image, it is necessary to perform restoration processing on the extended segmentation image to restore the lumen shape in the extended segmentation image to the lumen shape of the coronary artery in the original image. Optionally, the method for performing restoration processing may include: obtaining preset restoration parameters, performing restoration processing on the extended segmentation image based on the restoration parameters, and obtaining the restored image of the extended segmentation image.

[0074] Specifically, restoration parameters can be set based on the shape of the coronary artery lumen in the original image, and then the extended segmented image can be restored based on the restoration parameters to obtain a restored image with the same lumen shape as the original image. Optionally, multiple restoration parameters can be set according to the different lumen shapes of different branches in the coronary artery lumen, so as to make the obtained restored image more accurate.

[0075] The above embodiments exemplify the restoration processing of expanded segmented images based on erosion methods, but are not intended to limit the present invention. In other embodiments, the restoration parameters can also be set based on other methods, such as morphological restoration methods like cropping and sampling, to restore the expanded segmented images.

[0076] S130. Based on the initial segmented image, the expanded segmented image, and the restored image, the target segmented image of the original image is obtained.

[0077] In this embodiment of the invention, since the initial segmentation model can obtain the branch detail information of the original image, the branch connectivity information of the original image can be obtained based on the extended segmentation image, and the lumen shape information of the original image can be obtained based on the restored image; on this basis, the initial segmentation image, the extended segmentation image and the restored image are subjected to image fusion processing to obtain a target segmentation image that is consistent with the original image and has branch connectivity and branch richness, thereby improving the accuracy of the target segmentation image.

[0078] The technical solution of this invention is based on an initial segmentation model to obtain an initial segmentation image, acquire branch detail information of the original image, and simultaneously based on an extended segmentation model to obtain an extended segmentation image, acquire branch connectivity information of the original image. The extended segmentation image is then restored to ensure that the lumen shape of the subsequently obtained target segmentation result is consistent with the lumen shape in the original image. Furthermore, based on the branch detail information of the initial segmentation image, the branch connectivity information of the extended segmentation image, and the lumen shape information of the restored image, the target segmentation result of the generated original image is made closer to the original image, thereby improving the accuracy of the segmentation result.

[0079] Example 2

[0080] Figure 2 This is a flowchart of an image segmentation method provided in Embodiment 2 of the present invention. Based on the above embodiments, this embodiment optionally obtains a target segmentation image of the original image based on an initial segmented image, an expanded segmented image, and a restored image, including:

[0081] The initial segmented image and the expanded segmented image are compared to obtain the first processed image.

[0082] The first processed image and the restored image are stitched together to generate the second processed image;

[0083] The initial segmented image and the second processed image are stitched together to generate the target segmented image of the original image.

[0084] like Figure 2 As shown, the method includes:

[0085] S210. Obtain the original image. Based on the pre-trained initial segmentation model and extended segmentation model, obtain the initial segmentation image and extended segmentation image of the original image, respectively.

[0086] S220. Perform restoration processing on the extended segmented image to obtain the restored image.

[0087] S230. Perform overlap comparison processing on the initial segmented image and the expanded segmented image to generate the first processed image.

[0088] In this embodiment of the invention, the overlap comparison processing can be understood as performing intersection processing. In other words, performing overlap comparison processing on the initial segmented image and the extended segmented image can also be understood as obtaining the overlapping regions that intersect in the initial segmented image and the extended segmented image, and performing fusion processing on the overlapping regions to obtain the first processed image in this embodiment. See the exemplary version. Figure 3 In the figure, Pn represents the initial segmentation image, and Pd represents the extended segmentation image. Pn and Pd are compared and overlapped to obtain the overlap result (Pc), which is the first processed image. In this embodiment, the first processed image obtained after intersection processing retains neither the branch detail information of the initial segmentation image nor the branch connectivity information of the extended segmentation image. It is used as the reference image for the target segmentation image, and detail information and connectivity information are stitched together based on this reference image to obtain a target segmentation image that closely approximates the original image.

[0089] S240. The first processed image and the restored image are stitched together to generate the second processed image.

[0090] In this embodiment, stitching the first processed image and the restored image can also be understood as performing a union operation on the first processed image and the restored image, followed by image fusion, to obtain the processed second processed image. Specifically, based on the first processed image as the reference image for the target segmentation image, it is stitched together with the restored image to obtain the stitched second processed image. See the example below. Figure 4In the figure, Pc represents the first processed image, and Pr represents the restored image. Pc and Pr are concatenated to obtain a segmented image (Pb) with connectivity information, which is the second processed image. In this embodiment, the restored image retains the branch detail information of the original image, and its lumen shape is also consistent with the original image. Therefore, by fusing the restored image with the reference image, the resulting processed image, while consistent with the original image, adds connectivity information, thus ensuring the accuracy of the subsequently obtained target segmentation image to a certain extent.

[0091] S250. The initial segmented image and the second processed image are stitched together to generate the target segmented image of the original image.

[0092] In this embodiment of the invention, in order to obtain branch detail information in the segmented image, the second processed image is stitched together with the initial segmented image to obtain the target segmented image.

[0093] Optionally, the method for stitching the initial segmented image and the second processed image to generate the target segmented image of the original image may include: determining candidate regions based on the initial segmented image and the second processed image; determining at least one connected region among the candidate regions based on the region boundaries of the candidate regions and the region boundaries of the second processed image; and performing fusion processing based on the connected regions and the second processed image to obtain the target segmented image of the original image.

[0094] Specifically, since the initial segmentation image contains branch detail information of the original image, but the second processed image only contains branch connectivity information of the original image and not branch detail information, the initial segmentation image and the second processed image are subtracted to obtain the branch detail information of the initial segmentation image, which is the candidate region in this embodiment. It should be noted that since there are certain differences between the branch detail information obtained in the initial segmentation image and the branch detail information in the original image, it is necessary to filter the branch detail information in the initial segmentation image, that is, to filter the candidate regions region by region to obtain the filtered connected regions. Optionally, the method for filtering connected regions may include: for any candidate region, determining at least one region boundary of the candidate region, and determining whether any region boundary of the candidate region is connected to the boundary of the second processed image. Optionally, if connected, the candidate region is determined as a connected region; otherwise, the candidate region is discarded.

[0095] The effect of obtaining the connected region and thus the target segmented image based on the above method in this embodiment is that it can determine the region that has a connected relationship with the existing segmentation result in this embodiment. That is, the segmented image obtained after splicing the connected region with the existing segmentation result has the branch connectivity information in the original image. In addition, in the process of segmenting the coronary artery image, it avoids the problem of connecting the left and right coronary arteries as a whole, thus improving the accuracy of coronary artery segmentation.

[0096] Of course, other methods can also be used to filter candidate regions in this embodiment to obtain connected regions, and then the connected regions can be fused with the existing segmentation results to obtain the target segmentation image corresponding to the original image. This embodiment does not limit this.

[0097] See examples Figure 5 In the figure, Pn represents the initial segmented image, and Pb represents the second processed image, i.e., the baseline image of the segmentation result. Subtraction is performed on Pn and Pb to obtain candidate regions Pi with branch detail information. Optionally, connected regions of Pi are selected based on preset filtering conditions. See further... Figure 6 The second processed image Pb is fused with the filtered connected regions to obtain the target segmentation image P corresponding to the original image.

[0098] Based on the above implementation, the method for generating the target segmentation image of the original image by splicing the initial segmentation image and the second processed image in this embodiment may further include: splicing the initial segmentation image and the second processed image to obtain a fused processed image; removing unconnected regions in the fused processed image to obtain the target segmentation image of the original image.

[0099] See examples Figure 7 The second processed image Pb and the initial segmented image Pn are subjected to union processing to obtain a segmented image that contains both branch connectivity information and branch detail information. However, the segmented image at this point still includes branch regions that are not connected to the main segmentation result. These branch regions are removed to obtain the target segmented image P corresponding to the original image. It should be noted that the effect of obtaining the target segmented image based on the above method is to reduce the processing steps, thereby shortening the segmentation time and improving the segmentation efficiency while ensuring the accuracy of the segmentation result.

[0100] The technical solution of this invention is based on an initial segmentation model to obtain an initial segmentation image, acquiring branch detail information of the original image, and simultaneously based on an extended segmentation model to obtain an extended segmentation image, acquiring branch connectivity information of the original image. The extended segmentation image is then restored to ensure that the lumen shape of the subsequently obtained target segmentation result is consistent with the lumen shape in the original image. Furthermore, the intersection of the extended segmentation image and the initial segmentation image is performed to obtain a reference image for the segmentation result. Optionally, the reference image is stitched together based on the restored image, so that the obtained segmentation image contains the branch connectivity information of the original image. Optionally, the stitched image is stitched together based on the initial segmentation image, so that the obtained segmentation image contains the detail connectivity information of the original segmentation image, thereby obtaining the target segmentation image corresponding to the original image and improving the segmentation accuracy of the target segmentation image.

[0101] Example 3

[0102] Figure 8 This is a schematic diagram of the structure of an image segmentation device provided in Embodiment 3 of the present invention. Figure 8 As shown, the device includes: a segmented image acquisition module 310, a restored image acquisition module 320, and a target segmented image acquisition module 330; wherein,

[0103] The image segmentation acquisition module 310 obtains the initial segmented image and the extended segmented image of the original image, respectively;

[0104] The restored image acquisition module 320 is used to perform restoration processing on the extended segmented image to obtain a restored image;

[0105] The target segmentation image acquisition module 330 is used to obtain the target segmentation image of the original image based on the initial segmentation image, the extended segmentation image, and the restored image.

[0106] Based on the above embodiments, optionally, the target segmentation image acquisition module 330 includes:

[0107] The first image processing production submodule is used to perform overlap comparison processing on the initial segmented image and the extended segmented image to generate a first processed image;

[0108] The second image processing generation submodule is used to stitch the first processed image and the restored image together to generate the second processed image;

[0109] The target segmentation image generation submodule is used to stitch the initial segmentation image and the second processed image together to generate the target segmentation image of the original image.

[0110] Based on the above embodiments, optionally, the target segmentation image generation submodule includes:

[0111] The image fusion processing unit is used to stitch the initial segmented image and the second processed image together to obtain the fused image;

[0112] The first target segmentation image generation unit is used to remove unconnected regions in the fused image to obtain the target segmentation image of the original image.

[0113] Based on the above embodiments, optionally, the target segmentation image generation submodule includes:

[0114] A candidate region generation unit is used to determine candidate regions based on the initial segmented image and the second processed image;

[0115] A connected region determination unit is used to determine at least one connected region in the candidate region based on the region boundary of the candidate region and the region boundary of the second processed image;

[0116] The second target segmentation image generation unit is used to perform fusion processing based on the connected region and the second processed image to obtain the target segmentation image of the original image.

[0117] Based on the above embodiments, optionally, the restored image acquisition module 320 includes:

[0118] The restored image acquisition submodule is used to obtain preset restoration parameters, and perform restoration processing on the extended segmented image based on the restoration parameters to obtain the restored image of the extended segmented image.

[0119] Based on the above embodiments, optionally, the segmented image acquisition module 320 includes:

[0120] The initial segmentation image acquisition submodule is used to input the original image into the initial segmentation model to obtain the initial segmentation image output by the initial segmentation model;

[0121] An extended segmentation image acquisition module is used to input the original image into the extended segmentation model to obtain the extended segmentation image output by the extended segmentation model.

[0122] Optionally, based on the above embodiments, the device further includes a model training module; wherein the model training module includes:

[0123] The extended label acquisition submodule is used to acquire sample images for training the extended segmentation model, annotate the target regions in the sample images, and perform extended processing on the target regions to obtain extended labels for the target regions.

[0124] The segmentation prediction result acquisition submodule is used to input the sample image into the extended segmentation model to be trained, and obtain the segmentation prediction result output by the extended segmentation model;

[0125] An extended segmentation model acquisition submodule is used to train the extended segmentation model based on the extended labels of the target region and the segmentation prediction results to obtain the extended segmentation model.

[0126] The image segmentation apparatus provided in the embodiments of the present invention can execute the image segmentation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0127] Example 4

[0128] Figure 9 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0129] like Figure 9 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0130] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0131] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as image segmentation methods.

[0132] In some embodiments, the image segmentation method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the image segmentation method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the image segmentation method by any other suitable means (e.g., by means of firmware).

[0133] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0134] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0135] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0136] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0137] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0138] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0139] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0140] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An image segmentation method, characterized in that, include: The original image is obtained, and based on the pre-trained initial segmentation model and extended segmentation model, the initial segmentation image and extended segmentation image of the original image are obtained respectively. The extended segmented image is restored to obtain the restored image; Based on the initial segmented image, the extended segmented image, and the restored image, a target segmented image of the original image is obtained, wherein the initial segmented image is used to determine the branch detail information of the original image, the extended segmented image is used to determine the branch connectivity information of the original image, and the restored image is used to determine the lumen shape information of the original image; The process of obtaining the target segmentation image of the original image based on the initial segmentation image, the expanded segmentation image, and the restored image includes: The initial segmented image and the extended segmented image are subjected to overlap comparison processing to generate a first processed image, wherein the overlap comparison processing is used to determine the intersecting and overlapping regions between the initial segmented image and the extended segmented image; The first processed image and the restored image are stitched together to generate a second processed image; The initial segmented image and the second processed image are stitched together to generate the target segmented image of the original image; The step of stitching the initial segmented image and the second processed image to generate the target segmented image of the original image includes: The initial segmented image and the second processed image are stitched together to obtain a fused image; disconnected regions in the fused image are removed to obtain the target segmented image of the original image; or, Based on the initial segmented image and the second processed image, candidate regions are determined; based on the region boundaries of the candidate regions and the region boundaries of the second processed image, at least one connected region among the candidate regions is determined; wherein, the connected region is determined based on the branch detail information of the candidate regions; based on the connected region and the second processed image, a fusion processing is performed to obtain the target segmented image of the original image.

2. The method according to claim 1, characterized in that, The restoration process of the extended segmented image to obtain the restored image includes: Obtain preset restoration parameters, and perform restoration processing on the extended segmented image based on the restoration parameters to obtain the restored image of the extended segmented image.

3. The method according to claim 1, characterized in that, The process, based on the pre-trained initial segmentation model and extended segmentation model, yields the initial segmented image and extended segmented image of the original image, respectively, including: The original image is input into the initial segmentation model to obtain the initial segmented image output by the initial segmentation model; The original image is input into the extended segmentation model to obtain the extended segmented image output by the extended segmentation model.

4. The method according to claim 1, characterized in that, The training method for the extended segmentation model includes: Obtain sample images for training the extended segmentation model, annotate the target regions in the sample images, and extend the target regions to obtain extended labels for the target regions; The sample image is input into the extended segmentation model to be trained to obtain the segmentation prediction result output by the extended segmentation model; The extended segmentation model is trained based on the extended labels of the target region and the segmentation prediction results to obtain the extended segmentation model.

5. An image segmentation apparatus, characterized in that, include: The segmented image acquisition module is used to acquire the original image and obtain the initial segmented image and the extended segmented image of the original image based on the pre-trained initial segmentation model and extended segmentation model, respectively. The restored image acquisition module is used to perform restoration processing on the extended segmented image to obtain the restored image; The target segmentation image acquisition module is used to obtain a target segmentation image of the original image based on the initial segmentation image, the extended segmentation image, and the restored image, wherein the initial segmentation image is used to determine the branch detail information of the original image, the extended segmentation image is used to determine the branch connectivity information of the original image, and the restored image is used to determine the lumen shape information of the original image. The target segmentation image acquisition module includes: a first processed image generation submodule, used to perform overlap comparison processing on the initial segmentation image and the extended segmentation image to generate a first processed image, wherein the overlap comparison processing is used to determine the intersecting and overlapping regions between the initial segmentation image and the extended segmentation image; a second processed image generation submodule, used to perform stitching processing on the first processed image and the restored image to generate a second processed image; and a target segmentation image generation submodule, used to perform stitching processing on the initial segmentation image and the second processed image to generate a target segmentation image of the original image. The target segmentation image generation submodule includes: A fused image acquisition unit is used to stitch the initial segmented image and the second processed image together to obtain a fused image; a first target segmentation image generation unit is used to remove disconnected regions from the fused image to obtain a target segmentation image of the original image; or, A candidate region generation unit is used to determine candidate regions based on the initial segmented image and the second processed image; a connected region determination unit is used to determine at least one connected region among the candidate regions based on the region boundaries of the candidate regions and the region boundaries of the second processed image; wherein the connected region is determined based on the branch detail information of the candidate regions; a second target segmentation image generation unit is used to perform fusion processing based on the connected regions and the second processed image to obtain the target segmentation image of the original image.

6. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the image segmentation method according to any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the image segmentation method according to any one of claims 1-4.

Images