Patch Segmentation Method, Device and Computer Readable Storage Medium

Abstract

The present application relates to a plaque segmentation method, apparatus and computer-readable storage medium. The method includes: acquiring multi-planar images of a tubular structure, and acquiring straightened images of each tubular branch in the tubular structure; performing plaque segmentation based on the multi-planar images to obtain a plaque segmentation result of the multi-planar images; and performing plaque segmentation based on each straightened image to obtain a plaque segmentation result of a target straightened image; the target straightened image represents the straightened image of the tubular branch with a lesion; fusing the plaque segmentation result of the multi-planar images and the plaque segmentation result of the target straightened image to obtain a plaque segmentation result of the tubular structure. Using this method can improve the accuracy of plaque segmentation.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and particularly to a plaque segmentation method, device, and computer-readable storage medium. Background Art

[0002] With the development of image segmentation technology, image segmentation is increasingly applied in the medical field. Taking the plaque segmentation process of a tubular structure as an example, the plaque area in the tubular structure can be extracted by segmenting the tubular structure.

[0003] During the plaque segmentation process of a tubular structure, in the related art, a plaque segmentation model is mainly used to segment the tubular structure image to obtain the plaque segmentation result of the tubular structure.

[0004] However, due to the large bending degree of the tubular structure, the existing method has the problem of inaccurate plaque segmentation. Summary of the Invention

[0005] Based on this, in view of the above technical problems, it is necessary to provide a plaque segmentation method, device, and computer-readable storage medium that can improve the accuracy of plaque segmentation.

[0006] In a first aspect, this application provides a plaque segmentation method, which includes:

[0007] Obtain multi-plane images of the tubular structure, and obtain straightened images of each tubular branch in the tubular structure;

[0008] Perform plaque segmentation based on the multi-plane images to obtain the plaque segmentation result of the multi-plane images; and perform plaque segmentation based on each straightened image to obtain the plaque segmentation result of the target straightened image; the target straightened image represents the straightened image of the tubular branch with lesions;

[0009] Fuse the plaque segmentation result of the multi-plane images with the plaque segmentation result of the target straightened image to obtain the plaque segmentation result of the tubular structure.

[0010] In one embodiment, performing plaque segmentation based on the multi-plane images to obtain the plaque segmentation result of the multi-plane images includes:

[0011] Obtain at least one lesion area in the tubular structure on the multi-plane images;

[0012] Determine the region of interest on the multi-plane images according to each lesion area;

[0013] Perform plaque segmentation according to each region of interest to obtain the plaque segmentation result of the multi-plane images.

[0014] In one embodiment, determining regions of interest on multi-planar images according to each lesion region includes:

[0015] Regarding each lesion region and the region of a preset external range of each lesion region as the regions of interest on the multi-planar images.

[0016] In one embodiment, performing plaque segmentation according to each region of interest to obtain the plaque segmentation result of the multi-planar image, including:

[0017] Obtaining the detection region corresponding to the region of interest;

[0018] Inputting the detection region corresponding to the region of interest into a multi-planar image segmentation model, and performing plaque segmentation on the detection region through the multi-planar image segmentation model to obtain the plaque segmentation result of the multi-planar image; the multi-planar image segmentation model is trained according to multiple multi-planar sample images with labeled lumens and plaques.

[0019] In one embodiment, obtaining the detection region corresponding to the region of interest includes:

[0020] Obtaining the position information of the vertices in the multi-planar image in the region of interest;

[0021] Based on the position information, determining the background region of the region of interest;

[0022] Regarding the region of interest and the background region as the detection region corresponding to the region of interest.

[0023] In one embodiment, obtaining at least one lesion region in the tubular structure on the multi-planar image includes:

[0024] Performing lesion detection on the straightened images of each tubular branch to obtain the lesion regions on the target straightened image;

[0025] Inverse mapping the lesion regions on the target straightened image onto the multi-planar image to obtain at least one lesion region in the tubular structure on the multi-planar image.

[0026] In one embodiment, performing lesion detection on the straightened images of each tubular branch to obtain the lesion regions on the target straightened image includes:

[0027] Inputting the straightened images of each tubular branch into a preset lesion detection model, and using the lesion detection model to analyze each straightened image to obtain the lesion regions on the target straightened image.

[0028] In one embodiment, performing plaque segmentation based on the straightened images of each tubular branch to obtain the plaque segmentation result of the target straightened image, including:

[0029] Perform lesion detection on the straightened images of each tubular branch to obtain the lesion regions on the target straightened image;

[0030] Perform plaque segmentation on the target straightened image to obtain the plaque segmentation result of the target straightened image.

[0031] In one embodiment, performing plaque segmentation on the target straightened image to obtain the plaque segmentation result of the target straightened image includes:

[0032] Perform plaque segmentation on the target straightened image through a straightened image segmentation model to obtain the plaque segmentation result of the target straightened image; the straightened image segmentation model is trained based on multiple straightened sample images with labeled lumens and plaques.

[0033] In one embodiment, performing plaque segmentation on the target straightened image through a straightened image segmentation model to obtain the plaque segmentation result of the target straightened image includes:

[0034] For any target straightened image, cut the target straightened image along the center line of the target straightened image to obtain multiple straightened reconstruction segments;

[0035] Add position encoding to each straightened reconstruction segment;

[0036] Input each straightened reconstruction segment and the position encoding into the straightened image segmentation model, perform plaque segmentation on each straightened reconstruction segment, and obtain the plaque segmentation result of the target straightened image.

[0037] In one embodiment, adding position encoding to each straightened reconstruction segment includes:

[0038] Obtain the position information of each straightened reconstruction segment on the center line;

[0039] Based on the position information of each straightened reconstruction segment, add position encoding to each straightened reconstruction segment.

[0040] In one embodiment, fusing the plaque segmentation result of the multi-planar image with the plaque segmentation result of the target straightened image to obtain the plaque segmentation result of the tubular structure includes:

[0041] According to the position of the tubular branch with lesions in the tubular structure, inverse map the plaque segmentation result of the target straightened image into the multi-planar image, and fuse the plaque segmentation results of the multi-planar image and the target straightened image according to the corresponding pixel positions to obtain the plaque segmentation result of the tubular structure.

[0042] In one embodiment, obtaining the straightened images of each tubular branch in the tubular structure includes:

[0043] Perform tubular rough segmentation on each tubular branch of the tubular structure in the multi-planar image, and extract the centerline of each tubular branch;

[0044] Generate a straightened image of each tubular branch based on the centerline of each tubular branch.

[0045] In a second aspect, the present application also provides a plaque segmentation device, which includes:

[0046] An acquisition module for acquiring a multi-planar image of the tubular structure and a straightened image of each tubular branch in the tubular structure;

[0047] A segmentation module for performing plaque segmentation based on the multi-planar image to obtain a plaque segmentation result of the multi-planar image; and performing plaque segmentation based on the straightened image of each tubular branch to obtain a plaque segmentation result of the target straightened image; the target straightened image represents the straightened image of the tubular branch with lesions;

[0048] A fusion module for fusing the plaque segmentation result of the multi-planar image with the plaque segmentation result of the target straightened image to obtain a plaque segmentation result of the tubular structure.

[0049] In a third aspect, the present application also provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the content of any one of the plaque segmentation method embodiments in the first aspect above.

[0050] In a fourth aspect, the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the content of any one of the plaque segmentation method embodiments in the first aspect above.

[0051] In a fifth aspect, the present application also provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, it implements the content of any one of the plaque segmentation method embodiments in the first aspect above.

[0052] The above-mentioned plaque segmentation method, apparatus, and computer-readable storage medium obtain multi-planar images of a tubular structure and obtain straightened images of each tubular branch in the tubular structure; perform plaque segmentation based on the multi-planar images to obtain a plaque segmentation result of the multi-planar images; and perform plaque segmentation based on each straightened image to obtain a plaque segmentation result of the target straightened image; fuse the plaque segmentation result of the multi-planar images with the plaque segmentation result of the target straightened image to obtain a plaque segmentation result of the tubular structure. Among them, the target straightened image represents the straightened image of the tubular branch with a lesion. This method starts from two different types of images, namely the multi-planar images of the tubular structure and each straightened image, and performs plaque segmentation on the tubular structure from two different perspectives, straightening and bending, and then fuses the plaque segmentation results from the two different perspectives, which can more comprehensively perform plaque segmentation on the tubular structure and make the obtained plaque segmentation result more accurate. Description of the Drawings

[0053] Figure 1 It is an application environment diagram of the plaque segmentation method in an embodiment;

[0054] Figure 2 It is a schematic flowchart of the plaque segmentation method in an embodiment;

[0055] Figure 3 It is a schematic flowchart of the plaque segmentation method in an embodiment;

[0056] Figure 4 It is a schematic flowchart of the plaque segmentation method in an embodiment;

[0057] Figure 5 It is a schematic flowchart of the plaque segmentation method in an embodiment;

[0058] Figure 6 It is a schematic flowchart of the plaque segmentation method in an embodiment;

[0059] Figure 7 It is a schematic flowchart of the plaque segmentation method in an embodiment;

[0060] Figure 8 It is a schematic flowchart of the plaque segmentation method in an embodiment;

[0061] Figure 9 It is a schematic flowchart of the plaque segmentation method in an embodiment;

[0062] Figure 10 It is a schematic flowchart of the plaque segmentation method in an embodiment;

[0063] Figure 11 It is a schematic flowchart of the plaque segmentation method in an embodiment;

[0064] Figure 12 It is a schematic flowchart of a plaque segmentation method in an embodiment;

[0065] Figure 13 It is a structural block diagram of a plaque segmentation device in an embodiment. Detailed implementation manners

[0066] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0067] Before introducing the technical solutions of the present application in detail, the background technology of the present application will be briefly introduced first.

[0068] The plaque segmentation of the tubular structure plays an important auxiliary role in judging the stenosis rate of the tubular structure and plaque analysis. In the process of plaque segmentation of the tubular structure, the plaque segmentation process of coronary blood vessels is taken as an example for illustration. The coronary blood vessels are relatively thin, with many branches and a large degree of curvature. Small calcified plaques and non-calcified plaques with unclear low-density shadows are easily missed during the segmentation process. In addition, the boundaries of non-calcified plaques are also very difficult to identify. The above problems will all affect the judgment of the number of plaques on the coronary blood vessels and the degree of lumen occupation by the plaques.

[0069] When performing plaque segmentation on coronary blood vessels, multi-planar reconstruction (MPR) images of coronary blood vessels and straightened curved planar reconstruction (SCPR) images generated along the center line of the coronary blood vessels are usually used. The SCPR image is obtained by roughly segmenting the MPR image of the blood vessel. Among them, certain deformation and distortion will occur during the generation of the SCPR image, but the SCPR image can more intuitively display the morphological changes of a complete blood vessel, and the SCPR image can more accurately locate and judge the position and shape of the plaque. The MPR image can supplement the inaccurate detailed information in the SCPR image and the information of the remaining blood vessels and tissues in more local areas, so as to more accurately judge and analyze the boundary of the plaque.

[0070] In the prior art, mainly a plaque segmentation model is used to segment plaques for a single MPR image or a single SCPR image to obtain the plaque segmentation result. When segmenting plaques for an SCPR image, since the SCPR image can present a complete blood vessel, the vessel's alignment, the positions where bifurcations occur, the changes in the lumen shape, and the wall characteristics of the entire blood vessel can be more intuitively observed on the SCPR image. However, during the process of straightening the MPR image to obtain the SCPR image and inverse mapping the segmentation result of the SCPR image back to the MPR image, certain errors will occur, resulting in inaccurate plaque segmentation results. Additionally, when segmenting plaques for an MPR image, directly segmenting plaques on the MPR image using the plaque segmentation model can, to a certain extent, avoid the errors generated during the image conversion process. However, due to the limitations of the deep learning network structure itself, the plaque segmentation model can only obtain information within a relatively small neighboring region, making it difficult to fully utilize the image information and learn the characteristics of a complete blood vessel, thus leading to inaccurate plaque segmentation results.

[0071] To address the above technical problems, the present application proposes a plaque segmentation method. This method can fuse the plaque segmentation results of the SCPR image and the MPR image at the decision-making level, avoiding the errors caused by multiple image conversions and enabling the plaque segmentation model to obtain the characteristic information of complete blood vessel branches, thereby improving the segmentation accuracy of the plaque segmentation model.

[0072] The plaque segmentation method provided in the embodiments of the present application can be applied to an application environment as Figure 1 shown. For example, the computer device can be a server, a personal computer, a laptop computer, a smart phone, a tablet computer, a smart mobile phone, etc. The computer device may include a processor, a memory, and a network interface connected via a system bus or wirelessly. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device may include a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data during the plaque segmentation process. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, it implements a plaque segmentation method. Among them, the computer device can be implemented by an independent computer device or a computer device cluster composed of multiple computer devices. It should be noted that the memory of the computer device is not limited to the above-mentioned memory and may also include high-speed random access memory, volatile solid-state memory, etc. Additionally, the composition architecture of the computer device is not limited to the above situation, and some components can also be added or omitted.

[0073] In one embodiment, as Figure 2 shown, a plaque segmentation method is provided. Taking the computer device to which this method is applied Figure 1 as an example for illustration, the method includes the following steps:

[0074] S201. Obtain multi-planar images of the tubular structure, and obtain straightened images of each tubular branch in the tubular structure.

[0075] Among them, the tubular structure may refer to a structure in the shape of a tube or a sac. For example, the tubular structure may be blood vessels, stomach, intestine, bladder, fallopian tubes, etc. Since this application is to segment plaques on the tubular structure, and plaques are generally formed on blood vessels, therefore, the tubular structure may be various types of blood vessels. For example, it may be coronary artery vessels, intracranial or extracranial artery vessels, peripheral artery vessels, renal artery and subclavian artery vessels, aortic vessels, and iliofemoral artery vessels, etc. The above multi-planar images refer to two-dimensional images on different planes obtained by converting three-dimensional image data. For example, the three-dimensional image data may be Computed Tomography (CT) images, Magnetic Resonance Imaging (MRI) images, etc.

[0076] In this embodiment, when the multi-planar images of multiple tubular structures and the straightened images of the corresponding tubular branches are stored in the image database, the computer device can obtain the multi-planar images of the tubular structure from the image database according to the identification information of the tubular structure, and obtain the straightened images of each tubular branch in the tubular structure. Or, the computer device can use multi-planar reconstruction technology to perform multi-planar reconstruction on the images of the tubular structure to obtain the multi-planar images of the tubular structure. In addition, the computer device can also straighten each tubular branch in the multi-planar images to obtain the straightened images of each tubular branch in the tubular structure. This embodiment does not limit the methods for obtaining the multi-planar images of the tubular structure and the straightened images of each tubular branch in the tubular structure.

[0077] S202. Perform plaque segmentation based on the multi-planar images to obtain the plaque segmentation result of the multi-planar images; and perform plaque segmentation based on each straightened image to obtain the plaque segmentation result of the target straightened image; the target straightened image represents the straightened image of the tubular branch with lesions.

[0078] In this embodiment, the computer device may input the multi-plane image into the trained plaque segmentation model, and segment the plaques in the multi-plane image through the plaque segmentation model to obtain the plaque segmentation result of the multi-plane image. At the same time, the computer device may identify lesions in each straightened image through the lesion recognition model to obtain the lesion recognition result of each straightened image. Then, the straightened image with lesions is input into the trained plaque segmentation model, and the plaque segmentation model segments the plaques in the straightened image with lesions to obtain the plaque segmentation result of the target straightened image.

[0079] Optionally, the computer device may also determine the boundaries of the plaques based on the image parameter information of each pixel point in the multi-plane image. Then, based on the plaque boundaries in the multi-plane image, the plaques in the multi-plane image are segmented to obtain the plaque segmentation result of the multi-plane image. At the same time, the computer device may also determine the plaque boundaries in each target straightened image based on the image parameter information of each target straightened image. Then, based on the plaque boundaries in each target straightened image, the plaques in each target straightened image are segmented. The way of plaque segmentation in this embodiment is not limited.

[0080] S203, fuse the plaque segmentation result of the multi-plane image and the plaque segmentation result of the target straightened image to obtain the plaque segmentation result of the tubular structure.

[0081] In this embodiment, the computer device may convert the plaque segmentation result of the target straightened image onto the multi-plane image, and match the plaque segmentation result of the target straightened image with the plaque segmentation result of the multi-plane image to obtain the plaque segmentation result of the tubular structure. Alternatively, the computer device may also determine the plaque segmentation result of the tubular structure based on the weights corresponding to the plaque segmentation result of the target straightened image and the weights corresponding to the plaque segmentation result of the multi-plane image, based on the weights of the two plaque segmentation results.

[0082] In the above plaque segmentation method, a multi-plane image of a tubular structure is obtained, and straightened images of each tubular branch in the tubular structure are obtained; plaque segmentation is performed based on the multi-plane image to obtain the plaque segmentation result of the multi-plane image; and plaque segmentation is performed based on each straightened image to obtain the plaque segmentation result of the target straightened image; the plaque segmentation result of the multi-plane image and the plaque segmentation result of the target straightened image are fused to obtain the plaque segmentation result of the tubular structure. Among them, the target straightened image represents the straightened image of the tubular branch with lesions. This method starts from two different types of images, namely the multi-plane image of the tubular structure and each straightened image, and performs plaque segmentation on the tubular structure from two different perspectives, straightening and bending, and then fuses the plaque segmentation results from the two different perspectives, which can more comprehensively perform plaque segmentation on the tubular structure and make the obtained plaque segmentation result more accurate.

[0083] Based on the above embodiments, this embodiment introduces and explains the relevant content of step S202 in Figure 2 Figure 2 "Performing plaque segmentation on the multi-plane image to obtain the plaque segmentation result of the multi-plane image".

[0084] As Figure 3 shown, as a non-limiting example, the above step S202 may include the following content:

[0085] S301, obtaining at least one lesion region in the tubular structure on the multi-plane image.

[0086] Among them, the lesion region refers to the region in the tubular structure of the multi-plane image that is affected by the lesion.

[0087] Optionally, the computer device may input the multi-plane image into the lesion detection model, and detect the lesion region in the multi-plane image through the lesion detection model to obtain at least one lesion region in the tubular structure on the multi-plane image. Or, the computer device may obtain the image parameter information of the tubular structure on the multi-plane image, and based on this image parameter information, determine at least one lesion region in the tubular structure on the multi-plane image. Or, in order to ensure the accuracy of lesion detection, the computer device may first perform lesion detection on each straightened image to obtain the lesion region of the target straightened image. Then, according to the mapping relationship between each straightened image and the multi-plane image, determine the position of the lesion region in the tubular structure on the multi-plane image. This embodiment does not limit the method for obtaining at least one lesion region in the tubular structure on the multi-plane image.

[0088] S302, determining the region of interest on the multi-plane image according to each lesion region.

[0089] Among them, the region of interest includes the region where the lesion exists and the region within a preset range around the lesion.

[0090] In this embodiment, when multiple lesion regions on the multi-plane image are obtained, for any one lesion region, the computer device may perform an expansion process on each lesion region according to a preset expansion multiple. And determine the expanded lesion region as the region of interest on the multi-plane image.

[0091] S303, performing plaque segmentation according to each region of interest to obtain the plaque segmentation result of the multi-plane image.

[0092] In this embodiment, after obtaining the regions of interest corresponding to each lesion region, for any one lesion region, the computer device can directly perform a segmentation operation on the boundary of the region of interest to obtain the plaque segmentation result of the multi-planar image. Alternatively, the computer device can screen out the regions with a higher plaque probability from each region of interest, and perform a segmentation operation on the regions with a higher plaque probability to obtain the plaque segmentation result of the multi-planar image.

[0093] In the above plaque segmentation method, at least one lesion region in the tubular structure on the multi-planar image is obtained, regions of interest on the multi-planar image are determined according to each lesion region, and plaque segmentation is performed according to each region of interest to obtain the plaque segmentation result of the multi-planar image. This method can accurately obtain the regions of interest by obtaining the lesion regions in the tubular structure on the multi-planar image and based on each lesion region, so that plaque segmentation can be performed on the regions of interest, and the plaque segmentation result of the multi-planar image can be accurately obtained.

[0094] Based on the above embodiment, this embodiment introduces and explains the relevant content of Figure 3 step S302 "determine the regions of interest on the multi-planar image according to each lesion region". As a non-limiting example, the above step S302 may include the following content: each lesion region and the regions within the externally preset range of each lesion region are used as the regions of interest on the multi-planar image.

[0095] In this embodiment, for any one lesion region, since the range included in the lesion region is small, when segmenting the lesion region, the segmentation range needs to be expanded. Therefore, the computer device can use the lesion region and the regions within the externally preset range of the lesion region as the regions of interest on the multi-planar image. Among them, the externally preset range of the lesion region can be selected according to the actual situation.

[0096] In the above plaque segmentation method, each lesion region and the regions within the externally preset range of each lesion region are used as the regions of interest on the multi-planar image. In the process of obtaining the regions of interest, this method obtains the lesion region and the regions within the externally preset range of the lesion region, and uses the region obtained by combining the two regions as the region of interest, and avoids the problem of inaccurate plaque segmentation by expanding the lesion region.

[0097] Based on the above embodiment, this embodiment introduces and explains the relevant content of Figure 3 step S303 "perform plaque segmentation according to each region of interest to obtain the plaque segmentation result of the multi-planar image". As Figure 4 shown, as a non-limiting example, the above step S303 may include the following content:

[0098] S401. Obtain the detection region corresponding to the region of interest.

[0099] Among them, the detection region refers to the region in the multi-planar image where plaques may exist.

[0100] In this embodiment, after obtaining the region of interest in the multi-planar image, the computer device can perform an expansion process on the region of interest and use the expanded region of interest as the detection region. Alternatively, the computer device can obtain the circumscribed rectangle of the region of interest and use all the regions within the circumscribed rectangle as the detection region corresponding to the region of interest. This embodiment does not limit the method for obtaining the detection region corresponding to the region of interest.

[0101] S402. Input the detection region corresponding to the region of interest into the multi-planar image segmentation model, and perform plaque segmentation on the detection region through the multi-planar image segmentation model to obtain the plaque segmentation result of the multi-planar image; the multi-planar image segmentation model is trained based on multiple multi-planar sample images with labeled lumens and plaques.

[0102] In this embodiment, the computer device can use the detection region corresponding to the region of interest as an input signal and input it into the multi-planar image segmentation model. The multi-planar image segmentation model analyzes the detection region to obtain the plaque probability corresponding to each pixel point in the region of interest in the detection region. And based on the plaque probability corresponding to each pixel point, plaque segmentation is performed to obtain the plaque segmentation result of the multi-planar image.

[0103] It should be noted that the size of the input image that the multi-planar image segmentation model can accept is limited by the memory. Without losing the original image information, it is impossible to input the complete detection region into the multi-planar image segmentation model at one time. That is, before inputting the detection region into the multi-planar image segmentation model, the detection region can only be cut into multiple sub-detection regions of appropriate size, so that the multi-planar image segmentation model can analyze the multiple sub-detection regions separately to obtain the plaque segmentation probability map of the multi-planar image, and use this plaque segmentation probability map as the plaque segmentation result of the multi-planar image.

[0104] In the above plaque segmentation method, the detection region corresponding to the region of interest is obtained; the detection region corresponding to the region of interest is input into the multi-planar image segmentation model, and plaque segmentation is performed on the detection region through the multi-planar image segmentation model to obtain the plaque segmentation result of the multi-planar image. The multi-planar image segmentation model in this method is trained based on multiple multi-planar sample images with labeled lumens and plaques. The multi-planar image segmentation model trained through multiple labeled samples is more accurate. Using the multi-planar image segmentation model to perform plaque segmentation on the detection region corresponding to the region of interest, the obtained plaque segmentation result of the multi-planar image will also be more accurate.

[0105] Based on the above embodiments, this embodiment describes the relevant content of step S401, "Obtain the detection region corresponding to the region of interest" in Figure 4 . As a non-limiting example, as shown in Figure 5 , the above step S401 may include the following content:

[0106] S501, obtain the position information of the vertices in the multi-planar image within the region of interest.

[0107] In this embodiment, after obtaining the region of interest in the multi-planar image, the computer device can determine the pixel points where the vertices in the region of interest are located, and based on the pixel points where the vertices are located, determine the position information of the vertices in the multi-planar image.

[0108] S502, based on the position information, determine the background region of the region of interest.

[0109] In this embodiment, the computer device can, based on the position information, select a suitable bounding box that can completely enclose the region of interest and leave a suitable distance from the vertices of the region of interest. The other regions in the bounding box except the region of interest are used as the background region of the region of interest.

[0110] S503, determine the region of interest and the background region as the detection region corresponding to the region of interest.

[0111] In this embodiment, after obtaining the background region of the region of interest according to the position information of the vertices of the region of interest, the computer device can use the region composed of the background region and the region of interest as the detection region corresponding to the region of interest. Alternatively, after the computer device obtains the bounding box corresponding to the region of interest, it can use all the regions within the bounding box as the detection region corresponding to the region of interest.

[0112] In the above plaque segmentation method, obtain the position information of the vertices in the multi-planar image within the region of interest; based on the position information, determine the background region of the region of interest; determine the region of interest and the background region as the detection region corresponding to the region of interest. By obtaining the vertices of the region of interest, this method can accurately determine the position information of the vertices in the multi-planar image. Thus, based on this position information, the background information of the region of interest can be accurately obtained; and then based on this background information, the detection region corresponding to the region of interest can be accurately determined.

[0113] Based on the above embodiments, this embodiment describes the relevant content of step S301, "Obtain at least one lesion region in the tubular structure on the multi-planar image" in Figure 3 . As shown in Figure 6As shown, as a non-limiting example, the above step S301 may include the following content:

[0114] S601. Perform lesion detection on the straightened images of each tubular branch to obtain the lesion regions on the target straightened image.

[0115] In this embodiment, for the straightened image of any one tubular branch, the computer device may input the straightened image of the tubular branch into the lesion detection model, and perform lesion detection on the straightened image through the lesion detection model to obtain the lesion regions on the target straightened image. Alternatively, the computer device may also determine the lesion regions on the straightened image according to the parameter information of each pixel point in each straightened image.

[0116] S602. Inverse map the lesion regions on the target straightened image to the multi-planar image to obtain at least one lesion region in the tubular structure on the multi-planar image.

[0117] In this embodiment, the computer device may obtain the position information of the center line in the multi-planar image, and obtain the relative position information between each pixel point in the target straightened image and the center line. Based on the position information and relative position information of the center line, inverse map the lesion regions on the target straightened image to the corresponding regions on the multi-planar image. In this way, multiple lesion regions in the tubular structure on the multi-planar image can be obtained.

[0118] In the above plaque segmentation method, perform lesion detection on the straightened images of each tubular branch to obtain the lesion regions on the target straightened image; inverse map the lesion regions on the target straightened image to the multi-planar image to obtain at least one lesion region in the tubular structure on the multi-planar image. By performing lesion detection on the straightened image of each tubular branch, this method can avoid the situation where the lesion regions are blocked when the tubular structure is in a curved state, and more accurately obtain the lesion regions on the target straightened image. And inverse mapping the lesion regions to the multi-planar image can more accurately obtain the lesion regions on the multi-planar image.

[0119] Based on the above embodiments, this embodiment introduces and explains the relevant content of Figure 6 step S601 in "

[0120] In this embodiment, for the straightened image of any tubular branch, the computer device may input the straightened image of the tubular branch into a preset lesion detection model, and perform lesion detection on the straightened image through the lesion detection model. If there is a lesion area on the straightened image, the straightened image is determined as the target straightened image, and the lesion area on the target straightened image is obtained. If there is no lesion area on the straightened image, there is no need to pay attention to the straightened image.

[0121] In the above plaque segmentation method, the straightened images of each tubular branch are input into a preset lesion detection model, and the lesion detection model is used to analyze each straightened image to obtain the lesion area on the target straightened image. This method uses the preset lesion detection model to perform lesion detection on the straightened image of each tubular branch, and can quickly and accurately determine whether there is a lesion area in each straightened image. For the target straightened image with a lesion area, the lesion area on the target straightened image can be accurately obtained.

[0122] Based on the above embodiment, this embodiment introduces and explains the relevant content of Figure 2 step S202 in " Figure 7 Performing plaque segmentation based on the straightened images of each tubular branch to obtain the plaque segmentation result of the target straightened image". As

[0123] S701, performing lesion detection on the straightened images of each tubular branch to obtain the lesion area on the target straightened image.

[0124] In this embodiment, for the straightened image of any tubular branch, the computer device may input the straightened image of the tubular branch into a preset lesion detection model, and perform lesion detection on the straightened image through the lesion detection model. If there is a lesion area on the straightened image, the straightened image is determined as the target straightened image, and the lesion area on the target straightened image is obtained.

[0125] S702, performing plaque segmentation on the target straightened image to obtain the plaque segmentation result of the target straightened image.

[0126] In this embodiment, the computer device may use each target straightened image as the input information of the plaque segmentation model, and segment the plaques in each target straightened image according to the plaque segmentation model to obtain the plaque segmentation result of the target straightened image. Alternatively, the computer device may also obtain the outer contour of the lesion area on the target straightened image, and based on the outer contour, perform plaque segmentation on the target straightened image to obtain the plaque segmentation result of the target straightened image.

[0127] In the above plaque segmentation method, the straightened images of each tubular branch are subjected to lesion detection to obtain the lesion areas on the target straightened image, and the plaque segmentation of the target straightened image is performed to obtain the plaque segmentation result of the target straightened image. Through the lesion detection process, this method can accurately screen out the target straightened images with lesions from the straightened images of each tubular branch. And by performing plaque segmentation on the target straightened images with lesions, more accurate plaque segmentation results can be obtained.

[0128] Based on the above embodiments, this embodiment introduces and explains the relevant content of Figure 7 step S702 in "performing plaque segmentation on the target straightened image to obtain the plaque segmentation result of the target straightened image". As a non-limiting example, the above step S702 may include the following content: performing plaque segmentation on the target straightened image through a straightened image segmentation model to obtain the plaque segmentation result of the target straightened image; the straightened image segmentation model is trained based on multiple straightened sample images with annotated lumens and plaques.

[0129] In this embodiment, before performing plaque segmentation on the target straightened image, the computer device can use multiple straightened sample images with annotated lumens and plaques to train the initial straightened image segmentation model until the preset training conditions are met, obtaining the straightened image segmentation model. During the process of performing plaque segmentation on the target straightened image, for any target straightened image, the computer device can input the target straightened image into the plaque segmentation model, and the plaque segmentation model can segment the plaques in the target straightened image to obtain the plaque segmentation result of the target straightened image.

[0130] In the above plaque segmentation method, plaque segmentation is performed on the target straightened image through a straightened image segmentation model to obtain the plaque segmentation result of the target straightened image. The straightened image segmentation model in this method is trained based on multiple straightened sample images with annotated lumens and plaques. By training the straightened image segmentation model with multiple annotated sample images, the obtained straightened image segmentation model has higher accuracy. Using this straightened image segmentation model to segment the target straightened image, the obtained plaque segmentation result also has higher accuracy.

[0131] Based on the above embodiments, this embodiment introduces and explains the relevant content of "performing plaque segmentation on the target straightened image through a straightened image segmentation model to obtain the plaque segmentation result of the target straightened image" in the above embodiments. As Figure 8 shown, as a non-limiting example, the above process may include the following content:

[0132] S801, for any target straightened image, cut the target straightened image along the center line of the target straightened image to obtain a plurality of straightened reconstruction segments.

[0133] Among them, the center line of the target straightened image refers to the center line of the tubular structure in the target straightened image. The center line of the tubular structure in the target straightened image is determined according to the center line of the corresponding tubular branch in the multi-planar image. For example, when the target straightened image is the straightened image of the first tubular branch, the coordinates of the points on the center line of the target straightened image and the points on the center line of the first tubular branch in the world coordinate system are the same, that is, the center line of the target straightened image is the center line of the first tubular branch in the multi-planar image.

[0134] In this embodiment, since the length of the straightened image of a complete blood vessel in the Z-axis is much larger than its length in the X-axis and Y-axis, the size of the input image that the straightened image segmentation model can accept is limited by memory. Without losing the original image information, it is impossible to input the complete straightened image into the straightened image segmentation model at one time. To make full use of the image information, a deep learning network with an attention mechanism such as a transformer is used to replace the convolutional neural network, so that the straightened image segmentation model can better utilize the information in the straightened reconstruction segments of non-adjacent domains. Therefore, before using the straightened image segmentation model to perform plaque segmentation on the target straightened image, the computer device can cut the target straightened image along the center line into an appropriate size and then obtain the information of a certain segment in the straightened image, that is, obtain multiple straightened reconstruction segments. It should be noted that the length of each straightened reconstruction segment can be the same or different.

[0135] S802, Add positional encoding to each straightened reconstruction segment.

[0136] In this embodiment, for any straightened reconstruction segment, the computer device can obtain the position information of any position point on the straightened reconstruction segment and obtain the position information of the corresponding position points of other straightened reconstruction segments. Based on multiple position information, determine the positional encoding corresponding to each straightened reconstruction segment. And add the obtained positional encoding to the corresponding straightened reconstruction segment.

[0137] S803, Input each straightened reconstruction segment and the positional encoding into the straightened image segmentation model, perform plaque segmentation on each straightened reconstruction segment, and obtain the plaque segmentation result of the target straightened image.

[0138] In this embodiment, when each straightened reconstruction segment and the positional encoding are obtained, the computer device can use each straightened reconstruction segment and the positional encoding as input signals and input them into the straightened image segmentation model. Use the straightened image segmentation model to analyze each straightened reconstruction segment, determine the plaque segmentation probability map on the target straightened image, and determine the plaque segmentation probability map as the plaque segmentation result of the target straightened image.

[0139] In the above plaque segmentation method, for any target straightened image, the target straightened image is sliced along the center line of the target straightened image to obtain a plurality of straightened reconstruction segments; position encodings are added to each of the straightened reconstruction segments; each of the straightened reconstruction segments and the position encodings are input into a straightened image segmentation model to perform plaque segmentation on each of the straightened reconstruction segments, so as to obtain the plaque segmentation result of the target straightened image. This method slices the target straightened image and performs position encoding on each of the sliced straightened reconstruction segments. In the process of using the straightened image segmentation model for plaque segmentation, it will not be restricted by memory, reducing the waiting time due to memory limitations and improving the efficiency of the straightened image segmentation model for plaque segmentation.

[0140] Based on the above embodiments, this embodiment introduces and explains the relevant content of Figure 8 step S802 "adding position encodings to each of the straightened reconstruction segments" in

[0141] S901, obtaining the position information of each of the straightened reconstruction segments on the center line.

[0142] In this embodiment, each target straightened image includes a plurality of straightened reconstruction segments obtained by slicing. For each straightened reconstruction segment, since there is no overlapping area between adjacent straightened reconstruction segments, the computer device can obtain the position information of any point on the center line of this straightened reconstruction segment, and use the position information of this point as the position information of this straightened reconstruction segment on the center line.

[0143] S902, adding position encodings to each of the straightened reconstruction segments based on the position information of each of the straightened reconstruction segments.

[0144] In this embodiment, since the dimension of the position information of each of the straightened reconstruction segments is one-dimensional. For example, the position of each of the straightened reconstruction segments may include the position in the horizontal dimension or the position in the vertical direction. Therefore, the computer device can perform a sorting operation on each of the straightened reconstruction segments according to the position information. Then, in the order of arrangement, position encodings are added to each of the straightened reconstruction segments in turn.

[0145] In the above plaque segmentation method, the position information of each of the straightened reconstruction segments on the center line is obtained; position encodings are added to each of the straightened reconstruction segments based on the position information of each of the straightened reconstruction segments. This method can accurately determine the position of each of the straightened reconstruction segments based on the position information of each of the straightened reconstruction segments on the center line, so that position encodings can be added to each of the straightened reconstruction segments more accurately.

[0146] Based on the above embodiments, this embodiment is about Figure 2The relevant content of step S203 in "Fusing the patch segmentation results of the multi-planar image and the patch segmentation results of the target straightened image to obtain the patch segmentation results of the tubular structure" is introduced and explained. As a non-limiting example, the above step S203 may include the following content: According to the positions of the tubular branches with lesions in the tubular structure, the patch segmentation results of the target straightened image are inversely mapped to the multi-planar image, and the patch segmentation results of the multi-planar image and the target straightened image are fused according to the corresponding pixel positions to obtain the patch segmentation results of the tubular structure.

[0147] In this embodiment, after obtaining the patch segmentation results of the target straightened image, the computer device can inversely map the patch segmentation results of the target straightened image to the multi-planar image according to the position information of the center line in the multi-planar image and the relative position information of each pixel point in the target straightened image to the center line. Then, the weights learned by the model are used to fuse the patch segmentation results of the multi-planar image and the target straightened image to obtain the probability map of the patch segmentation results on the multi-planar image. Based on the positions of the tubular branches with lesions in the tubular structure, the probability map of the patch segmentation results on the multi-planar image can be classified to obtain the patch segmentation results of the tubular structure on the multi-planar image. It should be noted that since the straightened image segmentation model inputs multiple straightened reconstruction segments in the complete target straightened image, the straightened image segmentation model will perform multiple inferences on each straightened reconstruction segment to obtain multiple patch segmentation results. In the process of inversely mapping the patch segmentation results of the target straightened image to the multi-planar image, if a certain pixel point in the multi-planar image receives multiple patch segmentation results inversely mapped back from the target straightened image, the average value of the multiple patch segmentation results is taken as the final patch segmentation result of its target straightened image.

[0148] In the above patch segmentation method, according to the positions of the tubular branches with lesions in the tubular structure, the patch segmentation results of the target straightened image are inversely mapped to the multi-planar image, and the patch segmentation results of the multi-planar image and the target straightened image are fused according to the corresponding pixel positions to obtain the patch segmentation results of the tubular structure. This method inversely maps the patch segmentation results of the straightened image to the multi-planar image and then fuses the patch segmentation results at two different angles, which can more comprehensively perform patch segmentation on the tubular structure and make the obtained patch segmentation results more accurate.

[0149] On the basis of the above embodiment, this embodiment is an introduction and explanation of Figure 2 the relevant content of step S201 in "Obtaining the straightened images of each tubular branch in the tubular structure". As Figure 10 shown, as a non-limiting example, the above step S201 may include the following content:

[0150] S1001. Coarsely segment each tubular branch of the tubular structure in the multi-planar image, and extract the centerline of each tubular branch.

[0151] In this embodiment, the computer device can use a segmentation algorithm to coarsely segment each tubular branch of the tubular structure in the multi-planar image to obtain a mask for the coarse tubular segmentation. Analyze the mask for the coarse tubular segmentation to extract the position information of the centerline of each tubular branch in the tubular structure. And based on this position information, determine the centerline of each tubular branch. Among them, the segmentation algorithm can be a threshold segmentation method, a neural network model, etc.

[0152] S1002. Generate a straightened image for each tubular branch based on the centerline of each tubular branch.

[0153] In this embodiment, after obtaining the centerline of each tubular branch, the computer device can use curvature smoothing operation and frame smoothing operation to straighten the centerline of each tubular branch to obtain a corresponding straightened image for each tubular branch.

[0154] In the above plaque segmentation method, each tubular branch of the tubular structure in the multi-planar image is coarsely segmented to extract the centerline of each tubular branch; a straightened image for each tubular branch is generated based on the centerline of each tubular branch. This method can accurately extract the centerline of each tubular branch by coarsely segmenting the tubular branches in the multi-planar image. Thus, a straightened image for each tubular branch can be accurately generated based on this centerline.

[0155] As a specific embodiment of the present application, as Figure 11 shown, the plaque segmentation method includes:

[0156] S1101. Obtain a multi-planar image of the tubular structure;

[0157] S1102. Coarsely segment each tubular branch of the tubular structure in the multi-planar image, and extract the centerline of each tubular branch;

[0158] S1103. Generate a straightened image for each tubular branch based on the centerline of each tubular branch;

[0159] S1104. Detect lesions in the straightened image of each tubular branch to obtain the lesion area on the target straightened image;

[0160] S1105. Input the straightened image of each tubular branch into a preset lesion detection model, and use the lesion detection model to analyze each straightened image to obtain the lesion area on the target straightened image;

[0161] S1106, take each lesion area and the area of the externally preset range of each lesion area as the region of interest on the multi-planar image;

[0162] S1107, obtain the position information of the vertices in the multi-planar image in the region of interest;

[0163] S1108, determine the background area of the region of interest based on the position information;

[0164] S1109, determine the region of interest and the background area as the detection area corresponding to the region of interest;

[0165] S1110, input the detection area corresponding to the region of interest into the multi-planar image segmentation model, and perform plaque segmentation on the detection area through the multi-planar image segmentation model to obtain the plaque segmentation result of the multi-planar image;

[0166] S1111, perform lesion detection on the straightened images of each tubular branch to obtain the lesion area on the target straightened image;

[0167] S1112, for any one target straightened image, cut the target straightened image along the center line of the target straightened image to obtain a plurality of straightened reconstruction segments;

[0168] S1113, obtain the position information of each straightened reconstruction segment on the center line;

[0169] S1114, add position encoding to each straightened reconstruction segment based on the position information of each straightened reconstruction segment;

[0170] S1115, input each straightened reconstruction segment and the position encoding into the straightened image segmentation model, and perform plaque segmentation on each straightened reconstruction segment to obtain the plaque segmentation result of the target straightened image;

[0171] S1116, according to the position of the tubular branch with lesions in the tubular structure, inverse map the plaque segmentation result of the target straightened image into the multi-planar image, and fuse the plaque segmentation results of the multi-planar image and the target straightened image according to the corresponding pixel positions to obtain the plaque segmentation result of the tubular structure.

[0172] Figure 12The figure is a schematic flowchart of a plaque segmentation method, taking a vascular tubular structure as an example for illustration. The plaque segmentation method includes: S1201: Obtain multi-planar images of blood vessels; S1202: Coarsely segment the blood vessels in the multi-planar images to obtain the centerlines of the blood vessels, so as to generate straightened images of blood vessels corresponding to each blood vessel branch; S1203: Detect lesions on each straightened blood vessel image to obtain straightened blood vessel images with lesions; S1204: Segment plaques on the straightened blood vessel images with lesions to obtain plaque segmentation probability maps of each straightened blood vessel image; S1205: Inversely map the straightened blood vessel images with lesions into the multi-planar images to obtain the lesion regions in the multi-planar images; S1206: Determine the regions of interest in the multi-planar images based on the lesion regions in the multi-planar images; S1207: Segment plaques on the regions of interest in the multi-planar images to obtain plaque segmentation probability maps of the multi-planar images; S1208: Inversely map the plaque segmentation probability maps of each straightened blood vessel image onto the multi-planar images, and fuse the plaque segmentation probability maps of each straightened blood vessel image with the plaque segmentation probability maps of the multi-planar images. Based on the lesion positions in the straightened blood vessel images with lesions, obtain the plaque segmentation results of the multi-planar images.

[0173] It should be understood that although the steps in the flowcharts involved in the above embodiments are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0174] Based on the same inventive concept, an embodiment of the present application also provides a plaque segmentation device for implementing the above-mentioned plaque segmentation method. The solution provided by this device to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the plaque segmentation device provided below can refer to the limitations on the plaque segmentation method in the above text, and will not be repeated here.

[0175] In one embodiment, as Figure 13 shown, a plaque segmentation device is provided, including: an acquisition module 11, a segmentation module 12, and a fusion module 13, where:

[0176] The acquisition module 11 is used to obtain multi-planar images of a tubular structure and obtain straightened images of each tubular branch in the tubular structure;

[0177] A segmentation module 12, configured to perform plaque segmentation based on multi-plane images to obtain the plaque segmentation results of the multi-plane images; and perform plaque segmentation based on the straightened images of each tubular branch to obtain the plaque segmentation results of the target straightened image; the target straightened image represents the straightened image of the tubular branch with lesions.

[0178] A fusion module 13, configured to fuse the plaque segmentation results of the multi-plane images with the plaque segmentation results of the target straightened image to obtain the plaque segmentation results of the tubular structure.

[0179] In one embodiment, the above-mentioned segmentation module includes: an acquisition unit, a determination unit, and a first segmentation unit, where:

[0180] The acquisition unit is configured to acquire at least one lesion area in the tubular structure on the multi-plane image.

[0181] The determination unit is configured to determine the region of interest on the multi-plane image according to each lesion area.

[0182] The first segmentation unit is configured to perform plaque segmentation according to each region of interest to obtain the plaque segmentation results of the multi-plane image.

[0183] In one embodiment, the above-mentioned determination unit is further configured to use each lesion area and the area within the externally preset range of each lesion area as the region of interest on the multi-plane image.

[0184] In one embodiment, the above-mentioned first segmentation unit is further configured to acquire the detection region corresponding to the region of interest; input the detection region corresponding to the region of interest into the multi-plane image segmentation model, and perform plaque segmentation on the detection region through the multi-plane image segmentation model to obtain the plaque segmentation results of the multi-plane image; the multi-plane image segmentation model is trained according to multiple multi-plane sample images with labeled lumens and plaques.

[0185] In one embodiment, the above-mentioned first segmentation unit is further configured to acquire the position information of the vertices in the multi-plane image in the region of interest; based on the position information, determine the background region of the region of interest; and determine the region of interest and the background region as the detection region corresponding to the region of interest.

[0186] In one embodiment, the above-mentioned acquisition unit is further configured to perform lesion detection on the straightened image of each tubular branch to obtain the lesion area on the target straightened image; and inverse-map the lesion area on the target straightened image to the multi-plane image to obtain at least one lesion area in the tubular structure on the multi-plane image.

[0187] In one embodiment, the above-mentioned acquisition unit is further configured to input the straightened images of each tubular branch into a preset lesion detection model, and analyze each straightened image by using the lesion detection model to obtain the lesion area on the target straightened image.

[0188] In one embodiment, the above-mentioned segmentation module includes: a detection unit and a second segmentation unit, where:

[0189] The detection unit is configured to perform lesion detection on the straightened images of each tubular branch to obtain the lesion area on the target straightened image;

[0190] The second segmentation unit is configured to perform plaque segmentation on the target straightened image to obtain the plaque segmentation result of the target straightened image.

[0191] In one embodiment, the above-mentioned second segmentation unit is further configured to perform plaque segmentation on the target straightened image through a straightened image segmentation model to obtain the plaque segmentation result of the target straightened image; the straightened image segmentation model is trained according to a plurality of straightened sample images with labeled lumens and plaques.

[0192] In one embodiment, the above-mentioned second segmentation unit is further configured to, for any target straightened image, cut the target straightened image along the center line of the target straightened image to obtain a plurality of straightened reconstruction segments; add position encodings to each straightened reconstruction segment; input each straightened reconstruction segment and the position encoding into the straightened image segmentation model, and perform plaque segmentation on each straightened reconstruction segment to obtain the plaque segmentation result of the target straightened image.

[0193] In one embodiment, the above-mentioned second segmentation unit is further configured to obtain the position information of each straightened reconstruction segment on the center line; based on the position information of each straightened reconstruction segment, add position encodings to each straightened reconstruction segment.

[0194] In one embodiment, the above-mentioned fusion module includes a fusion unit, where:

[0195] The fusion unit is configured to inverse-map the plaque segmentation result of the target straightened image into a multi-plane image according to the positions of the tubular branches with lesions in the tubular structure, and fuse the multi-plane image and the plaque segmentation result of the target straightened image according to the corresponding pixel positions to obtain the plaque segmentation result of the tubular structure.

[0196] In one embodiment, the above-mentioned acquisition module includes: a third segmentation unit and a generation unit, where:

[0197] The third segmentation unit is configured to perform rough tubular segmentation on each tubular branch of the tubular structure in the multi-plane image and extract the center line of each tubular branch;

[0198] A generation unit for generating a straightened image of each tubular branch based on the center line of each tubular branch.

[0199] Each module in the above plaque segmentation device can be implemented in whole or in part by software, hardware, and their combination. Each of the above modules can be embedded in the processor in the computer device in hardware form or independent of it, or stored in the memory in the computer device in software form, so that the processor can call and execute the operations corresponding to each of the above modules.

[0200] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, it implements the content in any one of the above embodiments of the plaque segmentation method.

[0201] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it implements the content in any one of the above embodiments of the plaque segmentation method.

[0202] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, it implements the content in any one of the above embodiments of the plaque segmentation method.

[0203] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data that have been authorized by the user or fully authorized by all parties.

[0204] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include Read-Only Memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0205] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0206] The above embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A plaque segmentation method, characterized in that, The method includes: Obtaining multi-planar images of a tubular structure, and obtaining straightened images of each tubular branch in the tubular structure; Performing plaque segmentation based on the multi-planar images to obtain a plaque segmentation result of the multi-planar images; and performing plaque segmentation based on each of the straightened images to obtain a plaque segmentation result of a target straightened image; the target straightened image represents the straightened image of the tubular branch with a lesion; Fusing the plaque segmentation result of the multi-planar images with the plaque segmentation result of the target straightened image to obtain a plaque segmentation result of the tubular structure.

2. The method according to claim 1, wherein The performing plaque segmentation based on the multi-planar images to obtain a plaque segmentation result of the multi-planar images includes: Obtaining at least one lesion region in the tubular structure on the multi-planar images; Determining regions of interest on the multi-planar images according to each of the lesion regions; Performing plaque segmentation according to each of the regions of interest to obtain a plaque segmentation result of the multi-planar images.

3. The method according to claim 2, characterized in that, The determining regions of interest on the multi-planar images according to each of the lesion regions includes: Taking each of the lesion regions and regions within a preset range outside each of the lesion regions as regions of interest on the multi-planar images.

4. The method according to claim 2, wherein The performing plaque segmentation according to each of the regions of interest to obtain a plaque segmentation result of the multi-planar images includes: Obtaining a detection region corresponding to the region of interest; Inputting the detection region corresponding to the region of interest into a multi-planar image segmentation model, and performing plaque segmentation on the detection region through the multi-planar image segmentation model to obtain a plaque segmentation result of the multi-planar images; the multi-planar image segmentation model is trained according to a plurality of multi-planar sample images with annotated lumens and plaques.

5. The method according to claim 4, wherein The obtaining a detection region corresponding to the region of interest includes: Obtaining position information of vertices of the region of interest in the multi-planar images; Determining a background region of the region of interest based on the position information; Determining the region of interest and the background region as the detection region corresponding to the region of interest.

6. The method according to any one of claims 1-5, characterized in that, The performing plaque segmentation based on the straightened images of each of the tubular branches to obtain a plaque segmentation result of a target straightened image includes: Performing lesion detection on the straightened images of each of the tubular branches to obtain a lesion region on the target straightened image; Performing plaque segmentation on the target straightened image to obtain a plaque segmentation result of the target straightened image.

7. The method according to claim 6, wherein The performing plaque segmentation on the target straightened image to obtain a plaque segmentation result of the target straightened image includes: Performing plaque segmentation on the target straightened image through the straightened image segmentation model to obtain a plaque segmentation result of the target straightened image; the straightened image segmentation model is trained according to a plurality of straightened sample images with annotated lumens and plaques.

8. The method according to claim 7, characterized in that The performing plaque segmentation on the target straightened image through the straightened image segmentation model to obtain a plaque segmentation result of the target straightened image includes: For any one target straightened image, slicing the target straightened image along the center line of the target straightened image to obtain a plurality of straightened reconstruction segments; Add position encoding to each of the straightened and reconstructed segments; Input each of the straightened and reconstructed segments and the position encoding into the straightened image segmentation model, and perform patch segmentation on each of the straightened and reconstructed segments to obtain the patch segmentation result of the target straightened image.

9. A plaque segmentation device, characterized in that, The apparatus includes: An acquisition module, configured to acquire multi-planar images of a tubular structure and acquire straightened images of each tubular branch in the tubular structure; A segmentation module, configured to perform patch segmentation based on the multi-planar images to obtain the patch segmentation result of the multi-planar images; and perform patch segmentation based on the straightened images of each tubular branch to obtain the patch segmentation result of the target straightened image; the target straightened image represents the straightened image of the tubular branch with a lesion; A fusion module, configured to fuse the patch segmentation result of the multi-planar images with the patch segmentation result of the target straightened image to obtain the patch segmentation result of the tubular structure.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images