An image segmentation method, device, electronic device and storage medium

Abstract

The embodiment of the present invention discloses an image segmentation method, device, electronic device and storage medium, the image segmentation method comprising: taking each candidate sampling starting point as the actual sampling starting point, downsampling the original image to be segmented, and obtaining downsampled images corresponding to each candidate sampling starting point; inputting the downsampled images into a pre-trained neural network model, and obtaining the segmented images of the region of interest corresponding to each downsampled image; and splicing the segmented images of the region of interest corresponding to each downsampled image to the original image to be segmented, and obtaining the final image segmentation result. The image segmentation method provided by the embodiment of the present invention solves the problem that segmentation accuracy and resolution cannot be taken into account at the same time, and ensures both the accuracy of image segmentation and the resolution of the image.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image segmentation method, device, electronic equipment and storage medium. Background Art

[0002] When using deep learning technology for image segmentation, due to the limitation of video memory, it is often necessary to crop or downsample the original image before segmentation to solve the problem of insufficient GPU (Graphic Processing Unit) memory.

[0003] However, cropping the original image will lead to the loss of contextual information and reduce the accuracy of image segmentation; while directly segmenting the low-resolution image obtained by downsampling will lose detailed features, making it impossible to ensure both the accuracy of image segmentation and the high resolution of the image.

[0004] To address the above issues, the current conventional approach is to use a multi-scale segmentation strategy, specifically, first use a coarse segmentation network to segment the low-resolution image obtained by downsampling to obtain the approximate location of the region of interest, then crop the region of interest on the original image according to the approximate location, and finally use a fine segmentation network to obtain a segmentation result with the same resolution as the original image based on the cropped region of interest. Although this approach incorporates global context information in the coarse-scale segmentation stage, it loses global context information in the fine-scale segmentation stage because it is a fine segmentation based on the cropped image, and still does not solve the problem of balancing segmentation accuracy and resolution. Summary of the invention

[0005] The embodiments of the present invention provide an image segmentation method, device, electronic device and storage medium, which solve the problem that segmentation accuracy and resolution cannot be taken into account at the same time, and ensure both the accuracy of image segmentation and the resolution of the image.

[0006] In a first aspect, an embodiment of the present invention provides an image segmentation method, the method comprising:

[0007] Taking each candidate sampling starting point as the actual sampling starting point, downsampling the original image to be segmented, and obtaining downsampled images corresponding to each candidate sampling starting point;

[0008] The downsampled images are respectively input into a pre-trained neural network model to obtain a segmented image of the region of interest corresponding to each downsampled image;

[0009] The region of interest segmentation images corresponding to each downsampled image are respectively stitched into the original image to be segmented to obtain the final image segmentation result.

[0010] In a second aspect, an embodiment of the present invention further provides an image segmentation device, the device comprising:

[0011] A downsampling module is used to downsample the original image to be segmented by taking each candidate sampling starting point as the actual sampling starting point, and obtain downsampled images corresponding to each candidate sampling starting point;

[0012] A segmentation module, used to input the downsampled images into a pre-trained neural network model to obtain a segmented image of the region of interest corresponding to each downsampled image;

[0013] The stitching module is used to stitch the region of interest segmentation images corresponding to each downsampled image into the original image to be segmented to obtain a final image segmentation result.

[0014] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0015] one or more processors;

[0016] A memory for storing one or more programs;

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the image segmentation method as described in the embodiment of the present invention.

[0018] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the image segmentation method as described in the embodiment of the present invention.

[0019] The image segmentation method provided by the embodiment of the present invention downsamples the original image to be segmented by taking each candidate sampling starting point as the actual sampling starting point, and obtains downsampled images corresponding to each candidate sampling starting point; the downsampled images are respectively input into a pre-trained neural network model to obtain segmented images of the region of interest corresponding to each downsampled image; the segmented images of the region of interest corresponding to each downsampled image are respectively spliced ​​into the original image to be segmented to obtain the final image segmentation result. The technical means solves the problem that segmentation accuracy and resolution cannot be taken into account at the same time, and ensures both the accuracy of image segmentation and the resolution of the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a flowchart of an image classification method provided in Embodiment 1 of the present invention;

[0021] Figure 2is a schematic diagram of different down-sampled images corresponding to different actual sampling starting points provided by the first embodiment of the present invention;

[0022] Figure 3 is a schematic diagram of a region of interest segmentation image provided by Embodiment 1 of the present invention;

[0023] Figure 4 is a flowchart of an image segmentation method provided by Embodiment 2 of the present invention;

[0024] Figure 5 is a structural schematic diagram of an image segmentation device provided by Embodiment 3 of the present invention;

[0025] Figure 6 It is a structural schematic diagram of an electronic device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0026] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0027] It should be mentioned before discussing the exemplary embodiments in more detail that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the various steps as sequential processes, many of the steps therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the various steps can be rearranged. The process can be terminated when its steps are completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0028] Embodiment 1

[0029] Figure 1 The present invention provides a flowchart of an image segmentation method provided in Embodiment 1 of the present invention. The method is applicable to image segmentation and aims to solve the problem that segmentation accuracy and resolution cannot be taken into account during image segmentation. The image segmentation method can be performed by an image segmentation device, which can be implemented by software and / or hardware.

[0030] refer to Figure 1 As shown, the image segmentation method comprises the following steps:

[0031] Step 110: Taking each candidate sampling starting point as the actual sampling starting point, down-sample the original image to be segmented to obtain down-sampled images corresponding to each candidate sampling starting point.

[0032] Among them, the number of candidate sampling starting points is usually multiple, and the specific quantity can be determined according to the set sampling step length and the size of the original image to be segmented. The more the number of candidate sampling starting points, the more the number of corresponding downsampled images obtained, and the more comprehensive the image information of each part of the original image to be segmented can be obtained. The original image to be segmented can be a CT (Computer Tomography) image, a PET (Positron Emission Tomography) image, an MR (Magnetic Resonance) image or a fused image, where the fused image is a fused image of at least two of the CT image, the PET image and the MR image.

[0033] Specifically, the downsampling of the original image to be segmented includes:

[0034] Taking the actual sampling starting point as a reference point, obtaining a sampling point at every set sampling step length in each row of the original image to be segmented, and obtaining a sampling point at every set sampling step length in each column of the original image to be segmented; the sampling points form the downsampled image corresponding to the actual sampling starting point.

[0035] Exemplarily, referring to Figure 2 a schematic diagram of different downsampled images corresponding to different actual sampling starting points shown in the figure, where the label 210 represents the original image to be segmented, the width W of the original image to be segmented is 4, the height H is 4, and the set sampling step length N is 2; the label 220 represents the downsampled image corresponding to the actual sampling starting point (0, 0); the label 230 represents the downsampled image corresponding to the actual sampling starting point (1, 0); the label 240 represents the downsampled image corresponding to the actual sampling starting point (0, 1); the label 250 represents the downsampled image corresponding to the actual sampling starting point (1, 1).

[0036] Step 120: Input the downsampled images into a pre-trained neural network model respectively, and obtain the region of interest segmentation images corresponding to each downsampled image respectively.

[0037] Among them, the number of the downsampled images is multiple, and the specific quantity can be determined according to the set sampling step length and the size of the original image to be segmented. For example, if the set sampling step length is N, then according to the different positions of the actual sampling starting point, a two-dimensional original image to be segmented with a size of H*W can be disassembled into N*N downsampled images with a size of (H / N)*(W / N). The more the number of downsampled images, the more comprehensive the image information of each part of the original image to be segmented can be obtained.

[0038] Each downsampled image is input into a pre-trained neural network model, and the neural network model outputs a corresponding region of interest segmentation image for each downsampled image. The region of interest segmentation image refers to an image including a segmentation result, and the segmentation result is usually a specific region outlined by a rectangle, a circle, an ellipse or an irregular polygon. Exemplarily, referring to Figure 3 a schematic diagram of a region of interest segmentation image shown in FIG. 310 indicates the circled region of interest. It can be understood that what is directly seen at the visual level is the circled region of interest, while at the technical level, the coordinate positions of the pixels in the region of interest image can be obtained.

[0039] It should be noted that each image region of the downsampled image has the same resolution; and / or, the region of interest segmentation image is a binary image. Specifically, the binary image can be an image in which the pixel values of the pixels within the region of interest are 1 and the pixel values of the pixels outside the region of interest are 0, or an image in which the pixel values of the pixels within the region of interest are 0 and the pixel values of the pixels outside the region of interest are 1.

[0040] Specifically, the pre-trained neural network model can be pre-trained using different types of neural networks according to the characteristics of the original image to be segmented. For example, since the ascending aorta presents a circular morphological feature, the neural network model can specifically be a detection network, a segmentation network or a localization network. When the original image to be segmented is an image of the pulmonary artery, the neural network model can specifically include a neural network main body, a segmentation network branch and a point localization network branch, wherein the segmentation network branch is used to output the segmentation result of the main pulmonary artery, and the point localization network branch is used to output the position of the main pulmonary artery localization point.

[0041] Furthermore, the positions of the regions of interest of different target parts are different. For example, the position of the region of interest of the head and neck arteries is usually in the descending artery of the aortic arch; the position of the region of interest of the pulmonary artery is usually in the main right pulmonary artery; the position of the region of interest of the coronary artery is usually at the ascending aorta; the region of interest of the thoracic aorta is usually at the descending aorta at the tracheal bifurcation; the region of interest of the abdominal aorta is usually at the descending aorta at the renal artery level; the region of interest of the renal artery is usually at the descending artery at the renal artery level; the region of interest of the lower limbs is usually at the upper part of the bifurcation of the abdominal aorta; the region of interest of the upper limbs is usually at the aortic arch.

[0042] Wherein, in the training stage of the neural network model, the position of the sampling starting point is randomly generated, and then the original image to be segmented is downsampled to obtain different downsampled images respectively. Each downsampled image is input into the constructed neural network model to train the neural network model and optimize various model parameters of the neural network model.

[0043] Step 130: respectively stitch the region of interest segmentation images corresponding to each downsampled image into the original image to be segmented to obtain a final image segmentation result.

[0044] It is understandable that downsampling the original image to be segmented reduces the image resolution and reduces the memory requirements for the GPU, but also loses some image detail information. Therefore, the resolution of the image segmentation result obtained by directly performing image segmentation based on the downsampled image is usually not high. In response to this problem, the technical solution of this embodiment is to splice the region of interest segmentation image corresponding to each downsampled image to the original image to be segmented, and obtain the final image segmentation result with the same resolution as the original image to be segmented. Since the original image to be segmented is not cropped during the image segmentation process, the purpose of retaining the context image information is achieved, which is beneficial to the accuracy of image segmentation; at the same time, although the original image to be segmented is downsampled (this is to reduce the requirements for the GPU), the region of interest segmentation images corresponding to each downsampled image are finally spliced ​​separately, thereby ensuring the image resolution, that is, the image clarity, and improving the image segmentation result.

[0045] Exemplarily, the region of interest segmentation images corresponding to each downsampled image are respectively stitched into the original image to be segmented to obtain a final image segmentation result, including:

[0046] According to the mapping relationship between each pixel point of each region of interest segmentation image and each pixel point of the original image to be segmented, each region of interest segmentation image is mapped to the original image to be segmented to obtain a final image segmentation result.

[0047] The image segmentation method provided in the present embodiment downsamples the original image to be segmented by taking each candidate sampling starting point as the actual sampling starting point, and obtains downsampled images corresponding to each candidate sampling starting point; inputs the downsampled images into a pre-trained neural network model, and obtains a segmented image of the region of interest corresponding to each downsampled image; and splices the segmented image of the region of interest corresponding to each downsampled image into the original image to be segmented, to obtain the final image segmentation result. The technical means solves the problem that segmentation accuracy and resolution cannot be taken into account at the same time, and ensures both the accuracy of image segmentation and the resolution of the image.

[0048] Embodiment 2

[0049] Figure 4Schematic flowchart of an image segmentation method provided in the second embodiment of the present invention. On the basis of the above embodiment, this embodiment further provides a method for determining a candidate sampling starting point. Specifically, pixel points in the original image to be segmented with abscissa greater than 0 and less than or equal to a set sampling step length, and ordinate greater than 0 and less than or equal to the set sampling step length are respectively determined as the candidate sampling starting points. The advantage of such a setting is that all possible sampling points can be traversed to comprehensively obtain the image information of each part of the original image to be segmented. On the other hand, a specific implementation manner for obtaining the downsampled image and splicing the region-of-interest segmentation images corresponding to each downsampled image to the original image to be segmented is also given. Among them, the same or similar explanatory descriptions as those in the above embodiment can be referred to the above embodiment, and will not be repeated in this embodiment.

[0050] As Figure 4 shown, the image segmentation method includes the following steps:

[0051] Step 410: Pixel points in the original image to be segmented with abscissa greater than 0 and less than or equal to a set sampling step length, and ordinate greater than 0 and less than or equal to the set sampling step length are respectively determined as the candidate sampling starting points.

[0052] By setting the limiting conditions "abscissa greater than 0 and less than or equal to the set sampling step length, ordinate greater than 0 and less than or equal to the set sampling step length", the purpose of traversing all possible sampling points is achieved. The purpose of comprehensively obtaining the image information of each part of the original image to be segmented is achieved.

[0053] Step 420: Using each candidate sampling starting point as the actual sampling starting point, perform downsampling on the original image to be segmented, and respectively obtain the downsampled images corresponding to each candidate sampling starting point.

[0054] Specifically, based on the set sampling step length and the candidate sampling starting point, determine the mapping relationship between each pixel point of the downsampled image and the pixel point of the original image to be segmented;

[0055] Determine the downsampled image based on the mapping relationship.

[0056] For example, assume that the width of the original image to be segmented is W, the height is H, and the pixel value of the pixel point at the y-th row and x-th column of the original image to be segmented is I(x, y) (0 < y ≤ H, 0 < x ≤ W).

[0057] Specifically, based on the following formula, determine the pixel value of each pixel point of the downsampled image:

[0058] I ox，oy (x′, y′) = I(ox + (x′ - 1) * N, oy + (y′ - 1) * N) (0 < x′ ≤ W / N, 0 < y′ ≤ H / N)

[0059] Among them, the coordinates of the actual sampling starting point are (ox, oy), the set sampling step is N, W represents the width of the original image to be segmented, H represents the height of the original image to be segmented, (x', y') represents the coordinates of the sampling point, and I ox，oy (x′, y′) represents the pixel value of the pixel point (x', y') in the downsampled image, and I(ox+(x′-1)*N, oy+(y′-1)*N) represents the pixel value of the pixel point corresponding to the sampling point (x', y') in the original image to be segmented.

[0060] Step 430: Input the downsampled images into a pre-trained neural network model respectively, and obtain the region-of-interest segmentation images corresponding to each downsampled image respectively.

[0061] Step 440: Piece together the region-of-interest segmentation images corresponding to each downsampled image to the original image to be segmented respectively, and obtain the final image segmentation result.

[0062] Exemplarily, map each region-of-interest segmentation image to the original image to be segmented respectively according to the mapping relationship between each pixel point of each region-of-interest segmentation image and each pixel point of the original image to be segmented, and obtain the final image segmentation result.

[0063] Specifically, map each region-of-interest segmentation image to the original image to be segmented based on the following conversion formula:

[0064] M(x,y) = M REM(x / N),REM(y / N) (INT(x / N)+1,INT(y / N)+1)

[0065] Among them, M(x,y) represents the final image segmentation result, and M REM(x / N),REM(y / N) (INT(x / N)+1,INT(y / N)+1) represents the region-of-interest segmentation image corresponding to the downsampled image corresponding to the actual sampling starting point (REM(x / N), REM(y / N)), REM() represents the remainder operation, INT() represents the integer operation, N represents the set sampling step, and (x, y) represents the coordinates of the pixel point.

[0066] Furthermore, since the processing accuracy of the neural network model cannot reach 100%, the pixel points of the region-of-interest segmentation image corresponding to each downsampled image output by the neural network model may not completely coincide with the pixel points at the corresponding positions of the original image to be segmented. Therefore, the edge of the image segmentation result M(x,y) after piecing together may not be smooth. To address this problem, the image segmentation method provided in this embodiment further includes the following steps:

[0067] Perform post - processing on the image segmentation result to smooth the edges of the image segmentation result.

[0068] The post - processing includes, for example, image post - processing algorithms such as dilation and erosion. Among them, image dilation and erosion are two basic morphological operations, mainly used to find the maximum and minimum regions in the image. Among them, dilation is similar to "region expansion", expanding the highlighted region or white part in the image, and the resulting image after its operation is larger than the highlighted region of the original image. Erosion is similar to "region being eaten away", reducing and refining the highlighted region or white part in the image, and the resulting image after its operation is smaller than the highlighted region of the original image. Ultimately, the purpose of optimizing the image edges is achieved.

[0069] Based on the above - mentioned technical solution, the image segmentation method provided in this embodiment determines the pixel points with abscissa greater than 0 and less than or equal to the set sampling step length and ordinate greater than 0 and less than or equal to the set sampling step length in the original image to be segmented as the candidate sampling starting points respectively. The advantage of this setting is that it can traverse all possible sampling points to comprehensively obtain the image information of each part of the original image to be segmented; and an operation of performing post - processing on the image segmentation result is added, achieving the purpose of smoothing the edges of the image segmentation result, improving the image segmentation result, and taking into account the image resolution while retaining the image context information.

[0070] The following is an embodiment of the image segmentation device provided by the embodiments of the present invention. This device belongs to the same inventive concept as the image segmentation methods of the above - mentioned embodiments. For the details not described in detail in the embodiment of the image segmentation device, reference can be made to the embodiments of the above - mentioned image segmentation methods.

[0071] Embodiment III

[0072] Figure 5 As shown in the structural schematic diagram of an image segmentation device provided by Embodiment III of the present invention, Figure 5 the image segmentation device includes: a down - sampling module 510, a segmentation module 520, and a stitching module 530.

[0073] Among them, the down - sampling module 510 is used to perform down - sampling on the original image to be segmented with each candidate sampling starting point as the actual sampling starting point, and obtain the down - sampled images corresponding to each candidate sampling starting point respectively; the segmentation module 520 is used to input the down - sampled images into a pre - trained neural network model respectively, and obtain the region - of - interest segmentation images corresponding to each down - sampled image respectively; the stitching module 530 is used to stitch the region - of - interest segmentation images corresponding to each down - sampled image into the original image to be segmented respectively, and obtain the final image segmentation result.

[0074] Based on the above - mentioned technical solutions, the image segmentation device further includes:

[0075] A determination module, configured to determine the candidate sampling starting point according to a set sampling step.

[0076] Based on the above technical solutions, the determination module is specifically configured to:

[0077] Determine the pixel points in the to-be-segmented original image whose abscissa is greater than 0 and less than or equal to the set sampling step, and whose ordinate is greater than 0 and less than or equal to the set sampling step, as the candidate sampling starting points respectively.

[0078] Based on the above technical solutions, the downsampling module 510 is specifically configured to:

[0079] Determine the pixel value of each pixel point of the downsampled image based on the following formula:

[0080] I ox，oy (x′, y′) = I(ox + (x′ - 1) * N, oy + (y′ - 1) * N) (0 < x′ ≤ W / N, 0 < y′ ≤ H / N)

[0081] wherein, the coordinates of the actual sampling starting point are (ox, oy), the set sampling step is N, W represents the width of the to-be-segmented original image, H represents the height of the to-be-segmented original image, (x′, y′) represents the coordinates of the sampling point, and I ox，oy (x′, y′) represents the pixel value of the pixel point (x′, y′) in the downsampled image, and I(ox + (x′ - 1) * N, oy + (y′ - 1) * N) represents the pixel value of the pixel point corresponding to the sampling point (x′, y′) in the to-be-segmented original image.

[0082] Based on the above technical solutions, the stitching module 530 includes:

[0083] A mapping unit, configured to map each of the region-of-interest segmentation images to the to-be-segmented original image respectively according to the mapping relationship between each pixel point of each of the region-of-interest segmentation images and each pixel point of the to-be-segmented original image, to obtain the final image segmentation result.

[0084] Based on the above technical solutions, the mapping unit is specifically configured to:

[0085] Map each of the region-of-interest segmentation images to the to-be-segmented original image based on the following conversion formula:

[0086] M(x, y) = M REM(x / N),REM(y / N) (INT(x / N) + 1, INT(y / N) + 1)

[0087] wherein, M(x, y) represents the final image segmentation result, and M REM(x / N),REM(y / N)(INT(x / N)+1, INT(y / N)+1) represents the region of interest segmentation image corresponding to the downsampled image corresponding to the actual sampling starting point (REM(x / N), REM(y / N)), REM() represents the modulo operation, INT() represents the integer operation, N represents the set sampling step size, and (x, y) represents the coordinates of the pixel point. Based on the above technical solutions, the image segmentation device also includes:

[0088] The post-processing module is used to perform post-processing on the image segmentation result to smooth the edge of the image segmentation result.

[0089] The image segmentation device provided in the present embodiment downsamples the original image to be segmented by taking each candidate sampling starting point as the actual sampling starting point, and obtains downsampled images corresponding to each candidate sampling starting point; inputs the downsampled images into a pre-trained neural network model, and obtains a segmented image of the region of interest corresponding to each downsampled image; and splices the segmented image of the region of interest corresponding to each downsampled image into the original image to be segmented, to obtain the final image segmentation result. The technical means solves the problem that segmentation accuracy and resolution cannot be taken into account at the same time, and ensures both the accuracy of image segmentation and the resolution of the image.

[0090] The image segmentation device provided in the embodiment of the present invention can execute the image segmentation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the image segmentation method.

[0091] Embodiment 4

[0092] Figure 6 A schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. Figure 6 A block diagram of an exemplary electronic device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 6 The electronic device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0093] like Figure 6 As shown, the electronic device 12 is in the form of a general purpose computing electronic device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting various system components (including the system memory 28 and the processing unit 16).

[0094] Bus 18 represents one or more of several types of bus architectures, including a memory bus or memory controller, a peripheral bus, an Accelerated Graphics Port, a processor bus, or a local bus using any of a variety of bus architectures. By way of example, such architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

[0095] Electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 12, including both volatile and nonvolatile media, removable and non-removable media.

[0096] System memory 28 can include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Electronic device 12 may further include other removable / non-removable, volatile / nonvolatile computer system storage media. By way of example only, storage system 34 can be used for reading and writing on non-removable, nonvolatile magnetic media ( Figure 6 not shown and typically called a "hard disk drive"). Although Figure 6 not shown in the figures, a disk drive for reading and writing on removable nonvolatile disks (such as a "floppy disk"), and an optical disk drive for reading and writing on removable nonvolatile optical disks (such as a CD-ROM, DVD-ROM, or other optical media) can be provided. In these cases, each drive can be connected to bus 18 by one or more data media interfaces. System memory 28 can include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the present invention.

[0097] A program / utility 40 having a set (at least one) of program modules 42 can be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may include an implementation of a network environment. Program modules 42 typically carry out the functions and / or methods of the embodiments described herein.

[0098] The electronic device 12 can also communicate with one or more external devices 14 (such as a keyboard, a pointing device, a display 24, etc.), and can also communicate with one or more devices that enable a user to interact with the electronic device 12, and / or communicate with any device that enables the electronic device 12 to communicate with one or more other computing devices (such as a network card, a modem, etc.). Such communication can be carried out through the input / output (I / O) interface 22. Moreover, the electronic device 12 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 20. As shown in the figure, the network adapter 20 communicates with other modules of the electronic device 12 through the bus 18. It should be understood that although not shown in the figure, other hardware and / or software modules can be used in combination with the electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0099] The processing unit 16 executes various functional applications and image segmentation by running programs stored in the system memory 28, for example, implementing the steps of an image segmentation method provided by an embodiment of the present invention. The method includes:

[0100] Taking each candidate sampling starting point as the actual sampling starting point, performing downsampling on the original image to be segmented, and respectively obtaining downsampled images corresponding to each of the candidate sampling starting points;

[0101] Inputting the downsampled images into a pre-trained neural network model respectively, and respectively obtaining region-of-interest segmentation images corresponding to each downsampled image;

[0102] Stitching the region-of-interest segmentation images corresponding to each downsampled image to the original image to be segmented respectively, and obtaining the final image segmentation result.

[0103] Certainly, those skilled in the art can understand that the processor can also implement the technical solutions of the image segmentation method provided by any embodiment of the present invention.

[0104] Embodiment Five

[0105] Embodiment Five of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the steps of the image segmentation method provided by any embodiment of the present invention. The method includes:

[0106] Taking each candidate sampling starting point as the actual sampling starting point, performing downsampling on the original image to be segmented, and respectively obtaining downsampled images corresponding to each of the candidate sampling starting points;

[0107] Input the downsampled images into a pre-trained neural network model respectively to obtain the region-of-interest segmentation images corresponding to each downsampled image;

[0108] Piece together the region-of-interest segmentation images corresponding to each downsampled image to the original image to be segmented respectively to obtain the final image segmentation result.

[0109] The computer storage medium of the embodiments of the present invention can adopt any combination of one or more computer-readable media. The computer-readable media can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage medium can be any tangible medium that contains or stores a program, which can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0110] The computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0111] The program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0112] Computer program code for performing the operations of the present invention may be written in one or more programming languages or combinations thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and also including conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0113] Those of ordinary skill in the art should understand that the above-described modules or steps of the present invention can be implemented using a general-purpose computing device. They can be concentrated on a single computing device or distributed over a network composed of multiple computing devices. Optionally, they can be implemented using program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0114] Note that the above is only the preferred embodiment of the present invention and the technical principles applied. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein. Various obvious changes, re-adjustments, and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments. Without departing from the concept of the present invention, more other equivalent embodiments can be included, and the scope of the present invention is determined by the scope of the appended claims.

[0115] Those skilled in the art can understand that all or part of the steps for implementing the methods in the above embodiments can be completed by instructing relevant hardware through a program. The program is stored in a storage medium, including several instructions for causing a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs, etc., all of which can store program code.

[0116] Note that the above is only a preferred embodiment of the present invention and the technical principles applied. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, re-adjustments, and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments. Without departing from the concept of the present invention, it may also include more other equivalent embodiments, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. An image segmentation method, characterized in that, it includes: Taking each candidate sampling starting point as the actual sampling starting point, downsampling the original image to be segmented, and respectively obtaining a downsampled image corresponding to each candidate sampling starting point; wherein, the number of candidate sampling starting points is determined according to a set sampling step and the size of the original image to be segmented; Inputting the downsampled images into a pre-trained neural network model respectively, and respectively obtaining a region of interest segmentation image corresponding to each downsampled image; wherein, the region of interest segmentation image refers to an image including a segmentation result; Stitching the region of interest segmentation images corresponding to each downsampled image to the original image to be segmented respectively, to obtain a final image segmentation result; Wherein, the step of stitching the region of interest segmentation images corresponding to each downsampled image to the original image to be segmented respectively to obtain a final image segmentation result includes: According to the mapping relationship between each pixel point of each region of interest segmentation image and each pixel point of the original image to be segmented respectively, mapping each region of interest segmentation image to the original image to be segmented, to obtain the final image segmentation result.

2. The image segmentation method according to claim 1, characterized in that, it further includes: Determining the candidate sampling starting point according to the set sampling step.

3. The image segmentation method according to claim 2, characterized in that, The step of determining the candidate sampling starting point according to the set sampling step includes: Determining the pixel points with abscissa greater than 0 and less than or equal to the set sampling step and ordinate greater than 0 and less than or equal to the set sampling step in the original image to be segmented as the candidate sampling starting points respectively.

4. The image segmentation method according to claim 1, characterized in that, The step of taking each candidate sampling starting point as the actual sampling starting point, downsampling the original image to be segmented, and respectively obtaining a downsampled image corresponding to each candidate sampling starting point includes: Determining the mapping relationship between each pixel point of the downsampled image and the pixel points of the original image to be segmented based on the set sampling step and the candidate sampling starting point; Determining the downsampled image based on the mapping relationship.

5. The image segmentation method according to claim 1, characterized in that, The step of mapping each region of interest segmentation image to the original image to be segmented respectively according to the mapping relationship between each pixel point of each region of interest segmentation image and each pixel point of the original image to be segmented to obtain a final image segmentation result includes: Mapping each region of interest segmentation image to the original image to be segmented based on the following conversion formula: M(x,y) = M REM(x / N),REM(y / N) (INT(x / N)+1,INT(y / N)+1) Among them, M(x, y) represents the final image segmentation result, M REM(x / N),REM(y / N) (INT(x / N)+1, INT(y / N)+1) represents the region-of-interest segmentation image corresponding to the downsampled image corresponding to the actual sampling starting point (REM(x / N), REM(y / N)). REM() represents the remainder operation, INT() represents the integer operation, N represents the set sampling step, and (x, y) represents the coordinates of the pixel point.

6. The image segmentation method according to any one of claims 1-4, characterized in that, it further includes: Performing post-processing on the image segmentation result to smooth the edge of the image segmentation result.

7. The image segmentation method according to any one of claims 1-4, characterized in that, Each image region of the downsampled image has the same resolution; and / or, the region of interest segmentation image is a binary image.

8. The image segmentation method according to any one of claims 1-4, characterized in that, the original image to be segmented includes at least one of the following: CT image, PET image, MR image or fused image; wherein, the fused image is a fused image of at least two of CT image, PET image and MR image.

9. An electronic device, characterized in that, the electronic device includes: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the image segmentation method according to any one of claims 1-7.

Images