Image segmentation method, device, equipment and storage medium

Abstract

The embodiments of the present invention disclose an image segmentation method, apparatus, device and storage medium. The method comprises: obtaining a target image to be segmented; inputting the target image into a pre-trained target neural network model to obtain an output segmentation result corresponding to the target image; wherein the target neural network model is trained based on a standard segmentation image and standard delineation information, wherein the standard delineation information is used to characterize the positioning information of the image segmentation unit in the standard segmentation image in at least one dimensional direction. The embodiments of the present invention solve the problem of poor image segmentation effect of the existing neural network model by training the neural network model based on the standard segmentation image and standard delineation information, so that when the neural network model segments the image, it not only focuses on the visual feature information in the image but also focuses on the anisotropic information in the image, thereby improving the accuracy of image segmentation.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of image processing technology, and in particular to an image segmentation method, apparatus, device and storage medium. Background Art

[0002] Currently, image segmentation can be divided into manual and automatic segmentation. Although manual segmentation has high accuracy, its accuracy is largely dependent on the operator's prior knowledge. Furthermore, the manual segmentation process is time-consuming and requires considerable effort and time. Therefore, achieving automatic image segmentation is particularly important and urgent.

[0003] Since 2012, deep learning models have made a significant impact in the image processing field, and the accuracy of deep learning-based automatic segmentation algorithms has steadily improved. However, existing deep learning models are trained solely on the visual features of target objects in images. In practice, images contain more information than just visual features, resulting in low segmentation accuracy when using existing deep learning models. Summary of the Invention

[0004] Embodiments of the present invention provide an image segmentation method, apparatus, device, and storage medium to improve the accuracy of image segmentation of a neural network model.

[0005] In a first aspect, an embodiment of the present invention provides an image segmentation method, the method comprising:

[0006] Obtain the target image to be segmented;

[0007] Inputting the target image into a pre-trained target neural network model to obtain an output segmentation result corresponding to the target image;

[0008] The target neural network model is trained based on a standard segmentation image and standard delineation information, wherein the standard delineation information is used to represent the positioning information of the image segmentation unit in the standard segmentation image in at least one dimensional direction.

[0009] In a second aspect, an embodiment of the present invention further provides an image segmentation device, the device comprising:

[0010] A target image acquisition module is used to acquire the target image to be segmented;

[0011] A segmentation result output module is used to input the target image into a pre-trained target neural network model to obtain an output segmentation result corresponding to the target image;

[0012] The target neural network model is trained based on a standard segmentation image and standard delineation information, wherein the standard delineation information is used to represent the positioning information of the image segmentation unit in the standard segmentation image in at least one dimensional direction.

[0013] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0014] one or more processors;

[0015] a memory for storing one or more programs;

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement any of the above-mentioned image segmentation methods.

[0017] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer-executable instructions, wherein the computer-executable instructions are used to perform any of the above-mentioned image segmentation methods when executed by a computer processor.

[0018] The embodiment of the present invention solves the problem of poor image segmentation effect of existing neural network models by training the neural network model based on standard segmentation images and standard outline information, so that when the neural network model segments the image, it not only focuses on the visual feature information in the image but also focuses on the anisotropy information in the image, thereby improving the accuracy of image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flowchart of an image segmentation method provided by the first embodiment of the present invention;

[0020] Figure 2A This is a schematic diagram of standard delineation information provided by the first embodiment of the present invention;

[0021] Figure 2B is a schematic diagram of a target neural network model provided in Example 1 of the present invention;

[0022] Figure 2C is a schematic diagram of another target neural network model provided in Example 1 of the present invention;

[0023] Figure 2D is a schematic diagram of a specific example of a target neural network model provided in the first embodiment of the present invention;

[0024] Figure 3 This is a flow chart of an image segmentation method provided by the second embodiment of the present invention;

[0025] Figure 4is a schematic diagram of an image segmentation device provided by Embodiment 3 of the present invention;

[0026] Figure 5 This is a structural diagram of an electronic device provided in Example 4 of the present invention. DETAILED DESCRIPTION

[0027] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0028] Example 1

[0029] Figure 1 This is a flowchart of an image segmentation method provided in Example 1 of the present invention. This embodiment is applicable to image segmentation. The method can be performed by an image segmentation device, which can be implemented in software and / or hardware. The device can be configured in a terminal device. For example, the terminal device can be a mobile terminal, desktop computer, notebook computer, tablet computer, server, or other intelligent terminal. Specifically, the method includes the following steps:

[0030] S110: Acquire a target image to be segmented.

[0031] In one embodiment, the target image may be a two-dimensional image or a three-dimensional image. For example, the target image may be a medical image, such as a CT (Computed Tomography) image, an MRI (Magnetic Resonance Imaging) image, or a PET (Positron Emission Computed Tomography). Of course, the target image may also be a landscape image or a person image, and the target image type is not limited herein.

[0032] For example, in the medical field, according to the DICOM (Digital Imaging and Communications in Medicine) standard definition, the X-axis direction corresponds to the left and right directions of the human body, the Y-axis direction corresponds to the front (chest) and back (back) directions of the human body, and the Z-axis direction corresponds to the top (head) and bottom (feet) directions of the human body. In order to increase the imaging rate, there may be situations where the image units in the medical image have different image unit sizes in the three dimensions. For example, the image unit size in the X-axis and Y-axis directions is smaller, while the image unit size in the Z-axis direction is larger, resulting in the medical image obtained by imaging containing not only visual feature information, but also anisotropic information of the image units.

[0033] S120: Input the target image into a pre-trained target neural network model to obtain an output segmentation result corresponding to the target image.

[0034] In this embodiment, the target neural network model is trained based on the standard segmentation image and standard delineation information, wherein the standard delineation information is used to represent the positioning information of the image segmentation unit in the standard segmentation image in at least one dimensional direction.

[0035] Exemplarily, the standard segmented image is an image comprising image segmentation units, wherein the image segmentation units are used to describe image units belonging to the segmented portions in the standard segmented image. The standard delineation information can be used to describe anisotropy information corresponding to the segmented portions in the standard segmented image. In one embodiment, optionally, the standard segmented image includes a training standard segmented image and a testing standard segmented image, and correspondingly, the standard delineation information includes training standard delineation information and testing standard delineation information.

[0036] Specifically, when the standard segmentation image is a two-dimensional image, the image unit is an image pixel, and when the standard segmentation image is a three-dimensional image, the image unit is an image voxel. In one embodiment, when the target image is a two-dimensional image, the target delineation information output by the target neural network model includes the positioning information of the image segmentation pixels in the target segmentation image in the X-axis direction and / or the Y-axis direction. In another embodiment, when the target image is a three-dimensional image, the target delineation information output by the target neural network model includes the positioning information of the image segmentation voxels in the target segmentation image in at least one dimensional direction of the X-axis direction, the Y-axis direction, and the Z-axis direction.

[0037] Figure 2A This is a schematic diagram of standard delineation information provided by the first embodiment of the present invention. Figure 2A shows a standard segmented image containing 8*4 image pixels, Figure 2AThe shaded squares in the image represent the image segmentation pixels in the standard segmentation image, and the blank squares represent the non-image segmentation pixels in the standard segmentation image. If the pixel layer containing the image segmentation pixels is marked as 1 and the pixel layer not containing the image segmentation pixels is marked as 0, then the standard delineation information of the image segmentation pixels in the X-axis direction is [0, 0, 1, 1, 1, 0, 0], and the standard delineation information in the Y-axis direction is [0, 1, 1, 0].

[0038] In one embodiment, optionally, the target neural network model includes a feature extraction module, an image segmentation module, an intermediate deep supervision module and a terminal deep supervision module, the feature extraction module is used to output a feature vector based on the input target image, the intermediate deep supervision module is used to output intermediate delineation information based on the input feature vector, the image segmentation module is used to output a target segmentation image corresponding to the target image based on the input feature vector and the intermediate delineation information, and the terminal deep supervision module is used to output target delineation information based on the target segmentation image.

[0039] Specifically, the intermediate deep supervision module introduces the intermediate outline information learned during training as attention into the image segmentation module, so that the target neural network model has the ability to learn and recognize the anisotropic information in the image.

[0040] Figure 2B is a schematic diagram of a target neural network model provided in Example 1 of the present invention. Specifically, the dotted box represents the target neural network model, and the segmentation result corresponding to the target image includes a target segmented image and target delineation information. In one embodiment, optionally, the feature extraction module is an encoder module, and the image segmentation module is a decoder module. Exemplarily, the feature vector may be a feature map corresponding to the target image output by the encoder module.

[0041] In one embodiment, optionally, the intermediate deep supervision module includes a first intermediate deep supervision module and at least one second intermediate deep supervision module, the first intermediate deep supervision module is used to output first intermediate delineation information based on the input feature vector, the first image segmentation module is used to output a first segmented image based on the input feature vector and the first intermediate delineation information, the second intermediate deep supervision module is used to output second delineation information based on the first segmented image, and the second image segmentation module is used to output a second segmented image based on the first segmented image and the second delineation information; wherein the second segmented image includes the target segmented image.

[0042] Figure 2C Schematic diagram of another target neural network model provided by the first embodiment of the present invention. Specifically, Figure 2CThe target neural network model shown includes two second image segmentation modules and two second intermediate deep supervision modules. There is no limit on the number of second image segmentation modules and second intermediate deep supervision modules in the target neural network model.

[0043] Figure 2D : is a schematic diagram of a specific example of a target neural network model provided by the first embodiment of the present invention. Specifically, Figure 2D The feature extraction module in the target neural network model shown includes four encoding modules, and the image segmentation module is specifically a decoder module, which includes a first decoding module and two second decoding modules.

[0044] The technical solution of this embodiment solves the problem of poor image segmentation effect of existing neural network models by training the neural network model based on standard segmentation images and standard outline information. When the neural network model segments the image, it not only focuses on the visual feature information in the image but also focuses on the anisotropy information in the image, thereby improving the accuracy of image segmentation.

[0045] Example 2

[0046] Figure 3 It is a flowchart of an image segmentation method provided in the second embodiment of the present invention. The technical solution of this embodiment is a further refinement based on the above embodiment. Optionally, the standard segmentation image includes a training standard segmentation image, and the standard delineation information includes training standard delineation information. Accordingly, the training method of the target neural network model includes: obtaining the image to be trained in the training set and the image to be tested in the test set; inputting the image to be trained into the initial neural network model, iteratively training the initial neural network model based on the training standard segmentation image, the training standard delineation information, the training predicted segmentation image output by the initial neural network model, and the training predicted delineation information to obtain at least one intermediate neural network model; inputting the image to be tested into each of the intermediate neural network models, and determining the evaluation results corresponding to each of the intermediate neural network models based on the output test predicted segmentation image; and using the intermediate neural network model whose evaluation result meets the preset evaluation standard as the target neural network model.

[0047] The specific implementation steps of this embodiment include:

[0048] S210: Obtain images to be trained in the training set and images to be tested in the test set.

[0049] Specifically, the images to be trained in the training set are images used to iteratively train the initial neural network model, and the images to be tested in the test set are images used to test the intermediate neural network model obtained through training.

[0050] In one embodiment, optionally, the method further includes: obtaining an original image set, and preprocessing the original images in the original image set respectively to obtain preprocessed original images; dividing the preprocessed original images into images to be trained in a training set and images to be tested in a test set based on a preset ratio; performing data enhancement processing on the images to be trained in the training set, and adding the data enhanced images to be trained to the training set.

[0051] Exemplarily, the original image set includes at least two original images, and the original images may be CT images, MRI images, or PET images.

[0052] In one embodiment, the preprocessing optionally includes at least one of format conversion, truncation, and normalization. Specifically, when the original image is a two-dimensional image, the two-dimensional image can be converted into a three-dimensional image through format conversion. Specifically, the original image is truncation-processed based on a preset window level value and a preset window width value. Optionally, the original image can be multiple two-dimensional images.

[0053] Taking CT images as an example, CT equipment can identify density differences in 2,000 different grayscales, while the human eye can only distinguish 16 grayscales. Therefore, the CT value range in the CT image must exceed 125Hu to be recognized by the human eye. The preset window level value is used to describe the central CT value corresponding to the original image after truncation or the mean value of the CT values within the window width range. The window level should be equal to or close to the CT value corresponding to the tissue image to be segmented. The preset window width value is used to describe the CT value range corresponding to the original image after truncation. The width of the window width will affect the clarity and contrast of the original image after truncation. For example, the preset window level value is 50Hu and the preset window width value is 400Hu.

[0054] In one embodiment, optionally, format conversion, truncation processing, and normalization processing are performed on the original images in the original image set in sequence to obtain pre-processed original images.

[0055] For example, the preset ratio may be 7:3.

[0056] Exemplarily, the data augmentation process includes at least one of flipping, translating, and rotating. This configuration has the advantage of increasing the sample size of the training images in the training set, thereby improving the generalization ability of the target neural network model.

[0057] S220. Input the image to be trained into the initial neural network model, and iteratively train the initial neural network model based on the training standard segmentation image, the training standard delineation information, the training prediction segmentation image and the training prediction delineation information output by the initial neural network model to obtain at least one intermediate neural network model.

[0058] In this embodiment, the standard segmentation image includes a training standard segmentation image, and the standard delineation information includes training standard delineation information. The training standard segmentation image is an image used to train the initial neural network model.

[0059] In one embodiment, optionally, the initial neural network model is iteratively trained based on the training standard segmentation image, the training standard delineation information, the training predicted segmentation image and the training predicted delineation information output by the initial neural network model to obtain at least one intermediate neural network model, including: determining a first loss function based on the training predicted segmentation image and the training standard segmentation image, and determining a second loss function based on the training predicted delineation information and the training standard delineation information; and iteratively training the initial neural network model based on the first loss function, the second loss function and a preset optimizer to obtain at least one intermediate neural network model.

[0060] Exemplary methods for calculating the loss function include, but are not limited to, a 0-1 loss function, an absolute value loss function, a logarithmic loss function, a square loss function, an exponential loss function, a Hinge loss function, a cross-entropy loss function, a Dice loss function, a focal loss function, a region-based loss function, or a boundary-based loss function, etc. In one embodiment, optionally, the first loss function is calculated based on the cross-entropy loss function or the Dice loss function, and the second loss function is calculated based on the cross-entropy loss function.

[0061] Among them, exemplary, the preset optimizers include but are not limited to Adam optimizer, SGD optimizer, RMSprop optimizer, etc.

[0062] Specifically, the first loss function and the second loss function are output to a preset optimizer, and the preset optimizer outputs the change in model parameters based on a gradient descent algorithm, thereby iteratively training the model parameters of the initial neural network model and minimizing the loss function.

[0063] Based on the above embodiments, optionally, the method includes: obtaining a training standard segmentation image, determining a target dimension direction with the highest anisotropy based on the image unit sizes of the image units in the training standard segmentation image in at least two dimensional directions; obtaining positioning information of the image segmentation units in the training standard segmentation image in the target dimensional direction, and generating training standard delineation information based on the positioning information.

[0064] Exemplarily, it is assumed that the training standard segmentation image is a three-dimensional CT segmentation image, wherein the image unit size is the image voxel size, specifically including the voxel size in the X-axis direction, the Y-axis direction, and the Z-axis direction. The voxel size in each dimensional direction is compared to obtain the dimensional direction with the largest voxel size as the target dimensional direction with the highest anisotropy, such as the target dimensional direction is the Z-axis direction. The positioning information of the image segmentation voxels in the training standard segmentation image in the Z-axis direction is obtained. Specifically, the voxel layer containing the image segmentation voxels in the Z-axis direction is marked as 1, and the voxel layer not containing the image segmentation voxels is marked as 0, to obtain the training standard delineation information.

[0065] The advantage of this setting is that, in this embodiment, the training standard delineation information only includes delineation information in one dimensional direction, so that the trained target neural network model has a better image segmentation effect for the target image with anisotropic information in this dimensional direction, thereby further improving the image segmentation accuracy of the target image with anisotropic information in a single direction.

[0066] S230: Input the image to be tested into each intermediate neural network model, and determine the evaluation results corresponding to each intermediate neural network model based on the output test prediction segmentation image.

[0067] In this embodiment, the standard segmented image also includes a test standard segmented image. The test standard segmented image is used to evaluate the intermediate neural network model. Specifically, based on the preset evaluation algorithm, the test prediction segmented image and the test standard segmented image, the evaluation results corresponding to each intermediate neural network are determined. Exemplarily, the preset evaluation algorithm includes but is not limited to the Dice coefficient algorithm, the IOU algorithm and the Hausdorff-95 coefficient algorithm, etc. The preset evaluation algorithm is not limited here.

[0068] S240: The intermediate neural network model whose evaluation results meet the preset evaluation criteria is used as the target neural network model.

[0069] For example, when the evaluation result is an evaluation score, the preset evaluation standard may be that the evaluation score is greater than a preset score threshold.

[0070] S250: Acquire a target image to be segmented.

[0071] S260: Input the target image into a pre-trained target neural network model to obtain an output segmentation result corresponding to the target image.

[0072] The technical solution of this embodiment iteratively trains the initial neural network model based on the images to be trained in the training set, and evaluates at least one intermediate neural network model obtained through training based on the images to be tested in the test set, and uses the intermediate neural network model whose evaluation results meet the preset evaluation criteria as the target neural network model. Compared with the target neural network model selected by the existing training method based only on the training set, the target neural network model selected by this technical solution has a better segmentation effect on unknown images, thereby further ensuring the accuracy of image segmentation of the target neural network model.

[0073] Example 3

[0074] Figure 4 Schematic diagram of an image segmentation device provided in Embodiment 3 of the present invention. This embodiment is applicable to image segmentation. The device can be implemented using software and / or hardware and can be configured in a terminal device. The image segmentation device includes a target image acquisition module 310 and a segmentation result output module 320.

[0075] The target image acquisition module 310 is used to acquire the target image to be segmented;

[0076] The segmentation result output module 320 is used to input the target image into the pre-trained target neural network model to obtain the output segmentation result corresponding to the target image;

[0077] The target neural network model is trained based on a standard segmentation image and standard delineation information, wherein the standard delineation information is used to represent the positioning information of the image segmentation unit in the standard segmentation image in at least one dimensional direction.

[0078] The technical solution of this embodiment solves the problem of poor image segmentation effect of existing neural network models by training the neural network model based on standard segmentation images and standard outline information. When the neural network model segments the image, it not only focuses on the visual feature information in the image but also focuses on the anisotropy information in the image, thereby improving the accuracy of image segmentation.

[0079] Based on the above technical solution, optionally, the target neural network model includes a feature extraction module, an image segmentation module, an intermediate deep supervision module and a terminal deep supervision module. The feature extraction module is used to output a feature vector based on the input target image, the intermediate deep supervision module is used to output intermediate delineation information based on the input feature vector, the image segmentation module is used to output a target segmentation image corresponding to the target image based on the input feature vector and the intermediate delineation information, and the terminal deep supervision module is used to output target delineation information based on the target segmentation image.

[0080] On the basis of the above technical solution, optionally, the image segmentation module includes a first image segmentation module and at least one second image segmentation module, and accordingly, the intermediate deep supervision module includes a first intermediate deep supervision module and at least one second intermediate deep supervision module, the first intermediate deep supervision module is used to output first intermediate delineation information based on the input feature vector, the first image segmentation module is used to output a first segmented image based on the input feature vector and the first intermediate delineation information, the second intermediate deep supervision module is used to output second delineation information based on the first segmented image, and the second image segmentation module is used to output a second segmented image based on the first segmented image and the second delineation information; wherein, the second segmented image includes the target segmented image.

[0081] Based on the above technical solution, optionally, the standard segmentation image includes a training standard segmentation image, and the standard delineation information includes training standard delineation information. Accordingly, the device further includes:

[0082] The target neural network model training module is used to obtain the images to be trained in the training set and the images to be tested in the test set; the images to be trained are input into the initial neural network model, and the initial neural network model is iteratively trained based on the training standard segmentation image, the training standard delineation information, the training prediction segmentation image and the training prediction delineation information output by the initial neural network model to obtain at least one intermediate neural network model; the images to be tested are input into each intermediate neural network model, and the evaluation results corresponding to each intermediate neural network model are determined based on the output test prediction segmentation image; the intermediate neural network model whose evaluation results meet the preset evaluation criteria is used as the target neural network model.

[0083] Based on the above technical solution, an optional target neural network model training module is specifically used to:

[0084] Determining a first loss function based on the training predicted segmentation image and the training standard segmentation image, and determining a second loss function based on the training predicted delineation information and the training standard delineation information;

[0085] Based on the first loss function, the second loss function and the preset optimizer, the initial neural network model is iteratively trained to obtain at least one intermediate neural network model.

[0086] On the basis of the above technical solution, optionally, the device further includes:

[0087] The training standard delineation information generation module is used to obtain a training standard segmentation image, determine the target dimension direction with the highest anisotropy based on the image unit sizes in at least two dimensional directions of the image units in the training standard segmentation image; obtain the positioning information of the image segmentation units in the training standard segmentation image in the target dimension direction, and generate training standard delineation information based on the positioning information.

[0088] On the basis of the above technical solution, optionally, the device further includes:

[0089] The training set determination module is used to obtain an original image set and preprocess the original images in the original image set respectively to obtain preprocessed original images; based on a preset ratio, the preprocessed original images are divided into images to be trained in the training set and images to be tested in the test set; data enhancement processing is performed on the images to be trained in the training set, and the data-enhanced images to be trained are added to the training set.

[0090] The image segmentation device provided by the embodiment of the present invention can be used to execute the image segmentation method provided by the embodiment of the present invention, and has the corresponding functions and beneficial effects of executing the method.

[0091] It is worth noting that in the embodiment of the above-mentioned image segmentation device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0092] Example 4

[0093] Figure 5 This is a structural diagram of an electronic device provided by a fourth embodiment of the present invention. This embodiment of the present invention provides services for implementing the image segmentation method of the above embodiment of the present invention, and can configure the image segmentation device in the above embodiment. Figure 5 A block diagram of an exemplary electronic device 12 suitable for implementing embodiments of the present invention is shown. Figure 5 The electronic device 12 shown is only an example and should not limit the functionality and scope of use of the embodiments of the present invention.

[0094] like Figure 5 As shown, electronic device 12 is implemented as a general-purpose computing device. Components of electronic device 12 may include, but are not limited to, one or more processors or processing units 16, system memory 28, and a bus 18 that connects various system components (including system memory 28 and processing unit 16).

[0095] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. Examples of these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0096] The electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0097] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be configured to read and write non-removable, non-volatile magnetic media ( Figure 5 Not shown, often called a "hard drive"). Although Figure 5 Not shown, a magnetic disk drive for reading and writing to a removable non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of various embodiments of the present invention.

[0098] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment. Program modules 42 generally implement the functions and / or methods of the embodiments described herein.

[0099] The electronic device 12 may also communicate with one or more external devices 14 (e.g., a keyboard, a pointing device, a display 24, etc.), one or more devices that enable a user to interact with the electronic device 12, and / or any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may be performed through an input / output (I / O) interface 22. Furthermore, the electronic device 12 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 20. Figure 5As shown, the network adapter 20 communicates with the other modules of the electronic device 12 via the bus 18. It should be understood that although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0100] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28 , such as implementing the image segmentation method provided by the embodiment of the present invention.

[0101] Through the above-mentioned electronic device, the problem of poor image segmentation effect of the existing neural network model is solved, so that the neural network model not only focuses on the visual feature information in the image but also focuses on the anisotropy information in the image when segmenting the image, thereby improving the accuracy of image segmentation.

[0102] Example 5

[0103] The fifth embodiment of the present invention further provides a storage medium containing computer-executable instructions. When the computer-executable instructions are executed by a computer processor, the computer-executable instructions are used to perform an image segmentation method. The method includes:

[0104] Obtain the target image to be segmented;

[0105] Input the target image into the pre-trained target neural network model to obtain the output segmentation result corresponding to the target image;

[0106] The target neural network model is trained based on a standard segmentation image and standard delineation information, wherein the standard delineation information is used to represent the positioning information of the image segmentation unit in the standard segmentation image in at least one dimensional direction.

[0107] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or components, or any combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by an instruction execution system, device or device or used in combination with it.

[0108] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0109] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0110] The computer program code for performing the operations of the present invention can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0111] Of course, the computer executable instructions of the storage medium provided by the embodiment of the present invention are not limited to the above method operations, but can also execute related operations in the image segmentation method provided by any embodiment of the present invention.

[0112] Note that the above are only preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will appreciate that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments, and substitutions are possible for those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments and may include many other equivalent embodiments without departing from the scope of the present invention. The scope of the present invention is determined by the scope of the appended claims.

Claims

1. An image segmentation method, characterized in that: include: Obtain the target image to be segmented; Inputting the target image into a pre-trained target neural network model to obtain an output segmentation result corresponding to the target image; The target neural network model is trained based on a standard segmentation image and standard delineation information, wherein the standard delineation information is used to represent positioning information of an image segmentation unit in the standard segmentation image in at least one dimensional direction; The standard segmentation image includes a training standard segmentation image, and the standard delineation information includes training standard delineation information. Accordingly, the training method of the target neural network model includes: Obtain the images to be trained in the training set and the images to be tested in the test set; Inputting the image to be trained into an initial neural network model, and iteratively training the initial neural network model based on the training standard segmentation image, the training standard delineation information, the training prediction segmentation image and the training prediction delineation information output by the initial neural network model to obtain at least one intermediate neural network model; Inputting the image to be tested into each of the intermediate neural network models, and determining the evaluation results corresponding to each of the intermediate neural network models based on the output test prediction segmentation image; The intermediate neural network model whose evaluation results meet the preset evaluation criteria is used as the target neural network model; The method further comprises: Obtaining a training standard segmented image, and based on the image unit sizes of image units in at least two dimensional directions in the training standard segmented image, comparing the image unit sizes in each dimensional direction, and determining the dimensional direction with the largest image unit size as the target dimensional direction with the highest anisotropy; The positioning information of the image segmentation unit in the training standard segmented image in the target dimension direction is obtained, and the training standard delineation information is generated based on the positioning information.

2. The method according to claim 1, characterized in that The target neural network model includes a feature extraction module, an image segmentation module, an intermediate deep supervision module and a terminal deep supervision module. The feature extraction module is used to output a feature vector based on the input target image, the intermediate deep supervision module is used to output intermediate delineation information based on the input feature vector, the image segmentation module is used to output a target segmentation image corresponding to the target image based on the input feature vector and the intermediate delineation information, and the terminal deep supervision module is used to output target delineation information based on the target segmentation image.

3. The method according to claim 2, characterized in that The image segmentation module includes a first image segmentation module and at least one second image segmentation module. Accordingly, the intermediate deep supervision module includes a first intermediate deep supervision module and at least one second intermediate deep supervision module. The first intermediate deep supervision module is used to output first intermediate delineation information based on the input feature vector, the first image segmentation module is used to output a first segmented image based on the input feature vector and the first intermediate delineation information, the second intermediate deep supervision module is used to output second delineation information based on the first segmented image, and the second image segmentation module is used to output a second segmented image based on the first segmented image and the second delineation information; wherein, the second segmented image includes the target segmented image.

4. The method according to claim 1, wherein The iterative training of the initial neural network model based on the training standard segmentation image, the training standard delineation information, the training prediction segmentation image and the training prediction delineation information output by the initial neural network model to obtain at least one intermediate neural network model includes: Determining a first loss function based on the training predicted segmentation image and the training standard segmentation image, and determining a second loss function based on the training predicted delineation information and the training standard delineation information; Based on the first loss function, the second loss function and a preset optimizer, the initial neural network model is iteratively trained to obtain at least one intermediate neural network model.

5. The method according to claim 1, wherein The method further comprises: Acquire an original image set, and preprocess the original images in the original image set respectively to obtain preprocessed original images; Based on a preset ratio, the preprocessed original images are divided into images to be trained in a training set and images to be tested in a test set; Data enhancement processing is performed on the images to be trained in the training set, and the data-enhanced images to be trained are added to the training set.

6. An image segmentation device, characterized in that: include: A target image acquisition module is used to acquire the target image to be segmented; A segmentation result output module is used to input the target image into a pre-trained target neural network model to obtain an output segmentation result corresponding to the target image; The target neural network model is trained based on a standard segmentation image and standard delineation information, wherein the standard delineation information is used to represent positioning information of an image segmentation unit in the standard segmentation image in at least one dimensional direction; The standard segmentation image includes a training standard segmentation image, and the standard delineation information includes training standard delineation information. Accordingly, the apparatus further includes: A target neural network model training module is used to obtain an image to be trained in a training set and an image to be tested in a test set; input the image to be trained into an initial neural network model, iteratively train the initial neural network model based on a training standard segmentation image, training standard delineation information, a training prediction segmentation image output by the initial neural network model, and training prediction delineation information to obtain at least one intermediate neural network model; input the image to be tested into each of the intermediate neural network models, and determine an evaluation result corresponding to each of the intermediate neural network models based on the output test prediction segmentation image; and use the intermediate neural network model whose evaluation result meets the preset evaluation standard as the target neural network model; The device further comprises: A training standard delineation information generation module is used to obtain a training standard segmentation image, compare the image unit sizes in each dimensional direction based on the image unit sizes in at least two dimensional directions in the training standard segmentation image, and determine that the dimensional direction with the largest image unit size is the target dimensional direction with the highest anisotropy; obtain the positioning information of the image segmentation unit in the training standard segmentation image in the target dimensional direction, and generate training standard delineation information based on the positioning information.

7. An electronic device, characterized in that: The electronic device comprises: one or more processors; a memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the image segmentation method according to any one of claims 1 to 5.

8. A storage medium containing computer-executable instructions, characterized in that: When the computer executable instructions are executed by a computer processor, they are used to perform the image segmentation method according to any one of claims 1 to 5.

Images