Image segmentation method and device and storage medium
By acquiring the region distance in the image and using neural network update technology, the problem of blurred image segmentation results in the prior art is solved, and a more accurate and robust image segmentation effect is achieved.
Patent Information
- Application Number
- CN202411289985.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-14
- Publication Date
- 2024-12-27
AI Technical Summary
Existing image segmentation methods cannot accurately identify and segment the boundaries between large objects and small objects, resulting in blurred and inaccurate segmentation results.
By obtaining the distance between the first area and the external pixel points in the image to be segmented to the first area, the second area is determined, and these areas are updated using the second neural network, and the context segmentation method is performed in combination with the 3DU-Net and U-Net neural networks. , integrate local and global information to improve segmentation accuracy.
It improves the accuracy and robustness of image segmentation, clearly defines the region boundaries, reduces missegment and blurring, and enhances the continuity and consistency of segmentation results.
Smart Images
Figure CN119205812A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image segmentation, and in particular to an image segmentation method, device and storage medium. Background Art
[0002] Existing image segmentation methods are applied to a large number of visual recognition scenarios. A large number of existing images contain both large objects and small objects. During image segmentation, both large objects and small objects need to be identified and segmented at the same time.
[0003] In the prior art, small objects between large objects are ignored during the image segmentation process because they cannot be identified or have blurred boundaries, and accurate image segmentation cannot be performed.
[0004] However, existing image processing methods are unable to accurately identify the edges of small objects and simultaneously perform effective image segmentation between large objects and small objects. Summary of the invention
[0005] The present invention provides an image segmentation method, device and storage medium to solve the problem that the image cannot be accurately segmented in the prior art.
[0006] In a first aspect, the present application provides an image segmentation method, comprising:
[0007] Acquire an image to be segmented, and acquire a plurality of first regions in the image to be segmented through a preset first neural network;
[0008] Obtaining the distances from each pixel outside the first area in the image to be segmented to each first area;
[0009] Determine a plurality of second regions according to the distances from each pixel point to each first region, input all the second regions and the image to be segmented into a second neural network, and obtain updated second regions;
[0010] The image to be segmented is segmented according to the first region and the updated second region.
[0011] In this way, the distance information between the pixel point and different areas is used to improve the accuracy of segmentation. The second area can be determined based on the first area to accurately identify small objects around large objects, thereby eliminating the situation where the second area cannot be identified because it is too small. By updating the second area through the second neural network, the boundary of the second area can be accurately identified to prevent blurring between the edge of the second area and the first area. In this way, the spatial relationship between different areas is fully utilized, the characteristics of different areas in the image to be segmented are adapted, and the accuracy of image segmentation is improved.
[0012] Furthermore, the step of obtaining the image to be segmented and obtaining a plurality of first regions in the image to be segmented by using a preset first neural network is specifically as follows:
[0013] The image to be segmented is cropped to a preset size, and a plurality of first Ro l regions are obtained through a 3DU-Net neural network;
[0014] According to the positional relationship between each Ro l region, all the first Ro l regions are updated by the context segmentation method, and all the updated first Ro l regions are determined as the first region.
[0015] In this way, by using the 3DU-Net neural network to obtain local areas and updating these areas according to the positional relationship and contextual information, it helps to improve the accuracy and robustness of segmentation. The fine processing of local areas and the integration of global information can make the segmentation results more accurate. And by cropping the image to a preset size and segmenting it in the local area, the overall computational complexity can be reduced and the computational efficiency can be improved. Furthermore, by updating these local areas according to the positional relationship between the areas and the contextual information, the local information can be integrated into the global information to achieve local-global image segmentation.
[0016] Furthermore, the obtaining of the distances from each pixel outside the first region in the image to be segmented to each first region is specifically:
[0017] Acquire all external pixel points outside the first area in the image to be segmented;
[0018] The distances from each external pixel to all the pixels in each first region are calculated, and the shortest distance from the external pixel to each first region is determined to be the distance from the external pixel to each first region.
[0019] In this way, by calculating the shortest distance from the external pixel to all the pixels in the first region, the boundary area can be determined more accurately, and the accuracy of the segmentation result can be improved. This helps to reduce the fuzzy area in the segmentation result, making the boundary clearer and more accurate. And by considering the shortest distance from the external pixel to each first region, the segmentation result can be smoothed to avoid over-segmentation or breakage. Thereby improving the continuity and consistency of the segmentation result, making the image boundary more natural.
[0020] Furthermore, the determining of the plurality of second regions according to the distances from each pixel point to each first region is specifically:
[0021] Determine whether the distance from each pixel point to each first area is greater than a preset distance;
[0022] When there is a pixel point whose distance to each first area is less than a preset distance, the pixel point is determined to be a pixel point in the second area;
[0023] When there is a pixel point whose distance from each first area is greater than a preset distance, it is determined that the pixel point is not in the second area.
[0024] By setting a preset distance to determine the second area, the boundary between the first area and the second area can be clearly defined, making the segmentation result more interpretable. And by setting a preset distance to determine the second area, it is possible to effectively avoid erroneously assigning pixels that should belong to the first area to the second area, thereby reducing the occurrence of mis-segmentation. This improves the accuracy and reliability of the segmentation results.
[0025] Furthermore, the step of inputting all the second regions and the image to be segmented into a second neural network to obtain updated second regions is specifically as follows:
[0026] dividing the second region into a plurality of parts of preset sizes, and obtaining a third region by a context segmentation method;
[0027] Determine a second Ro l region according to the position of the third region in the image to be segmented, and input the second Ro l region and the third region into a second neural network to obtain an updated second region;
[0028] The second neural network is a U-Net neural network, and the second neural network is obtained by training a training set consisting of a number of images with preset labels.
[0029] After the second region is divided into parts of preset size, a more detailed division of the region can be achieved, making the segmentation result more refined and accurate. The subtle structures and features in the image can be better captured, and the detail display capability of the segmentation result can be improved. And the second Ro l region is determined according to the position of the third region in the image to be segmented, and the position information can be introduced to guide and optimize the segmentation result, thereby improving the spatial consistency and accuracy of the segmentation result. This helps to avoid positional dislocation or discontinuity in the segmentation result.
[0030] In a second aspect, the present application provides an image segmentation device, including: a first area module, a distance module, a second area module and a segmentation module;
[0031] The first region module is used to obtain the image to be segmented, and obtain a plurality of first regions in the image to be segmented through a preset first neural network;
[0032] The distance module is used to obtain the distances from each pixel outside the first area in the image to be segmented to each first area;
[0033] The second region module is used to determine a plurality of second regions according to the distance from each pixel point to each first region, and input all the second regions and the image to be segmented into a second neural network to obtain updated second regions;
[0034] The segmentation module is used to segment the image to be segmented according to the first region and the updated second region.
[0035] Further, the first area module includes: a cropping unit and an area unit;
[0036] The cropping unit is used to crop the image to be segmented into a preset size, and obtain a plurality of first Ro l regions through a 3DU-Net neural network;
[0037] The regional unit is used to update all first Ro l regions through the context segmentation method according to the positional relationship between each Ro l region, and determine all the updated first Ro l regions as the first region.
[0038] Further, the distance module includes: an external pixel unit and a shortest distance unit;
[0039] The external pixel point unit is used to obtain all external pixel points outside the first area in the image to be segmented;
[0040] The shortest distance unit is used to calculate the distance from each external pixel to all pixels in each first area, and determine the shortest distance from the external pixel to each first area as the distance from the external pixel to each first area.
[0041] Further, the second area module includes: a first judgment unit and an area division unit;
[0042] The first judgment unit is used to judge whether the distance from each pixel point to each first area is greater than a preset distance;
[0043] The area division unit is used for determining that a pixel point is a pixel point in the second area when the first judgment unit determines that the distance between the pixel point and each first area is less than a preset distance;
[0044] The area division unit is further configured to determine that a pixel point is not in the second area when the first judgment unit determines that the distance between a pixel point and each first area is greater than a preset distance.
[0045] Further, the second area module includes: a third area unit and an update unit;
[0046] The third region unit is used to divide the second region into a plurality of parts of preset sizes, and obtain the third region by a context segmentation method;
[0047] The updating unit is used to determine the second Ro l region according to the position of the third region in the image to be segmented, and input the second Ro l region and the third region into the second neural network to obtain an updated second region;
[0048] The second neural network is a U-Net neural network, and the second neural network is obtained by training a training set consisting of a number of images with preset labels.
[0049] Correspondingly, the present invention also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the image segmentation methods described above.
[0050] In this way, the distance information between the pixel point and different areas is used to improve the accuracy of segmentation. The second area can be determined based on the first area to accurately identify small objects around large objects, thereby eliminating the situation where the second area cannot be identified because it is too small. By updating the second area through the second neural network, the boundary of the second area can be accurately identified to prevent blurring between the edge of the second area and the first area. In this way, the spatial relationship between different areas is fully utilized, the characteristics of different areas in the image to be segmented are adapted, and the accuracy of image segmentation is improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] Figure 1 : A flow chart of an embodiment of an image segmentation method provided by the present invention;
[0052] Figure 2 : A schematic diagram of the atrioventricular and mitral valves of an embodiment of an image segmentation method provided by the present invention, wherein (a) is a schematic diagram of the process of obtaining the updated second region by the context segmentation method and (b) is a visualized schematic diagram of the valve annotation;
[0053] Figure 3 : A schematic diagram of a second region in an image to be segmented according to an embodiment of an image segmentation method provided by the present invention, wherein (a) is a schematic diagram of the segmentation result of the second region and (b) AV and (c) MV information thereof;
[0054] Figure 4 : A module structure diagram of an embodiment of an image segmentation device provided by the present invention. DETAILED DESCRIPTION
[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0056] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.
[0057] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.
[0058] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.
[0059] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.
[0060] Embodiment 1
[0061] Please refer to Figure 1 , an image segmentation method provided by an embodiment of the present invention, includes steps S1 to S4, each step is specifically as follows:
[0062] Step S1: obtaining an image to be segmented, and obtaining a plurality of first regions in the image to be segmented through a preset first neural network;
[0063] In one embodiment, the step of obtaining the image to be segmented and obtaining a plurality of first regions in the image to be segmented by using a preset first neural network is specifically as follows:
[0064] The image to be segmented is cropped into a preset size, and a plurality of first Ro l (Region of Interest) regions are obtained through a 3DU-Net neural network;
[0065] According to the positional relationship between each Ro l region, all the first Ro l regions are updated by the context segmentation method, and all the updated first Ro l regions are determined as the first region.
[0066] In this way, by using the 3DU-Net neural network to obtain local areas and updating these areas according to the positional relationship and contextual information, it helps to improve the accuracy and robustness of segmentation. The fine processing of local areas and the integration of global information can make the segmentation results more accurate. And by cropping the image to a preset size and segmenting it in the local area, the overall computational complexity can be reduced and the computational efficiency can be improved. Furthermore, by updating these local areas according to the positional relationship between the areas and the contextual information, the local information can be integrated into the global information to achieve local-global image segmentation.
[0067] In an optional specific embodiment, the image to be segmented may be a cropped CT image containing the area to be segmented, which can eliminate irrelevant areas, reduce redundant calculations, and improve efficiency.
[0068] The step of cropping the image to be segmented into a preset size specifically includes adjusting the CT image to a size of 128×128×128.
[0069] In a specific embodiment, a plurality of first Ro l regions are obtained through a 3D U-Net neural network, specifically:
[0070] When the image to be segmented is a regional image including the heart and its connected vascular trunks, the left atrium LA (Left Atr i um), left ventricle LV (Left Ventr icle) and aorta AO (Aorta) regions in the region to be segmented are obtained as three first Ro l regions through the 3DU-Net neural network;
[0071] In an optional embodiment, the first Ro l area is updated by corroding and removing small isolated islands.
[0072] In a specific embodiment, each first Ro l region is standardized and divided into a number of blocks of size 128×128×128 in a sliding window manner, and all first Ro l regions are updated by the context segmentation method based on the anatomical spatial relationship between LA, LV and AO, and all the updated first Ro l regions are determined as LA, LV and AO regions as the first regions.
[0073] It should be noted that the 3D U-Net is a 3D version of the U-Net architecture, which is specifically used to process image segmentation tasks of three-dimensional volume data. The network structure of 3D U-Net is similar to that of traditional U-Net, and also includes an encoder and a decoder, as well as jump connections for feature fusion.
[0074] The context segmentation method includes feature extraction, context modeling, segmentation decision and post-processing.
[0075] The context model is the anatomical spatial relationship between LA, LV and AO, and the segmentation decision is to make a segmentation decision for each pixel based on the anatomical spatial relationship between LA, LV and AO, and assign the pixel points to different areas or categories.
[0076] The segmentation decision is cluster segmentation, and the cluster segmentation is to cluster pixels into different groups based on the similarity or feature distance between pixels to achieve image segmentation. This method is suitable for segmenting images based on pixel features.
[0077] Step S2: Obtaining the distances from each pixel outside the first area in the image to be segmented to each first area;
[0078] In one embodiment, the obtaining of the distances from each pixel outside the first region in the image to be segmented to each first region is specifically:
[0079] Acquire all external pixel points outside the first area in the image to be segmented;
[0080] The distances from each external pixel to all the pixels in each first region are calculated, and the shortest distance from the external pixel to each first region is determined to be the distance from the external pixel to each first region.
[0081] In a specific embodiment, the distances from each external pixel to all pixels in each first region are calculated, and the shortest distance from the external pixel to each first region is determined to be the distance from the external pixel to each first region, specifically:
[0082] The Euclidean distance of each external pixel to each pixel in the AO, LV, and LA regions is calculated. Thus, each external pixel obtains three values corresponding to the shortest distance to AO, LV, or LA, respectively.
[0083] In this way, by calculating the shortest distance from the external pixel to all the pixels in the first region, the boundary area can be determined more accurately, and the accuracy of the segmentation result can be improved. This helps to reduce the fuzzy area in the segmentation result, making the boundary clearer and more accurate. And by considering the shortest distance from the external pixel to each first region, the segmentation result can be smoothed to avoid over-segmentation or breakage. Thereby improving the continuity and consistency of the segmentation result, making the image boundary more natural.
[0084] Step S3: determining a number of second regions according to the distances from each pixel point to each first region, inputting all the second regions and the image to be segmented into a second neural network to obtain updated second regions;
[0085] In one embodiment, the determining of the plurality of second regions according to the distances from each pixel point to each first region is specifically:
[0086] Determine whether the distance from each pixel point to each first area is greater than a preset distance;
[0087] When there is a pixel point whose distance to each first area is less than a preset distance, the pixel point is determined to be a pixel point in the second area;
[0088] When there is a pixel point whose distance from each first area is greater than a preset distance, it is determined that the pixel point is not in the second area.
[0089] In a specific embodiment, the preset distance is between [0, 60] to the AO and LV areas, and between [0, 30] to the LA area.
[0090] This distance effectively includes the aortic root and the junction of the LA and LV, the locations of the atrioventricular (AV) and mitral valve (MV), respectively.
[0091] By setting a preset distance to determine the second area, the boundary between the first area and the second area can be clearly defined, making the segmentation result more interpretable. And by setting a preset distance to determine the second area, it is possible to effectively avoid erroneously assigning pixels that should belong to the first area to the second area, thereby reducing the occurrence of mis-segmentation. This improves the accuracy and reliability of the segmentation results.
[0092] In one embodiment, the step of inputting all the second regions and the image to be segmented into a second neural network to obtain an updated second region is as follows:
[0093] dividing the second region into a plurality of parts of preset sizes, and obtaining a third region by a context segmentation method;
[0094] Determine a second Ro l region according to the position of the third region in the image to be segmented, and input the second Ro l region and the third region into a second neural network to obtain an updated second region;
[0095] like Figure 2 Schematic diagram of the atrioventricular and mitral valves, where (a) is a schematic diagram of the process of obtaining the updated second region through the context segmentation method and (b) is a visualization diagram of the valve annotation.
[0096] The second neural network is a U-Net neural network, and the second neural network is obtained by training a training set consisting of a number of images with preset labels.
[0097] In a specific embodiment, the second region is divided into a plurality of parts of preset sizes, and the third region is obtained by a context segmentation method, specifically:
[0098] Determine position information between a plurality of second regions according to information about positions of relevant anatomical structures, and update the second regions according to a context segmentation method to obtain a third region;
[0099] The updating of the second region according to the context segmentation method is specifically as follows:
[0100] The second region is divided into multiple blocks of size 128×128×128, and is input into U-Net in a sliding window manner for segmentation to obtain the third region.
[0101] In a specific embodiment, the second Ro l region is determined according to the position of the third region in the image to be segmented, specifically:
[0102] Generate an image frame diagram based on the position of the third area in the image to be segmented and determine the second Ro l area.
[0103] like Figure 3Schematic diagram of the second region in the image to be segmented, wherein (a) is a schematic diagram of the segmentation result of the second region and (b) AV and (c) MV information.
[0104] Step S4: Segment the image to be segmented according to the first region and the updated second region.
[0105] Embodiment 2
[0106] This embodiment provides an image segmentation method for the present invention applied to a cardiac image segmentation scenario. In cardiac image segmentation, in order to avoid the recurrence of the disease after surgery, myocardial resection requires surgeons to have rich experience, and the thickness and range of hypertrophic myocardial resection are not easy to accurately grasp. In the prior art, it is often necessary to visualize the desired patient's heart structure in 3D form through three-dimensional printing to help surgeons formulate personalized surgical strategies for patients. Existing image segmentation methods cannot effectively segment large tissue structures (such as the left ventricle, left atrium, aorta and preserved myocardium) and small anatomical structures in the heart, and the preserved myocardium and the resected myocardium belong to the same category, and there is no obvious boundary between the two, which makes it difficult to accurately segment them.
[0107] In this embodiment, an image segmentation method provided by the present invention is applied to a cardiac image segmentation scenario to more accurately segment the cardiac image.
[0108] Extract the region containing the heart and its connected vascular trunks. The input is the CT image of the original size, and the output is a cropped CT image containing only the target region.
[0109] Specifically, the CT image is resized to 128×128×128 and then input into 3D U-Net for LA, LV and AO segmentation to obtain the RoI region. After segmentation, some simple post-processing techniques are applied, such as corrosion and removal of small isolated islands.
[0110] The cropped CT image containing only the target area is input to the 3D U-Net, and the RoI regions of AO, LA, LV, myocardium, and resected myocardium are output.
[0111] The RoI region was normalized and divided into blocks of size 128 × 128 × 128 in a sliding window manner. The first region was determined by contextually segmenting the RoI region using the anatomical spatial relationship between AO, LA, and LV.
[0112] The Euclidean distance between each first region outer pixel and each inner pixel of AO, LV or LA is calculated, and the shortest distance is assigned. Thus, each outer pixel obtains three values corresponding to the shortest distance of AO, LV or LA, respectively.
[0113] The pixels whose distances AO and LV are between [0, 60] and whose distance LA is between [0, 30] are regarded as the second area.
[0114] The second region is divided into multiple blocks of size 128×128×128, and is put into U-Net for segmentation in a sliding window manner to obtain the third region;
[0115] According to the position of the third region in the image to be segmented, the second Ro l region is determined, and the second Ro l region and the third region are input into the second neural network to obtain an updated second region.
[0116] The second region is the finely segmented AV and MV regions.
[0117] After the second region is divided into parts of preset size, a more detailed division of the region can be achieved, making the segmentation result more refined and accurate. The subtle structures and features in the image can be better captured, and the detail display capability of the segmentation result can be improved. And the second Ro l region is determined according to the position of the third region in the image to be segmented, and the position information can be introduced to guide and optimize the segmentation result, thereby improving the spatial consistency and accuracy of the segmentation result. This helps to avoid positional dislocation or discontinuity in the segmentation result.
[0118] Embodiment 3
[0119] Please refer to Figure 4 , an image segmentation device provided by an embodiment of the present invention, comprising: a first area module 210, a distance module 220, a second area module 230 and a segmentation module 240;
[0120] The first region module 210 is used to obtain the image to be segmented, and obtain a plurality of first regions in the image to be segmented through a preset first neural network;
[0121] The distance module 220 is used to obtain the distances from each pixel outside the first area in the image to be segmented to each first area;
[0122] The second region module 230 is used to determine a plurality of second regions according to the distances from each pixel point to each first region, and input all the second regions and the image to be segmented into a second neural network to obtain updated second regions;
[0123] The segmentation module 240 is used to segment the image to be segmented according to the first region and the updated second region.
[0124] Further, the first area module includes: a cropping unit and an area unit;
[0125] The cropping unit is used to crop the image to be segmented into a preset size, and obtain a plurality of first Ro l regions through a 3DU-Net neural network;
[0126] The regional unit is used to update all first Ro l regions through the context segmentation method according to the positional relationship between each Ro l region, and determine all the updated first Ro l regions as the first region.
[0127] Further, the distance module includes: an external pixel unit and a shortest distance unit;
[0128] The external pixel point unit is used to obtain all external pixel points outside the first area in the image to be segmented;
[0129] The shortest distance unit is used to calculate the distance from each external pixel to all pixels in each first area, and determine the shortest distance from the external pixel to each first area as the distance from the external pixel to each first area.
[0130] Further, the second area module includes: a first judgment unit and an area division unit;
[0131] The first judgment unit is used to judge whether the distance from each pixel point to each first area is greater than a preset distance;
[0132] The area division unit is used for determining that a pixel point is a pixel point in the second area when the first judgment unit determines that the distance between the pixel point and each first area is less than a preset distance;
[0133] The area division unit is further configured to determine that a pixel point is not in the second area when the first judgment unit determines that the distance between a pixel point and each first area is greater than a preset distance.
[0134] Further, the second area module includes: a third area unit and an update unit;
[0135] The third region unit is used to divide the second region into a plurality of parts of preset sizes, and obtain the third region by a context segmentation method;
[0136] The updating unit is used to determine the second Ro l region according to the position of the third region in the image to be segmented, and input the second Ro l region and the third region into the second neural network to obtain an updated second region;
[0137] The second neural network is a U-Net neural network, and the second neural network is obtained by training a training set consisting of a number of images with preset labels.
[0138] Embodiment 4
[0139] Correspondingly, the present invention also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the image segmentation method described in any one of the above embodiments.
[0140] In this way, the distance information between the pixel point and different areas is used to improve the accuracy of segmentation. The second area can be determined based on the first area to accurately identify small objects around large objects, thereby eliminating the situation where the second area cannot be identified because it is too small. By updating the second area through the second neural network, the boundary of the second area can be accurately identified to prevent blurring between the edge of the second area and the first area. In this way, the spatial relationship between different areas is fully utilized, the characteristics of different areas in the image to be segmented are adapted, and the accuracy of image segmentation is improved.
[0141] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.
Claims
1. An image segmentation method, characterized in that: include: Acquire an image to be segmented, and acquire a plurality of first regions in the image to be segmented through a preset first neural network; Obtaining the distances from each pixel outside the first area in the image to be segmented to each first area; Determine a plurality of second regions according to the distances from each pixel point to each first region, input all the second regions and the image to be segmented into a second neural network, and obtain updated second regions; The image to be segmented is segmented according to the first region and the updated second region.
2. The image segmentation method according to claim 1, characterized in that: The step of obtaining the image to be segmented and obtaining a plurality of first regions in the image to be segmented by using a preset first neural network is specifically as follows: The image to be segmented is cropped to a preset size, and a plurality of first Rol regions are obtained through a 3DU-Net neural network; According to the positional relationship between the various RoI regions, all the first RoI regions are updated by using a context segmentation method, and all the updated first RoI regions are determined as the first region.
3. The image segmentation method according to claim 1, characterized in that: The obtaining of the distances from each pixel point outside the first region in the image to be segmented to each first region is specifically: Acquire all external pixel points outside the first area in the image to be segmented; The distances from each external pixel to all the pixels in each first region are calculated, and the shortest distance from the external pixel to each first region is determined to be the distance from the external pixel to each first region.
4. The image segmentation method according to claim 1, characterized in that: The determining of the plurality of second regions according to the distances from each pixel point to each first region is specifically: Determine whether the distance from each pixel point to each first area is greater than a preset distance; When there is a pixel point whose distance to each first area is less than a preset distance, the pixel point is determined to be a pixel point in the second area; When there is a pixel point whose distance from each first area is greater than a preset distance, it is determined that the pixel point is not in the second area.
5. The image segmentation method according to claim 1, characterized in that: The step of inputting all the second regions and the image to be segmented into the second neural network to obtain the updated second regions is specifically as follows: dividing the second region into a plurality of parts of preset sizes, and obtaining a third region by a context segmentation method; Determine a second Ro l region according to the position of the third region in the image to be segmented, and input the second Ro l region and the third region into a second neural network to obtain an updated second region; The second neural network is a U-Net neural network, and the second neural network is obtained by training a training set consisting of a number of images with preset labels.
6. An image segmentation device, characterized in that: include: A first area module, a distance module, a second area module and a segmentation module; The first region module is used to obtain the image to be segmented, and obtain a plurality of first regions in the image to be segmented through a preset first neural network; The distance module is used to obtain the distances from each pixel outside the first area in the image to be segmented to each first area; The second region module is used to determine a plurality of second regions according to the distance from each pixel point to each first region, and input all the second regions and the image to be segmented into a second neural network to obtain updated second regions; The segmentation module is used to segment the image to be segmented according to the first region and the updated second region.
7. The image segmentation device according to claim 6, characterized in that: The first area module includes: a cropping unit and an area unit; The cropping unit is used to crop the image to be segmented into a preset size, and obtain a plurality of first Ro l regions through a 3DU-Net neural network; The regional unit is used to update all first Ro l regions through the context segmentation method according to the positional relationship between each Ro l region, and determine all the updated first Ro l regions as the first region.
8. The image segmentation device according to claim 6, characterized in that: The distance module includes: an external pixel unit and a shortest distance unit; The external pixel point unit is used to obtain all external pixel points outside the first area in the image to be segmented; The shortest distance unit is used to calculate the distance from each external pixel to all pixels in each first area, and determine the shortest distance from the external pixel to each first area as the distance from the external pixel to each first area.
9. The image segmentation device according to claim 6, characterized in that: The second area module includes: a first judgment unit and an area division unit; The first judgment unit is used to judge whether the distance from each pixel to each first area is greater than a preset distance; The area division unit is used for determining that a pixel point is a pixel point in the second area when the first judgment unit determines that the distance between the pixel point and each first area is less than a preset distance; The area division unit is further configured to determine that a pixel point is not in the second area when the first judgment unit determines that the distance between a pixel point and each first area is greater than a preset distance.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the image segmentation method according to any one of claims 1 to 5.