Multi-scale image segmentation method and device

Abstract

The present application provides a multi-scale image segmentation method and device, wherein the multi-scale image segmentation method includes: obtaining an image to be processed; dividing the image to be processed into sub-images corresponding to at least two segmentation scales according to at least two segmentation scales; adjusting the sub-images to meet preset conditions to obtain an intermediate image; performing image segmentation on target features in the intermediate image to obtain a target image area corresponding to the target features; and fusing the target image area to obtain a target image.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a multi-scale image segmentation method and device. Background Art

[0002] With the development of computer technology, image processing technology is increasingly widely used, such as image segmentation, image editing, etc. Image segmentation refers to dividing an image into several non-intersecting regions based on features such as grayscale, color, spatial texture, and geometric shape, so that these features show consistency or similarity in the same region, but show obvious differences between different regions. At the same time, in many image processing fields that have special requirements for image segmentation, such as the field of image segmentation for remote sensing images, the requirements for image segmentation accuracy are getting higher and higher. Summary of the invention

[0003] In view of this, the present application provides a multi-scale image segmentation method, a multi-scale image segmentation apparatus, a computing device, and a computer-readable storage medium.

[0004] The present application provides a multi-scale image segmentation method, comprising:

[0005] Get the image to be processed;

[0006] According to at least two segmentation scales, the image to be processed is segmented into sub-images corresponding to the at least two segmentation scales;

[0007] Adjusting the sub-image to meet a preset condition to obtain an intermediate image;

[0008] Performing image segmentation on the target features in the intermediate image to obtain a target image region corresponding to the target features;

[0009] The target image region is fused to obtain a target image.

[0010] Optionally, performing image segmentation on the target feature in the intermediate image to obtain a target image region corresponding to the target feature includes:

[0011] Calculating the probability value that the smallest image unit contained in the intermediate image is the target feature;

[0012] The minimum image unit whose probability value is greater than the first probability threshold is selected to form a target image area corresponding to the target feature.

[0013] Optionally, fusing the target image region to obtain the target image includes:

[0014] Calculating the optimized probability value of the minimum image unit contained in the image to be processed according to the probability value of the minimum image unit contained in the target image area;

[0015] The minimum image units whose optimization probability values ​​are greater than a second probability threshold are segmented from the image to be processed to form a target image area as the target image.

[0016] Optionally, fusing the target image region to obtain the target image includes:

[0017] According to a one-to-one correspondence between the sub-images and the intermediate images, determining a target image region obtained after image segmentation of the intermediate image corresponding to each segmentation scale;

[0018] Merge the target image area obtained after image segmentation of the intermediate image corresponding to each segmentation scale to obtain a merged image area corresponding to each segmentation scale;

[0019] The merged image regions corresponding to the at least two segmentation scales are fused to obtain the target image.

[0020] Optionally, fusing the merged image regions corresponding to the at least two segmentation scales to obtain the target image includes:

[0021] Calculating the optimized probability value of the minimum image unit contained in the image to be processed according to the probability value of the minimum image unit contained in the merged image area;

[0022] The minimum image units whose optimization probability values ​​are greater than a second probability threshold are segmented from the image to be processed to form a target image area as the target image.

[0023] Optionally, the optimized probability value includes: a weighted average of probability values ​​of minimum image units contained in the target image region of the at least two segmentation scales.

[0024] Optionally, the step of performing image segmentation on the target features in the intermediate image to obtain the target image region corresponding to the target features is implemented using an image segmentation model;

[0025] Among them, the input of the image segmentation model is the intermediate image, and the output is the target image area corresponding to the target feature.

[0026] Optionally, the image segmentation model performs image segmentation on the intermediate image, including:

[0027] Calculating the probability value that the smallest image unit contained in the intermediate image is the target feature;

[0028] The minimum image unit whose probability value is greater than the first probability threshold is selected to form a target image area corresponding to the target feature.

[0029] Optionally, there are overlapping areas between sub-images corresponding to any one of the at least two segmentation scales, and there are overlapping areas between intermediate images of any one of the segmentation scales;

[0030] Correspondingly, the probability value that the smallest image unit included in the overlapping area is the target feature refers to the average value of the probability values ​​that the smallest image unit included in the overlapping area is the target feature in the corresponding intermediate image.

[0031] Optionally, adjusting the sub-image to meet a preset condition to obtain an intermediate image includes:

[0032] The image sizes of the sub-images corresponding to the at least two segmentation scales are scaled up / down to obtain intermediate images of the at least two segmentation scales with equal image sizes.

[0033] Optionally, the scaling the image sizes of the sub-images corresponding to the at least two segmentation scales to obtain intermediate images of at least two segmentation scales with equal image sizes includes:

[0034] For a sub-image corresponding to at least one segmentation scale among the at least two segmentation scales, determining whether an image size of the sub-image corresponding to the segmentation scale is larger than a reference image size;

[0035] If yes, reducing the image size of the sub-image corresponding to the segmentation scale to be consistent with the size of the reference image, and obtaining an intermediate image of the segmentation scale;

[0036] If not, determining whether the image scale of the sub-image corresponding to the segmentation scale is smaller than the reference image size;

[0037] If it is smaller than, the image size of the sub-image corresponding to the segmentation size is enlarged to be consistent with the reference image size, so as to obtain an intermediate image of the segmentation scale.

[0038] Optionally, the step of performing image segmentation on the target features in the intermediate image to obtain the target image region corresponding to the target features is executed on a graphics processor in a batch processing manner.

[0039] Optionally, the image to be processed includes a remote sensing image;

[0040] The smallest image unit contained in the remote sensing image includes a pixel unit in the remote sensing image.

[0041] The present application also provides a multi-scale image segmentation device, comprising:

[0042] An image acquisition module is configured to acquire an image to be processed;

[0043] An image segmentation module is configured to segment the image to be processed into sub-images corresponding to at least two segmentation scales according to at least two segmentation scales;

[0044] A sub-image adjustment module, configured to adjust the sub-image to meet a preset condition to obtain an intermediate image;

[0045] An image segmentation module is configured to perform image segmentation on the target features in the intermediate image to obtain a target image region corresponding to the target features;

[0046] The region fusion module is configured to fuse the target image region to obtain a target image.

[0047] The present application also provides a computing device, including:

[0048] Memory and processor;

[0049] The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions:

[0050] Get the image to be processed;

[0051] According to at least two segmentation scales, the image to be processed is segmented into sub-images corresponding to the at least two segmentation scales;

[0052] Adjusting the sub-image to meet a preset condition to obtain an intermediate image;

[0053] Performing image segmentation on the target features in the intermediate image to obtain a target image region corresponding to the target features;

[0054] The target image region is fused to obtain a target image.

[0055] The present application also provides a computer-readable storage medium storing computer instructions, which implement the steps of the multi-scale image segmentation method when executed by a processor.

[0056] The multi-scale image segmentation method provided by the present application segments the image to be processed according to at least two segmentation scales respectively, into sub-images corresponding to the at least two segmentation scales, and then adjusts the sub-images at the at least two segmentation scales obtained after the segmentation to the same size, performs image segmentation on the target features of the intermediate image obtained after the adjustment, obtains the target image area corresponding to the target feature, and finally fuses the target image area, so as to improve the recognition accuracy in the image segmentation process, thereby improving the precision effect of image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a processing flow chart of a multi-scale image segmentation method provided in an embodiment of the present application;

[0058] Figure 2 is a schematic diagram of a multi-scale image segmentation process provided in an embodiment of the present application;

[0059] Figure 3 is a schematic diagram of another multi-scale image segmentation process provided in an embodiment of the present application;

[0060] Figure 4 is a schematic diagram of image segmentation provided in an embodiment of the present application;

[0061] Figure 5 is a processing flow chart of a multi-scale image segmentation method applied to an organ tissue segmentation scenario provided by an embodiment of the present specification;

[0062] Figure 6 is a schematic diagram of a multi-scale image segmentation device provided in an embodiment of the present application;

[0063] Figure 7 It is a structural block diagram of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0065] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0066] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0067] An embodiment of the present application provides a multi-scale image segmentation method, a multi-scale image segmentation device, a computing device, and a computer-readable storage medium. The following is a detailed description of each of the embodiments provided in this specification, and each step of the method is described.

[0068] An embodiment of a multi-scale image segmentation method provided by the present application is as follows:

[0069] See attached Figure 1 , which shows a processing flow chart of a multi-scale image segmentation method provided by this embodiment, see the attached Figure 2 , which shows a schematic diagram of a multi-scale image segmentation process provided by this embodiment, see the attached Figure 3 , which shows a schematic diagram of another multi-scale image segmentation process provided by this embodiment, see the attached Figure 4 , which shows a schematic diagram of image segmentation provided by this embodiment.

[0070] Step S102, obtaining an image to be processed.

[0071] The multi-scale image segmentation method provided in the present application starts from the multi-scale segmentation of the image to be processed, segments the image to be processed according to multiple segmentation scales, and performs multi-scale image segmentation on the basis of the multi-scale segmentation of the image to be processed, and improves the recognition accuracy in the image segmentation process by fusing the image segmentation results of the multi-scale segmentation, thereby improving the precision effect of image segmentation; as shown in the attached Figure 2 As shown, the character image is segmented according to two or more segmentation scales, and the sub-images obtained by segmentation are adjusted to obtain intermediate images of two or more segmentation scales, and the target features of the intermediate images are segmented to obtain the target image area corresponding to the target features, and the target image area is fused to obtain the target image. It should be noted that the target image is attached Figure 2 The area where the face of the person shown in shaded form in the bottom human image is located.

[0072] In particular, in some special image processing scenarios, such as remote sensing image segmentation scenarios, due to the particularity of remote sensing images, in the process of image segmentation of remote sensing images, a cutting process itself is required. By applying the multi-scale image segmentation method provided in the present application to the remote sensing image segmentation scenario, the remote sensing image is cut into at least two cutting scales in the cutting stage, and the image segmentation results of at least two cutting scales are fused in the merging stage, so as to achieve a deep fusion of remote sensing scenes and multi-scale image segmentation, which is more suitable for the application of remote sensing scenes, thereby further improving the precision and accuracy of remote sensing image segmentation.

[0073] The image to be processed in the embodiment of the present application refers to an image that needs to be segmented, such as a remote sensing image in a remote sensing scene. Figure 3 A person image 302 in a face recognition scene is shown.

[0074] Step S104: according to at least two segmentation scales, the image to be processed is segmented into sub-images corresponding to the at least two segmentation scales.

[0075] The segmentation scale refers to the size of the segmentation during the segmentation of the image to be processed. In this step, the image to be processed is segmented according to at least two segmentation scales to obtain sub-images corresponding to each segmentation scale. Here, the sub-images obtained after segmentation at each segmentation scale have the same size.

[0076] As attached Figure 3 As shown, the person image 302 is divided into 4 sub-images 304 of equal size, and the division scale used to divide the person image 302 into 4 sub-images 304 of equal size is called the first division scale; the person image 302 is divided into 9 sub-images 306 of equal size, and the division scale used to divide the person image 302 into 9 sub-images 306 of equal size is called the second division scale. Similarly, other division scales can be used to divide the person image 302 into 16, 25 or other numbers of sub-images of equal size.

[0077] In practical applications, after the image to be processed is segmented by at least two segmentation scales, image segmentation is performed on the basis of the sub-images obtained by segmentation. In order to improve the accuracy of image segmentation processing for the image to be processed, an optional implementation provided by the embodiment of the present application can also allow overlapping areas between the segmented sub-images during the segmentation of the image to be processed, as shown in the attached figure. Figure 4As shown, compared with the segmentation method without overlapping areas, the advantage of this segmentation method is that it will not cut off the connection between the features of the boundary areas during the segmentation process, and can maintain the image feature connection of the boundary areas during the segmentation process through the overlapping areas. On this basis, the accuracy of image segmentation is higher.

[0078] Step S106, adjusting the sub-image to meet a preset condition to obtain an intermediate image.

[0079] After the image to be processed is segmented at least two times in different segmentation scales, the sub-image needs to be adjusted. The preset condition is a preset condition that needs to be satisfied for adjusting the sub-image.

[0080] Optionally, the sub-images corresponding to at least two segmentation scales are adjusted to the same image size by scaling the image size of the sub-image corresponding to each segmentation scale, thereby obtaining intermediate images of at least two segmentation scales with equal image sizes, wherein the sub-images after adjustment have the same image size, which is the preset condition. In addition, the preset condition may also be that the difference between the image sizes of the sub-images after adjustment meets a preset threshold, etc., which is not limited here.

[0081] As attached Figure 3 As shown, the four sub-images 304 of equal size after segmentation are enlarged to obtain intermediate images 308 after the four sub-images 304 are enlarged; and the nine sub-images 306 of equal size after segmentation are enlarged to obtain intermediate images 310 after the nine sub-images 306 are enlarged, and the sizes of all the intermediate images are the same or the difference meets the preset threshold, and similarly, the other number of sub-images of equal size are enlarged or reduced to obtain corresponding intermediate images of the same size or the difference meets the preset threshold.

[0082] It should be noted that the image size involved in the preset condition can be set to a fixed value, that is, the sub-image corresponding to each segmentation size can be enlarged or reduced to a fixed value; specifically, this fixed value can be determined according to the actual scene requirements, such as setting the image size of the preset condition to be equal to the size of the image to be processed.

[0083] In an optional implementation manner provided by the embodiment of the present application, the image sizes of the sub-images corresponding to the at least two segmentation scales are scaled / enlarged in the following manner:

[0084] For a sub-image corresponding to at least one segmentation scale among the at least two segmentation scales, determining whether an image size of the sub-image corresponding to the segmentation scale is larger than a reference image size;

[0085] If yes, reducing the image size of the sub-image corresponding to the segmentation scale to be consistent with the size of the reference image, and obtaining an intermediate image of the segmentation scale;

[0086] If not, determining whether the image scale of the sub-image corresponding to the segmentation scale is smaller than the reference image size;

[0087] If it is smaller than, enlarging the image size of the sub-image corresponding to the segmentation size to be consistent with the reference image size, and obtaining an intermediate image of the segmentation scale;

[0088] If it is not less than, it indicates that the image size of the sub-image is consistent with the reference image size of the intermediate image and no processing is required.

[0089] As described above, in the process of segmenting the image to be processed, overlapping areas can also exist between the segmented sub-images. Based on this, in the intermediate image obtained by enlarging or reducing the sub-images, there are also overlapping areas after enlargement or reduction.

[0090] Step S108, performing image segmentation on the target features in the intermediate image to obtain a target image region corresponding to the target features.

[0091] In this embodiment, performing image segmentation on the target features in the intermediate image refers to identifying the target image region where the target features contained in the intermediate graphic are located, and segmenting the target image region where the target features are located. Specifically, in an optional implementation manner provided in the embodiment of the present application, performing image segmentation on the target features in the intermediate image to obtain the target image region corresponding to the target features is specifically implemented in the following manner:

[0092] Calculating the probability value that the smallest image unit contained in the intermediate image is the target feature;

[0093] The minimum image unit whose probability value is greater than the first probability threshold is selected to form a target image area corresponding to the target feature.

[0094] For the multiple intermediate images in the at least two segmentation scales, the probability value of the minimum image unit contained in these intermediate images is the target feature, and the minimum image unit with a probability value greater than a first probability threshold is selected to form a target image area corresponding to the target feature in the intermediate image.

[0095] It should be noted that the minimum image unit may be a single pixel point contained in the intermediate image, or an image block or image area composed of multiple pixels in the intermediate image. In the case where the minimum image unit is a single pixel point in the intermediate image, in the process of image segmentation of the intermediate image, it is necessary to calculate the probability value of each pixel point in the intermediate image being the target feature, so as to determine the target image area composed of all pixels in the intermediate image whose probability values ​​are greater than the first probability threshold, and the accuracy of image segmentation processing of the intermediate image is higher. In the case where the minimum image unit is an image block or image area composed of multiple pixels in the intermediate image, in the process of image segmentation of the intermediate image, it is necessary to calculate the probability value of each image block or image area in the intermediate image being the target feature, so as to determine the target image area composed of all image blocks or image areas in the intermediate image whose probability values ​​are greater than the first probability threshold, which can improve the efficiency of image segmentation processing of the intermediate image.

[0096] In specific implementation, in order to improve the efficiency of image segmentation for target features in the intermediate image, save image segmentation processing time, and also to improve the accuracy of image segmentation for target features, an optional implementation provided in an embodiment of the present application uses an image segmentation model to implement the image segmentation process for target features in the intermediate image. Specifically, the intermediate images of at least two segmentation scales are respectively input into the image segmentation model for image segmentation processing, and the target image areas of the target features in the intermediate images of at least two segmentation scales are output.

[0097] Furthermore, in order to improve the efficiency of image segmentation for target features in the intermediate image, save image segmentation processing time, and also to improve the accuracy of image segmentation for target features, an image segmentation model is used to implement image segmentation for target features in the intermediate image. In an optional implementation provided in an embodiment of the present application, the image segmentation model implements image segmentation of the intermediate image in the following manner:

[0098] Calculating the probability value that the smallest image unit contained in the intermediate image is the target feature;

[0099] The minimum image unit whose probability value is greater than the first probability threshold is selected to form a target image area corresponding to the target feature.

[0100] In this embodiment, the specific implementation of image segmentation for target features in the intermediate image using an image segmentation model is similar to the specific implementation of performing image segmentation on the target features in the intermediate image to obtain the target image area corresponding to the target features. The specific implementation of performing image segmentation on the target features in the intermediate image to obtain the target image area corresponding to the target features may be referred to, and will not be repeated here.

[0101] As attached Figure 3 As shown, the image segmentation performed on the intermediate image 308 and the intermediate image 310 refers to detecting facial features in the intermediate image 308 and the intermediate image 310, and segmenting the facial features detected in the intermediate image 308 and the intermediate image 310 to obtain the target image area corresponding to the facial features; specifically, the probability value of the pixel point in the intermediate image being a facial feature is calculated, and whether the probability value is greater than a first probability threshold is determined to identify whether the pixel point is a facial feature. If it is greater than, it indicates that the pixel point is a pixel point in the facial feature area; if it is less than, it indicates that the pixel point is not a pixel point in the facial feature area. Finally, the image area composed of all the pixels in all the intermediate images whose probability values ​​are greater than the first probability threshold is the target image area corresponding to the facial features contained in the intermediate images.

[0102] As described above, in the process of segmenting the image to be processed, overlapping areas can also exist between the segmented sub-images, and in the intermediate image obtained by enlarging or reducing the sub-image, there are also overlapping areas after the enlargement or reduction process. In this case, the overlapping area exists in multiple intermediate images, and the probability value of the minimum image unit contained in the overlapping area is the target feature is calculated by taking the average value of the probability value of the minimum image unit contained in the overlapping area being the target feature in each intermediate image to which it belongs as the probability value of the minimum image unit contained in the overlapping area being the target feature, thereby improving the accuracy of the probability value calculation of the minimum image unit contained in the overlapping area being the target feature and avoiding the influence of image segmentation on the minimum image unit in the segmentation boundary area.

[0103] In this embodiment, image segmentation processing is performed after scaling the sub-images of at least two segmentation scales to a uniform size. Based on the feature that the intermediate images of at least two segmentation scales obtained after scaling are of equal size, in order to improve the processing efficiency of image segmentation, an optional implementation manner provided in an embodiment of the present application performs the image segmentation processing process of the intermediate images of at least two segmentation scales on a graphics processor in a batch processing manner, thereby saving image segmentation processing time.

[0104] Step S110, fusing the target image region to obtain a target image.

[0105] Based on the above step S108 of performing image segmentation on the target features of the intermediate images of at least two segmentation scales and obtaining the target image region corresponding to the target features, in this step, the target image generated by performing multi-scale image segmentation on the image to be processed is obtained by fusing the target image region.

[0106] In a first optional implementation manner provided in the embodiment of the present application, the target image area is fused in the following manner:

[0107] 1) calculating the optimized probability value of the minimum image unit contained in the image to be processed according to the probability value of the minimum image unit contained in the target image area;

[0108] Specifically, the probability value of the minimum image unit refers to the probability value that the minimum image unit is a target feature; optionally, the optimized probability value refers to the weighted average of the probability values ​​of the minimum image units contained in the target image regions of multiple scales;

[0109] 2) Segmenting the image to be processed into minimum image units whose optimization probability value is greater than a second probability threshold to form a target image region, and obtaining the target image.

[0110] Using the above example, the target image area corresponding to the facial features is fused. Specifically, the weighted average is calculated based on the probability value of the pixel points in the target image area being the facial features to obtain the optimized probability value of the pixel points contained in the person image. On the basis of calculating the optimized probability value, the target image area composed of the pixel points in the person image whose probability value is greater than the second probability threshold is segmented to obtain the image shown in the attached figure. Figure 3 The area where the face of the person is located in the character image shown in a shaded manner in the target image 316.

[0111] Similarly, in addition to the above-mentioned method for fusing the target image area, in a second optional implementation manner provided in the embodiment of the present application, the target image area is fused to obtain the target image, which is specifically implemented in the following manner:

[0112] According to a one-to-one correspondence between the sub-images and the intermediate images, determining a target image region obtained after image segmentation of the intermediate image corresponding to each segmentation scale;

[0113] Merge the target image area obtained after image segmentation of the intermediate image corresponding to each segmentation scale to obtain a merged image area corresponding to each segmentation scale;

[0114] The merged image regions corresponding to the at least two segmentation scales are fused to obtain the target image.

[0115] Specifically, the merged image area refers to the merged image area corresponding to each segmentation scale obtained by performing image merging on the target image area corresponding to each segmentation scale. In practical applications, based on the above-mentioned image segmentation of the target features of the intermediate image of at least two segmentation scales and obtaining the target image area corresponding to the target features, in the target image area corresponding to the obtained target features, according to the one-to-one correspondence between the sub-image and the intermediate image, the target image area obtained after image segmentation of the intermediate image corresponding to each segmentation scale is determined, and then the target image area corresponding to each segmentation scale is merged to obtain the merged image area corresponding to each segmentation scale, and the merged image areas corresponding to various segmentation scales in at least two segmentation scales are fused to obtain the target image.

[0116] Using the above example, in the target image area obtained by segmenting the target features in the intermediate image, as shown in the attached Figure 3 As shown, according to the one-to-one correspondence between the sub-image 304 and the intermediate image 308, the target image area corresponding to the facial features obtained after the image segmentation of the four intermediate images 308, that is, the target image area corresponding to the first segmentation scale, is determined; and according to the one-to-one correspondence between the sub-image 306 and the intermediate image 310, the target image area corresponding to the facial features obtained after the image segmentation of the nine intermediate images 310, that is, the target image area corresponding to the second segmentation scale, the target image area corresponding to the first segmentation scale is merged to obtain a merged image area (specifically, it is shown by adding a shadow to the area where the face is located in the image 312), and the target image area corresponding to the second segmentation scale is merged to obtain a merged image area (specifically, it is shown by adding a shadow to the area where the face is located in the image 314), and the two merged image areas are fused to obtain the area where the face is located in the character image shown in a shaded manner in the target image 316 after fusion.

[0117] In this embodiment, by merging the target image area obtained after image segmentation of the intermediate image corresponding to each segmentation scale, a merged image area corresponding to each segmentation scale is obtained, which makes the segmentation situation of each segmentation scale more intuitive and easy to compare.

[0118] Further, on the basis of the second optional implementation manner provided above, the merged image areas corresponding to at least two segmentation scales are fused, thereby improving the recognition accuracy in the image segmentation process, thereby improving the precision effect of image segmentation. In the first optional implementation manner provided in the embodiment of the present application, the merged image areas corresponding to the at least two segmentation scales are fused to obtain the target image, which is specifically implemented in the following manner:

[0119] Calculating the optimized probability value of the minimum image unit contained in the image to be processed according to the probability value of the minimum image unit contained in the merged image area;

[0120] The minimum image units whose optimization probability values ​​are greater than a second probability threshold are segmented from the image to be processed to form a target image area as the target image.

[0121] Optionally, the optimized probability value refers to a weighted average of probability values ​​of minimum image units contained in the merged image regions of multiple scales.

[0122] Specifically, the specific implementation of calculating the optimized probability value of the minimum image unit contained in the image to be processed according to the probability value of the minimum image unit contained in the merged image area is similar to the specific implementation of calculating the optimized probability value of the minimum image unit contained in the image to be processed according to the probability value of the minimum image unit contained in the target image area. The specific implementation of calculating the optimized probability value of the minimum image unit contained in the image to be processed according to the probability value of the minimum image unit contained in the target image area may be referred to, and will not be repeated here.

[0123] Using the above example, the two merged image regions are fused. Specifically, the weighted average is calculated based on the probability values ​​of the pixels in the two merged image regions being facial features to obtain the optimized probability values ​​of the pixels contained in the person image. On the basis of calculating the optimized probability values, the target image region composed of the pixels in the person image whose optimized probability values ​​are greater than the second probability threshold is segmented to obtain the attached Figure 3 The area where the face of the person is located in the character image shown in a shaded manner in the target image 316.

[0124] In summary, the multi-scale image segmentation method provided by the present application segments the image to be processed according to at least two segmentation scales, into sub-images corresponding to at least two segmentation scales, and then adjusts the sub-images at at least two segmentation scales obtained after segmentation to the same size, performs image segmentation on the target features of the intermediate image obtained after adjustment, obtains the target image area corresponding to the target feature, and finally fuses the target image area, so as to improve the recognition accuracy in the image segmentation process, thereby improving the precision effect of image segmentation.

[0125] The following combination Figure 5 Taking the application of the multi-scale image segmentation method provided in this specification in the organ tissue segmentation scenario as an example, the multi-scale image segmentation method is further described. Figure 5 A processing flow chart of a multi-scale image segmentation method applied to an organ tissue segmentation scenario provided by an embodiment of the present specification is shown, and the specific steps include step S502 to step S516.

[0126] Step S502, acquiring a tomographic image.

[0127] Specifically, the tomographic image refers to a tomographic image of an organ tissue, and the organ tissue may be an organ tissue such as the lung, kidney or liver of a human or animal.

[0128] Step S504: according to at least two segmentation scales, the tomographic image is segmented into sub-images corresponding to the at least two segmentation scales.

[0129] Step S506, for a sub-image corresponding to at least one segmentation scale among the at least two segmentation scales, determining whether the image scale of the sub-image corresponding to the segmentation scale is smaller than the reference image size;

[0130] If yes, it indicates that the image size of the sub-image corresponding to the segmentation scale needs to be enlarged, and the following step S508 is executed;

[0131] If not, it indicates that the image size of the sub-image corresponding to the segmentation scale does not need to be enlarged.

[0132] Step S508: enlarge the image size of the sub-image corresponding to the segmentation size to be consistent with the reference image size, and obtain an intermediate image of the segmentation scale.

[0133] Step S510, calculating the probability value that the smallest image unit contained in the intermediate image is the target feature in the intermediate image.

[0134] Step S512: Select the minimum image unit whose probability value is greater than the first probability threshold to form a target image area corresponding to the target feature.

[0135] Specifically, step S510 and step S512 can be implemented using an image segmentation model;

[0136] Among them, the input of the image segmentation model is the intermediate image, and the output is the target image area corresponding to the target feature.

[0137] Step S514: Calculate the optimized probability value of the minimum image unit included in the tomographic image according to the probability value of the minimum image unit included in the target image area.

[0138] Specifically, the probability value of the minimum image unit refers to the probability value of the minimum image unit being the target feature in the above step; the optimized probability value includes: the weighted average value of the probability values ​​of the minimum image units contained in the target image areas of the at least two segmentation scales.

[0139] Step S516, segmenting the tomographic image into minimum image units whose optimized probability values ​​are greater than the second probability threshold to form a target image region as the organ tissue in the tomographic image.

[0140] In summary, the multi-scale image segmentation method divides the tomographic scan image according to at least two segmentation scales, into sub-images corresponding to at least two segmentation scales, and then adjusts the sub-images at at least two segmentation scales obtained after segmentation to the same size, performs image segmentation on the target features of the intermediate image obtained after adjustment, obtains the target image area corresponding to the target feature, and finally fuses the target image area, so as to improve the recognition accuracy in the image segmentation process, thereby improving the precision of image segmentation.

[0141] An embodiment of a multi-scale image segmentation device provided by the present application is as follows:

[0142] In the above-mentioned embodiment, a multi-scale image segmentation method is provided, and correspondingly, a multi-scale image segmentation device is also provided, which is described below with reference to the accompanying drawings.

[0143] See attached Figure 6 , which shows a schematic diagram of a multi-scale image segmentation device provided by this embodiment.

[0144] Since the device embodiment corresponds to the method embodiment, the description is relatively simple, and the relevant parts can refer to the corresponding description of the method embodiment provided above. The device embodiment described below is only illustrative.

[0145] The present application provides a multi-scale image segmentation device, comprising:

[0146] An image acquisition module 602 is configured to acquire an image to be processed;

[0147] An image segmentation module 604 is configured to segment the image to be processed into sub-images corresponding to at least two segmentation scales according to at least two segmentation scales;

[0148] The sub-image adjustment module 606 is configured to adjust the sub-image to meet a preset condition to obtain an intermediate image;

[0149] An image segmentation module 608 is configured to perform image segmentation on the target feature in the intermediate image to obtain a target image region corresponding to the target feature;

[0150] The region fusion module 610 is configured to fuse the target image region to obtain a target image.

[0151] Optionally, the image segmentation module 608 includes:

[0152] A probability value calculation submodule is configured to calculate a probability value that the smallest image unit contained in the intermediate image is the target feature;

[0153] The region selection submodule is configured to select minimum image units whose probability values ​​are greater than a first probability threshold to form a target image region corresponding to the target feature.

[0154] Optionally, the region fusion module 610 includes:

[0155] The submodule for calculating the optimized probability value is configured to calculate the optimized probability value of the smallest image unit contained in the image to be processed according to the probability value of the smallest image unit contained in the target image area;

[0156] The segmentation region submodule is configured to segment the minimum image units whose optimization probability value is greater than the second probability threshold in the image to be processed to form a target image region as the target image.

[0157] Optionally, the region fusion module 610 includes:

[0158] A region determination submodule is configured to determine a target image region obtained after image segmentation of the intermediate image corresponding to each segmentation scale according to a one-to-one correspondence between the sub-image and the intermediate image;

[0159] An image merging submodule is configured to merge the target image regions obtained after image segmentation of the intermediate image corresponding to each segmentation scale, so as to obtain a merged image region corresponding to each segmentation scale;

[0160] The image region fusion submodule is configured to fuse the merged image regions corresponding to the at least two segmentation scales to obtain the target image.

[0161] Optionally, the image region fusion submodule includes:

[0162] an optimized probability value calculating unit, configured to calculate the optimized probability value of the minimum image unit contained in the image to be processed according to the probability value of the minimum image unit contained in the merged image area;

[0163] The segmentation area unit is configured to segment the minimum image units whose optimization probability values ​​are greater than the second probability threshold in the image to be processed to form a target image area as the target image.

[0164] Optionally, the optimized probability value includes: a weighted average of probability values ​​of minimum image units contained in the target image region of the at least two segmentation scales.

[0165] Optionally, the image segmentation module 608 is implemented using an image segmentation model;

[0166] Among them, the input of the image segmentation model is the intermediate image, and the output is the target image area corresponding to the target feature.

[0167] Optionally, the image segmentation model performs image segmentation on the intermediate image, including:

[0168] Calculating the probability value that the smallest image unit contained in the intermediate image is the target feature;

[0169] The minimum image unit whose probability value is greater than the first probability threshold is selected to form a target image area corresponding to the target feature.

[0170] Optionally, there are overlapping areas between sub-images corresponding to any one of the at least two segmentation scales, and there are overlapping areas between intermediate images of any one of the segmentation scales;

[0171] Correspondingly, the probability value that the smallest image unit included in the overlapping area is the target feature refers to the average value of the probability values ​​that the smallest image unit included in the overlapping area is the target feature in the corresponding intermediate image.

[0172] Optionally, the sub-image adjustment module 606 is specifically configured to perform scaling processing on the image sizes of the sub-images corresponding to the at least two segmentation scales to obtain intermediate images of the at least two segmentation scales with equal image sizes.

[0173] Optionally, the sub-image adjustment module 606 includes:

[0174] A first judgment submodule is configured to judge, for a sub-image corresponding to at least one of the at least two segmentation scales, whether an image size of the sub-image corresponding to the segmentation scale is larger than a reference image size;

[0175] If yes, run the image reduction submodule; the image reduction submodule is configured to reduce the image size of the subimage corresponding to the segmentation scale to be consistent with the reference image size, and obtain an intermediate image of the segmentation scale;

[0176] If not, executing a second judgment submodule; the second judgment submodule is configured to judge whether the image scale of the sub-image corresponding to the segmentation scale is smaller than the reference image size;

[0177] If it is smaller, the image enlargement submodule is run; the image enlargement submodule is configured to enlarge the image size of the sub-image corresponding to the segmentation size to be consistent with the reference image size, so as to obtain the intermediate image of the segmentation scale.

[0178] Optionally, the image segmentation module 608 is executed on a graphics processor in a batch processing manner.

[0179] Optionally, the image to be processed includes a remote sensing image;

[0180] The smallest image unit contained in the remote sensing image includes a pixel unit in the remote sensing image.

[0181] An embodiment of a computing device provided by the present application is as follows:

[0182] Figure 7 7 is a block diagram showing a computing device 700 according to an embodiment of the present application. The components of the computing device 700 include but are not limited to a memory 710 and a processor 720. The processor 720 is connected to the memory 710 via a bus 730, and the database 750 is used to store data.

[0183] The computing device 700 also includes an access device 740 that enables the computing device 700 to communicate via one or more networks 760. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 740 may include one or more of any type of network interface (e.g., a network interface card (NIC)) whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0184] In one embodiment of the present application, the above components of the computing device 700 and Figure 7 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 7 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0185] The computing device 700 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. The computing device 700 may also be a mobile or stationary server.

[0186] The present application provides a computing device, including a memory 710, a processor 720, and computer instructions stored in the memory and executable on the processor, wherein the processor 720 is configured to execute the following computer executable instructions:

[0187] Get the image to be processed;

[0188] According to at least two segmentation scales, the image to be processed is segmented into sub-images corresponding to the at least two segmentation scales;

[0189] Adjusting the sub-image to meet a preset condition to obtain an intermediate image;

[0190] Performing image segmentation on the target features in the intermediate image to obtain a target image region corresponding to the target features;

[0191] The target image region is fused to obtain a target image.

[0192] Optionally, performing image segmentation on the target feature in the intermediate image to obtain a target image region corresponding to the target feature includes:

[0193] Calculating the probability value that the smallest image unit contained in the intermediate image is the target feature;

[0194] The minimum image unit whose probability value is greater than the first probability threshold is selected to form a target image area corresponding to the target feature.

[0195] Optionally, fusing the target image region to obtain the target image includes:

[0196] Calculating the optimized probability value of the minimum image unit contained in the image to be processed according to the probability value of the minimum image unit contained in the target image area;

[0197] The minimum image units whose optimization probability values ​​are greater than a second probability threshold are segmented from the image to be processed to form a target image area as the target image.

[0198] Optionally, fusing the target image region to obtain the target image includes:

[0199] According to a one-to-one correspondence between the sub-images and the intermediate images, determining a target image region obtained after image segmentation of the intermediate image corresponding to each segmentation scale;

[0200] Merge the target image area obtained after image segmentation of the intermediate image corresponding to each segmentation scale to obtain a merged image area corresponding to each segmentation scale;

[0201] The merged image regions corresponding to the at least two segmentation scales are fused to obtain the target image.

[0202] Optionally, fusing the merged image regions corresponding to the at least two segmentation scales to obtain the target image includes:

[0203] Calculating the optimized probability value of the minimum image unit contained in the image to be processed according to the probability value of the minimum image unit contained in the merged image area;

[0204] The minimum image units whose optimization probability values ​​are greater than a second probability threshold are segmented from the image to be processed to form a target image area as the target image.

[0205] Optionally, the optimized probability value includes: a weighted average of probability values ​​of minimum image units contained in the target image region of the at least two segmentation scales.

[0206] Optionally, the step of performing image segmentation on the target features in the intermediate image to obtain the target image region corresponding to the target features is implemented using an image segmentation model;

[0207] Among them, the input of the image segmentation model is the intermediate image, and the output is the target image area corresponding to the target feature.

[0208] Optionally, the image segmentation model performs image segmentation on the intermediate image, including:

[0209] Calculating the probability value that the smallest image unit contained in the intermediate image is the target feature;

[0210] The minimum image unit whose probability value is greater than the first probability threshold is selected to form a target image area corresponding to the target feature.

[0211] Optionally, there are overlapping areas between sub-images corresponding to any one of the at least two segmentation scales, and there are overlapping areas between intermediate images of any one of the segmentation scales;

[0212] Correspondingly, the probability value that the smallest image unit included in the overlapping area is the target feature refers to the average value of the probability values ​​that the smallest image unit included in the overlapping area is the target feature in the corresponding intermediate image.

[0213] Optionally, adjusting the sub-image to meet a preset condition to obtain an intermediate image includes:

[0214] The image sizes of the sub-images corresponding to the at least two segmentation scales are scaled up / down to obtain intermediate images of the at least two segmentation scales with equal image sizes.

[0215] Optionally, the scaling the image sizes of the sub-images corresponding to the at least two segmentation scales to obtain intermediate images of at least two segmentation scales with equal image sizes includes:

[0216] For a sub-image corresponding to at least one segmentation scale among the at least two segmentation scales, determining whether an image size of the sub-image corresponding to the segmentation scale is larger than a reference image size;

[0217] If yes, reducing the image size of the sub-image corresponding to the segmentation scale to be consistent with the size of the reference image, and obtaining an intermediate image of the segmentation scale;

[0218] If not, determining whether the image scale of the sub-image corresponding to the segmentation scale is smaller than the reference image size;

[0219] If it is smaller than, the image size of the sub-image corresponding to the segmentation size is enlarged to be consistent with the reference image size, so as to obtain an intermediate image of the segmentation scale.

[0220] Optionally, the step of performing image segmentation on the target features in the intermediate image to obtain the target image region corresponding to the target features is executed on a graphics processor in a batch processing manner.

[0221] Optionally, the image to be processed includes a remote sensing image;

[0222] The smallest image unit contained in the remote sensing image includes a pixel unit in the remote sensing image.

[0223] An embodiment of a computer-readable storage medium provided by the present application is as follows:

[0224] An embodiment of the present application provides a computer-readable storage medium storing computer instructions, which implement the steps of the multi-scale image segmentation method when executed by a processor.

[0225] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the multi-scale image segmentation method described above are of the same concept, and the details not described in detail in the technical scheme of the storage medium can be found in the description of the technical scheme of the multi-scale image segmentation method described above.

[0226] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0227] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0228] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0229] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0230] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A multi-scale image segmentation method, comprising: Get the image to be processed; According to at least two segmentation scales, the image to be processed is segmented into sub-images corresponding to the at least two segmentation scales; Adjusting the sub-image to meet a preset condition to obtain an intermediate image; Performing image segmentation on the target features in the intermediate image to obtain a target image region corresponding to the target features; The target image region is fused to obtain a target image.

2. According to the multi-scale image segmentation method of claim 1, the step of performing image segmentation on the target features in the intermediate image to obtain the target image region corresponding to the target features comprises: Calculating the probability value that the smallest image unit contained in the intermediate image is the target feature; The minimum image unit whose probability value is greater than the first probability threshold is selected to form a target image area corresponding to the target feature.

3. According to the multi-scale image segmentation method of claim 2, the step of fusing the target image region to obtain the target image comprises: Calculating an optimized probability value of the minimum image unit contained in the image to be processed according to the probability value of the minimum image unit contained in the target image area, wherein the probability value of the minimum image unit refers to the probability value that the minimum image unit is a target feature, and the optimized probability value refers to a weighted average of the probability values ​​of the minimum image unit contained in the target image area at multiple scales; The minimum image units whose optimization probability values ​​are greater than a second probability threshold are segmented from the image to be processed to form a target image area as the target image.

4. According to the multi-scale image segmentation method of claim 2, the step of fusing the target image region to obtain the target image comprises: According to a one-to-one correspondence between the sub-images and the intermediate images, determining a target image region obtained after image segmentation of the intermediate image corresponding to each segmentation scale; Merge the target image area obtained after image segmentation of the intermediate image corresponding to each segmentation scale to obtain a merged image area corresponding to each segmentation scale; The merged image regions corresponding to the at least two segmentation scales are fused to obtain the target image.

5. According to the multi-scale image segmentation method of claim 4, the step of fusing the merged image regions corresponding to the at least two segmentation scales to obtain the target image comprises: Calculating an optimized probability value of the minimum image unit contained in the image to be processed according to the probability value of the minimum image unit contained in the merged image area, wherein the probability value of the minimum image unit refers to the probability value that the minimum image unit is a target feature, and the optimized probability value refers to a weighted average of the probability values ​​of the minimum image unit contained in the merged image area at multiple scales; The minimum image units whose optimization probability values ​​are greater than a second probability threshold are segmented from the image to be processed to form a target image area as the target image.

6. The multi-scale image segmentation method according to claim 3, wherein the optimization probability value comprises: The weighted average of the probability values ​​of the minimum image units contained in the target image area of ​​the at least two segmentation scales.

7. According to the multi-scale image segmentation method of claim 1, the step of performing image segmentation on the target features in the intermediate image to obtain the target image region corresponding to the target features is implemented by using an image segmentation model; in, The input of the image segmentation model is the intermediate image, and the output is the target image region corresponding to the target feature.

8. The multi-scale image segmentation method according to claim 7, wherein the image segmentation model performs image segmentation on the intermediate image, comprising: Calculating the probability value that the smallest image unit contained in the intermediate image is the target feature; The minimum image unit whose probability value is greater than the first probability threshold is selected to form a target image area corresponding to the target feature.

9. The multi-scale image segmentation method according to claim 2, wherein there are overlapping areas between sub-images corresponding to any one of the at least two segmentation scales, and there are overlapping areas between intermediate images of any one of the segmentation scales; Correspondingly, the probability value that the smallest image unit included in the overlapping area is the target feature refers to the average value of the probability values ​​that the smallest image unit included in the overlapping area is the target feature in the corresponding intermediate image.

10. The multi-scale image segmentation method according to claim 1, wherein the step of adjusting the sub-image to meet a preset condition to obtain an intermediate image comprises: The image sizes of the sub-images corresponding to the at least two segmentation scales are scaled up / down to obtain intermediate images of the at least two segmentation scales with equal image sizes.

11. The multi-scale image segmentation method according to claim 10, wherein the image sizes of the sub-images corresponding to the at least two segmentation scales are scaled up / down to obtain intermediate images of at least two segmentation scales with equal image sizes, comprising: For a sub-image corresponding to at least one segmentation scale among the at least two segmentation scales, determining whether an image size of the sub-image corresponding to the segmentation scale is larger than a reference image size; If yes, reducing the image size of the sub-image corresponding to the segmentation scale to be consistent with the size of the reference image, and obtaining an intermediate image of the segmentation scale; If not, determining whether the image scale of the sub-image corresponding to the segmentation scale is smaller than the reference image size; If it is smaller than, the image size of the sub-image corresponding to the segmentation size is enlarged to be consistent with the reference image size, so as to obtain an intermediate image of the segmentation scale.

12. According to the multi-scale image segmentation method of claim 1, the step of performing image segmentation on the target features in the intermediate image to obtain the target image area corresponding to the target features is executed on a graphics processor in a batch processing manner.

13. The multi-scale image segmentation method according to claim 2, wherein the image to be processed comprises a remote sensing image; The smallest image unit contained in the remote sensing image includes a pixel unit in the remote sensing image.

14. A multi-scale image segmentation device, comprising: An image acquisition module is configured to acquire an image to be processed; An image segmentation module is configured to segment the image to be processed into sub-images corresponding to at least two segmentation scales according to at least two segmentation scales; A sub-image adjustment module, configured to adjust the sub-image to meet a preset condition to obtain an intermediate image; An image segmentation module is configured to perform image segmentation on the target features in the intermediate image to obtain a target image region corresponding to the target features; The region fusion module is configured to fuse the target image region to obtain a target image.

15. A computing device comprising: Memory and processor; The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions: Get the image to be processed; According to at least two segmentation scales, the image to be processed is segmented into sub-images corresponding to the at least two segmentation scales; Adjusting the sub-image to meet a preset condition to obtain an intermediate image; Performing image segmentation on the target features in the intermediate image to obtain a target image region corresponding to the target features; The target image region is fused to obtain a target image.

16. A computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the multi-scale image segmentation method according to any one of claims 1 to 13.

17. A computer program product, characterized in that The method comprises computer instructions which, when executed by a processor, implement the steps of the multi-scale image segmentation method according to any one of claims 1 to 13.

Images