Multiscale-based detail-enhanced image segmentation method, system, device, and medium

Abstract

The application discloses a multi-scale-based detail enhancement image segmentation method, system, device and medium, and the method comprises the following steps: acquiring an image to be segmented; inputting the image to be segmented into a segmentation network composed of multiple scale convolution channels to obtain multiple output subsets corresponding to the scale convolution channels; splicing all the output subsets to obtain an output set; inputting the output set into multiple preset detail enhancement units respectively, and outputting multiple feature information corresponding to the detail enhancement units; the multiple detail enhancement units have multiple scale convolutions; and fusing the feature information to obtain a segmentation prediction map. The application can enhance the skeleton of the convolution kernel, increase the effective receptive field of the network backbone, and reduce the influence of redundant information on subsequent feature fusion operations.

Description

Technical Field

[0001] The present invention relates to the field of time-frequency transformation, and in particular to a method, system, device and medium for image segmentation based on multi-scale detail enhancement. Background Art

[0002] In the field of medical image segmentation, how to obtain accurate detail information to complete high-quality segmentation is a challenge. In the real world, objects and structures often have multi-scale features, which means that they exhibit different properties or characteristics at different scales or levels of detail. Therefore, there are usually two problems: the feature information of each channel is highly redundant and similar. As the channel dimension of the feature map continues to expand in the encoder stage, the redundant information also increases. The frequent cross-level feature fusion operations in Unet-shape will also amplify the impact of redundancy, dilute important detail information, and affect the determination of the boundaries of the target object. Secondly, the single-scale convolution kernel limits the network's ability to obtain target background information. In addition, existing technologies usually use standard convolution within a single scale, which may limit the feature extraction capabilities of multi-scale structures. Summary of the Invention

[0003] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes a multi-scale detail-enhanced image segmentation method, system, device, and medium. These methods enhance the skeleton of the convolution kernel, increase the effective receptive field of the network backbone, and reduce the impact of redundant information on subsequent feature fusion operations.

[0004] In a first aspect, an embodiment of the present invention provides a multi-scale detail enhancement image segmentation method, the multi-scale detail enhancement image segmentation method comprising:

[0005] Obtain the image to be segmented;

[0006] Inputting the image to be segmented into a segmentation network composed of multiple scale convolution channels to obtain multiple output subsets corresponding to the scale convolution channels;

[0007] splicing all the output subsets to obtain an output set;

[0008] Inputting the output set into a plurality of preset detail enhancement units respectively, and outputting a plurality of feature information corresponding to the detail enhancement units one by one; the plurality of detail enhancement units have a plurality of scale convolutions;

[0009] The feature information is fused to obtain a segmentation prediction map.

[0010] The method according to the embodiment of the present invention has at least the following beneficial effects:

[0011] First, the image to be segmented is input into a segmentation network composed of multiple scale convolution channels, and feature information is extracted under channels of different scales, which can collect subtle size changes and detail information, and bring better feature extraction capabilities to the network; secondly, the output subsets are spliced ​​to obtain an output set, and the feature information of different scales is fused to provide a good data basis for subsequent detail enhancement; then the output sets are input into multiple detail enhancement units respectively to obtain the feature information of the detail enhancement units one-to-one. Through multiple scale convolutions, different degrees of attention can be paid to the features, which enhances the sensitivity to the feature information and ensures the accuracy of the recognition of the feature information; finally, the feature information is fused to obtain a segmentation prediction map. Combining the feature information of detail enhancement at different scales, prediction maps of different depths can be generated. By fusing the prediction maps of different depths, a segmentation prediction map can be obtained, which achieves high-quality segmentation without significantly increasing the computational cost, thereby achieving excellent performance improvement.

[0012] According to some embodiments of the present invention, the image to be segmented is input into a segmentation network composed of multiple scale convolution channels, and multiple output subsets corresponding one-to-one to the scale convolution channels are obtained, including:

[0013] Dividing the image to be segmented into multiple input subsets;

[0014] The input subset is input into the corresponding scale convolution channel to obtain an output subset corresponding one-to-one to the scale convolution channel; the scale convolution channel includes a depth-wise separable convolution kernel composed of depth-wise convolution and point-wise convolution, and the depth-wise separable convolution kernel is composed of a large convolution kernel larger than a preset size threshold.

[0015] According to some embodiments of the present invention, the plurality of detail enhancement units are obtained by the following method:

[0016] Screening the preset convolution kernels greater than the stacking threshold in the plurality of detail enhancement units to obtain the convolution kernels to be stacked;

[0017] Superimposing the convolution kernel to be superimposed by a standard convolution kernel and an asymmetric convolution kernel to obtain a superimposed convolution kernel;

[0018] The detail enhancement units are arranged from large to small according to the superimposed convolution kernels to obtain the multiple detail enhancement units.

[0019] According to some embodiments of the present invention, superimposing the convolution kernels to be superimposed by a standard convolution kernel and an asymmetric convolution kernel to obtain a superimposed convolution kernel includes:

[0020] Setting both the standard convolution kernel and the asymmetric convolution kernel as the large convolution kernel;

[0021] The large convolution kernel is superimposed through additivity of the two-dimensional convolution kernel to obtain a superimposed convolution kernel; wherein, a calculation formula of superimposing the large convolution kernel through the additivity of the two-dimensional convolution kernel comprises:

[0022]

[0023] wherein, wherein I represents an input matrix, K represents a two-dimensional convolution kernel of a compatible size, represents a corresponding position element addition operation of kernel parameters.

[0024] According to some embodiments of the present application, the feature information is fused to obtain a segmentation prediction map, comprising:

[0025] The feature information is fused through a Decoder architecture to obtain fused feature information.

[0026] The fused feature information is obtained through an upsampling method to obtain the segmentation prediction map.

[0027] Each of the scale convolution channels is independently subjected to a convolution operation.

[0028] According to some embodiments of the present application, the large convolution kernel is compatible from a small convolution kernel smaller than the preset size threshold.

[0029] In a second aspect, embodiments of the present application provide a multi-scale based detail enhanced image segmentation system, comprising:

[0030] An image acquisition module is configured to acquire an image to be segmented.

[0031] A channel multi-scale module is configured to input the image to be segmented into a segmentation network composed of a plurality of scale convolution channels to obtain a plurality of output subsets corresponding to the scale convolution channels.

[0032] An output subset splicing module is configured to splice all the output subsets to obtain an output set.

[0033] A multi-scale detail enhancement module is configured to input the output set into a plurality of preset detail enhancement units respectively, and output a plurality of feature information corresponding to the detail enhancement units; the plurality of detail enhancement units have a plurality of scale convolutions.

[0034] A feature fusion module is configured to fuse the feature information to obtain a segmentation prediction map.

[0035] In a third aspect, an embodiment of the present invention provides an electronic device comprising at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor to enable the at least one control processor to perform the multi-scale detail enhancement image segmentation method as described in the first aspect.

[0036] In a fourth aspect, an embodiment of the present invention provides a computer storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the multi-scale detail enhancement image segmentation method as described in the first aspect.

[0037] It should be noted that the beneficial effects of the second and third aspects of the present invention over the prior art are the same as the beneficial effects of the plug-in quick access method of the first aspect, and will not be described in detail here.

[0038] Other features and advantages of the present invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments with reference to the accompanying drawings, in which:

[0040] Figure 1 This is a flowchart of a multi-scale detail enhancement image segmentation method provided by one embodiment of the present invention;

[0041] Figure 2 This is a flowchart of inputting an image to be segmented into a segmentation network composed of multiple scale convolution channels to obtain multiple output subsets corresponding one-to-one to the scale convolution channels, provided by an embodiment of the present invention;

[0042] Figure 3 is a flowchart of a method for obtaining multiple detail enhancement units provided by an embodiment of the present invention;

[0043] Figure 4 This is a flow chart of obtaining a superimposed convolution kernel by superimposing a convolution kernel to be superimposed by a standard convolution kernel and an asymmetric convolution kernel, provided by one embodiment of the present invention;

[0044] Figure 5 This is a flow chart of fusing feature information to obtain a segmentation prediction map provided by an embodiment of the present invention;

[0045] Figure 6 This is a comparison diagram of the independent scale convolution channel and the Res2net module channel provided by one embodiment of the present invention;

[0046] Figure 7 is a schematic diagram of convolution kernel superposition provided by one embodiment of the present invention;

[0047] Figure 8 is a schematic diagram of feature information enhancement using multiple detail enhancement units of different scales according to an embodiment of the present invention;

[0048] Figure 9 This is a data flow diagram of a multi-scale detail enhancement image segmentation method provided by one embodiment of the present invention;

[0049] Figure 10 1 is a schematic structural diagram of a multi-scale detail enhancement image segmentation system provided by an embodiment of the present invention;

[0050] Figure 11 It is a structural diagram of an electronic device provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0051] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0052] In the description of the present invention, if there is a description of first, second, etc., it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0053] In the description of the present invention, it should be understood that descriptions involving orientation, such as the orientation or positional relationship indicated by up, down, etc., are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0054] In the description of the present invention, it should be noted that, unless otherwise clearly defined, terms such as setting, installing, and connecting should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0055] Reference Figure 1 In some embodiments of the present invention, a multi-scale detail enhancement image segmentation method is provided, and the multi-scale detail enhancement image segmentation method includes:

[0056] Step S100: Obtain an image to be segmented.

[0057] Step S200: Input the image to be segmented into a segmentation network composed of multiple scale convolution channels to obtain multiple output subsets corresponding to the scale convolution channels.

[0058] Step S300: concatenate all output subsets to obtain an output set.

[0059] Step S400: Input the output set into a plurality of preset detail enhancement units respectively, and output a plurality of feature information corresponding to the detail enhancement units one by one; the plurality of detail enhancement units have a plurality of scale convolutions.

[0060] Step S500: Fuse feature information to obtain a segmentation prediction map.

[0061] First, the image to be segmented is input into a segmentation network composed of multiple scale convolution channels, and feature information is extracted under channels of different scales, which can collect subtle size changes and detail information, and bring better feature extraction capabilities to the network; secondly, the output subsets are spliced ​​to obtain an output set, and the feature information of different scales is fused to provide a good data basis for subsequent detail enhancement; then the output sets are input into multiple detail enhancement units respectively to obtain the feature information of the detail enhancement units one-to-one. Through multiple scale convolutions, different degrees of attention can be paid to the features, which enhances the sensitivity to the feature information and ensures the accuracy of the recognition of the feature information; finally, the feature information is fused to obtain a segmentation prediction map. Combining the feature information of detail enhancement at different scales, prediction maps of different depths can be generated. By fusing the prediction maps of different depths, a segmentation prediction map can be obtained, which achieves high-quality segmentation without significantly increasing the computational cost, thereby achieving excellent performance improvement.

[0062] Reference Figure 2 In some embodiments of the present invention, an image to be segmented is input into a segmentation network composed of multiple scale convolution channels, and multiple output subsets corresponding to the scale convolution channels are obtained, including:

[0063] Step S201: Divide the image to be segmented into multiple input subsets.

[0064] Step S202: Input the input subset into the corresponding scaled convolution channel to obtain an output subset corresponding to the scaled convolution channel one by one; the scaled convolution channel includes a depth-wise separable convolution kernel composed of depth-wise convolution and point-wise convolution, and the depth-wise separable convolution kernel is composed of a large convolution kernel larger than a preset size threshold.

[0065] It should be noted that, referring to Figure 6In some neural networks, especially segmentation networks, feature fusion is frequently performed, which can amplify the effects of redundant information generated during feature extraction. Excessive redundant information can dilute important boundary information, resulting in blurred boundaries in the segmentation results. Therefore, the Res2Net module is improved to separate each scale branch and perform independent convolution operations to optimize the use of redundant information. Furthermore, the original stacked 3×3 convolutions are replaced with corresponding larger convolution kernels to increase the effective receptive field.

[0066] By using convolutional channels of different scales, we can reduce the generation of redundant information and make full use of the existing redundant information for multi-scale feature extraction. At the same time, we replace the serialized 3×3 convolutions with larger convolution kernels that are more conducive to segmentation, which increases the effective receptive field of the backbone. The introduction of depthwise separable convolution reduces the increased computational cost associated with large convolution kernels while ensuring the accuracy of the segmentation results.

[0067] Reference Figure 3 In some embodiments of the present invention, the plurality of detail enhancement units are obtained by the following method:

[0068] Step S410: Filter preset convolution kernels greater than a superposition threshold in a plurality of detail enhancement units to obtain a convolution kernel to be superimposed.

[0069] It should be noted that the superposition of asymmetric convolution is used to replace the standard convolution to improve the inference speed without reducing the representation ability. However, due to the limitation of the convolution kernel size, a superposition threshold is set to control the superposition of asymmetric convolution.

[0070] Step S420: Superimpose the convolution kernel to be superimposed by the standard convolution kernel and the asymmetric convolution kernel to obtain a superimposed convolution kernel.

[0071] It should be noted that, referring to Figure 7 ,The convolution kernel to be superimposed needs to be superimposed by the standard convolution kernel and the asymmetric convolution kernel due to the ,step size and compatibility.

[0072] Step S430: Arrange the detail enhancement units from large to small according to the superimposed convolution kernels to obtain multiple detail enhancement units.

[0073] It should be noted that, referring to Figure 8 In order to comply with the top-down peripheral-central feedback mechanism, the convolution kernels are arranged from small to large according to the stacking, so as to facilitate the perception of different degrees of detail enhancement. As an example, the output of the three detail enhancement units is expressed as follows:

[0074] DK=Concat(E B (F in ),E H(F iN ),E V (F in ))

[0075] Where DE represents the output of all detail enhancement units, E B 、E H and E V They all represent the output of a detail enhancement unit. It is not difficult to see from the output that the output of DE is composed of three parts, namely the darkest central part, the second darkest peripheral backbone part and the lightest edge part. The depth of the color represents the degree of information enhancement obtained. The darker the color, the stronger the information enhancement obtained, and vice versa.

[0076] It is understood that central vision is used to directly observe objects, while peripheral vision displays other areas of the visual field, namely the peripheral areas that can be seen. Brain imaging studies have found that input from the peripheral visual field can be decoded in the primary visual cortex that represents the central visual field, suggesting the existence of a top-down peripheral-central feedback mechanism in visual discrimination tasks such as shape, color, and object category judgment. Therefore, using asymmetric convolutions in different directions and parallel superposition of standard convolutions can generate different levels of attention within and outside the receptive field, which is very similar to human central and peripheral vision.

[0077] By superimposing the standard convolution kernel and the asymmetric convolution kernel, the peripheral backbone and central parts of the convolution kernel skeleton are enhanced to varying degrees. Not only is the skeleton of the convolution kernel strengthened, but the central enhanced part of each scale also contains the entire receptive field of the previous scale, making the central receptive field gradually expand and contrast outward, thereby enhancing the convolution kernel backbone and forming a distribution with strong center and weak periphery, which helps the network to be more sensitive to detailed information.

[0078] Reference Figure 4 In some embodiments of the present invention, the convolution kernel to be superimposed is superimposed by a standard convolution kernel and an asymmetric convolution kernel to obtain a superimposed convolution kernel, including:

[0079] Step S421: Set both the standard convolution kernel and the asymmetric convolution kernel as large convolution kernels.

[0080] Step S422: Superimpose the large convolution kernels using the additivity of the two-dimensional convolution kernels to obtain a superimposed convolution kernel. The calculation formula for superimposing using the additivity of the two-dimensional convolution kernels includes:

[0081]

[0082] Where I represents the input matrix, K represents the two-dimensional convolution kernel of compatible size, Represents the addition operation of the corresponding position elements of the kernel parameters.

[0083] It should be noted that the skeleton of the convolution kernel is enhanced by superposing large convolution kernels of [2(i-1)-1]×(2i-1), (2i-1)×[2(i-1)-1] and (2i-1)×(2i-1) instead of the basic 3×1, 1×3 and 3×3 convolutions.

[0084] By using large convolution kernels and superimposing them through the additivity of two-dimensional convolution kernels, the skeleton of the convolution kernel is enhanced, and the peripheral backbone and central parts of the convolution kernel skeleton are enhanced to varying degrees. Moreover, by superimposing them through the additivity of two-dimensional convolution kernels, the compatibility of the convolution kernel can be guaranteed.

[0085] Reference Figure 5 In some embodiments of the present invention, the feature information is fused to obtain a segmentation prediction map, including:

[0086] Step S501: fuse the feature information through the Decoder architecture to obtain fused feature information.

[0087] Step S502: The fused feature information is up-sampled to obtain a segmentation prediction map.

[0088] It should be noted that within the Decoder structure, feature information between adjacent levels will be fused.

[0089] The decoder structure can efficiently handle variable-length sequence problems and reduce the computational complexity of fusion. Therefore, it can provide efficient processing performance for multi-scale feature information fusion.

[0090] In some embodiments of the present invention, convolution operations are performed independently between each scale convolution channel.

[0091] It should be noted that for the Res2net module, further improvements are made to separate the convolution channels of each scale and complete the convolution operation independently, which can optimize the utilization of redundant information.

[0092] By performing convolution operations independently between scale convolution channels, not only the generation of redundant information is reduced, but also feature extraction is completed during the encoding process through independent and parallel multi-scale branches, which increases the effective receptive field of the network backbone and reduces the impact of redundant information on subsequent feature fusion operations.

[0093] In some embodiments of the present invention, a large convolution kernel is obtained by compatibility with a small convolution kernel smaller than a preset size threshold.

[0094] It should be noted that the small convolution kernel can be "patched" or "padding" into a compatible large convolution kernel. Formally, B represents the large convolution kernel and S represents the small convolution kernel, so the large convolution kernel can be compatible with it using the following formula:

[0095] H B ≥H S ,W B ≥W S ,M B =Patch(M S )

[0096] Among them, H and W represent the height and width of the convolution kernel respectively, and M represents the shape of the convolution kernel.

[0097] By compatibility of small convolution kernels to obtain large convolution kernels, the backbone of the convolution kernel is enhanced, forming a distribution with strong center and weak periphery, which enhances the sensitivity to texture information. Combined with the progressive multi-scale output, the central receptive field of each scale contains all the receptive fields of the previous scale, which can be regarded as an exploration process of comparing detailed information of different scales, which can more accurately mine feature information and ensure the accuracy of feature information.

[0098] Reference Figure 9 To facilitate understanding by those skilled in the art, a specific embodiment of the present invention provides a multi-scale detail enhancement image segmentation method, comprising the following steps:

[0099] The first step is to extract feature information through multi-scale convolution channels.

[0100] The Res2net module is improved in a targeted way, so that each scale branch is separated and the convolution operation is completed independently, in order to optimize the use of redundant information. At the same time, the original superimposed 3×3 convolution is replaced with the corresponding large convolution kernel to improve the effective receptive field. As with the Res2net module, the input image to be segmented is divided into s input subsets, denoted as x i , where i∈{1,2…s}. The difference is that in the multi-scale convolution channel, no information fusion is performed between the input subsets. Each input subset is subjected to convolution operations at different scales to obtain the corresponding output subset. This process is expressed as:

[0101] y i =Conv (2i-1)×(2i-1) .

[0102] All output subsets are concatenated to obtain an overall output set, which is then fed into a multi-scale detail enhancement unit.

[0103] The second step is to enhance feature information through multi-scale detail enhancement units.

[0104] Screening multiple detail enhancement units with preset convolution kernels greater than the superposition threshold to obtain a convolution kernel to be superimposed; superimposing the convolution kernel to be superimposed by a standard convolution kernel and an asymmetric convolution kernel to obtain a superimposed convolution kernel; arranging the detail enhancement units from large to small according to the superimposed convolution kernel to obtain multiple detail enhancement units; inputting the output set into multiple detail enhancement units respectively, and outputting multiple feature information corresponding to the detail enhancement units; wherein, the superposition of basic 3×1, 1×3 and 3×3 convolutions is replaced by the superposition of large convolution kernels of [2(i-1)-1]×(2i-1), (2i-1)×[2(i-1)-1] and (2i-1)×(2i-1), and the large convolution kernels are superimposed by the additivity of two-dimensional convolution kernels to obtain the superimposed convolution kernel; the calculation formula for superposition by the additivity of two-dimensional convolution kernels includes:

[0105]

[0106] Where I represents the input matrix, K represents the two-dimensional convolution kernel of compatible size, Represents the addition operation of the corresponding position elements of the kernel parameters. Preferably, the small convolution kernel can be "patched" or "padding" into a compatible large convolution kernel. Formally, B represents the large convolution kernel and S represents the small convolution kernel, so the large convolution kernel can be compatible with it by the following formula:

[0107] H B ≥H S ,W B ≥W S ,M B =Patch(M S )

[0108] Among them, H and W represent the height and width of the convolution kernel respectively, and M represents the shape of the convolution kernel.

[0109] Finally, As an example, the output of the three detail enhancement units is expressed as follows:

[0110] DE=Concat(E B (F in ),E H (F in ),E V (F in ))

[0111] Where DE represents the output of all detail enhancement units, E B 、E H and E VEach represents the output of a detail enhancement unit. It is not difficult to see from the output that the output of DE is composed of three parts: the darkest central part, the second darkest peripheral core part, and the lightest edge part. The depth of the color represents the degree of information enhancement obtained. The darker the color, the stronger the information enhancement obtained, and vice versa. The output of the darkest central part, the second darkest peripheral core part, and the lightest edge part can be expressed by the following formula:

[0112] CP=Concat(E B (F in ),E H (F in ),E V (F in ))

[0113] PSP H =Concat(E B (E in ) PSp ,E H (F in ) PSP )

[0114] PSP V =Concat(E B (E in ) PSp ,E V (F in ) PSP )

[0115] EP=E B (F in ) EP

[0116] Among them, CP represents the darkest central part, PSP H and PSP V It indicates the peripheral backbone part with the second lightest color, and EP indicates the edge part with the lightest color;

[0117] It is not difficult to see from the formula that the closer to the edge, the less information enhancement is obtained. However, the PSP and CP on the convolution kernel skeleton both obtain different degrees of enhancement.

[0118] In the third step, the feature information is passed to the decoder structure. Within the decoder, feature information from adjacent layers is fused. Upsampling is also performed to generate a prediction map for that depth. Information fusion is performed from bottom to top, generating prediction maps at different depths. Finally, supervision is performed to obtain the segmentation prediction result map.

[0119] Reference Figure 10One embodiment of the present invention further provides a multi-scale detail enhancement image segmentation system, comprising an image acquisition module 1001, a channel multi-scale module 1002, an output subset stitching module 1003, a multi-scale detail enhancement module 1004, and a feature fusion module 1005, wherein:

[0120] The image acquisition module 1001 is used to acquire the image to be segmented.

[0121] The channel multi-scale module 1002 is used to input the image to be segmented into a segmentation network composed of multiple scale convolution channels to obtain multiple output subsets corresponding to the scale convolution channels.

[0122] The output subset splicing module 1003 is used to splice all output subsets to obtain an output set.

[0123] The multi-scale detail enhancement module 1004 is used to input the output set into a plurality of preset detail enhancement units respectively, and output a plurality of feature information corresponding to the detail enhancement units one by one; the plurality of detail enhancement units have a plurality of scale convolutions.

[0124] The feature fusion module 1005 is used to fuse feature information to obtain a segmentation prediction map.

[0125] It should be noted that since the multi-scale detail enhancement image segmentation system in this embodiment and the above-mentioned multi-scale detail enhancement image segmentation method are based on the same inventive concept, the corresponding contents in the method embodiment are also applicable to the present device embodiment and will not be described in detail here.

[0126] refer to Figure 11 Another embodiment of the present invention further provides an electronic device, which can be any type of smart terminal, such as a mobile phone, a tablet computer, a personal computer, etc.

[0127] Specifically, the electronic device 6000 includes: one or more control processors 6001 and a memory 6002, Figure 11 In the example, a control processor 6001 and a memory 6002 are used. The control processor 6001 and the memory 6002 can be connected via a bus or other means. Figure 11 The bus connection is taken as an example.

[0128] The memory 6002 is a non-transitory computer-readable storage medium that can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as program instructions / modules corresponding to an electronic device in an embodiment of the present invention;

[0129] The control processor 6001 executes various functional applications and data processing of a multi-scale detail enhancement image segmentation method by running the non-transitory software programs, instructions and modules stored in the memory 6002, that is, implements a multi-scale detail enhancement image segmentation method based on the above-mentioned method embodiment.

[0130] The memory 6002 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created by using a multi-scale detail enhancement image segmentation method, etc. In addition, the memory 6002 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 6002 may optionally include a memory remotely located relative to the control processor 6001, and these remote memories may be connected to the electronic device 6000 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0131] In one or more modules stored in the memory 6002, when executed by the one or more control processors 6001, a multi-scale detail enhancement image segmentation method in the above method embodiment is executed, for example, the above described Figures 1 to 5 method steps.

[0132] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0133] It should be noted that, since the electronic device in this embodiment and the above-mentioned multi-scale detail enhancement image segmentation method are based on the same inventive concept, the corresponding contents in the method embodiment are also applicable to the device embodiment and will not be described in detail here.

[0134] One embodiment of the present invention further provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute: the multi-scale detail enhancement image segmentation method as described in the above embodiment.

[0135] It should be noted that since the computer-readable storage medium in this embodiment and the above-mentioned multi-scale detail enhancement image segmentation method are based on the same inventive concept, the corresponding content in the method embodiment is also applicable to the device embodiment and will not be described in detail here.

[0136] Those skilled in the art will appreciate that all or some of the steps and systems in the method disclosed above can be implemented as software, firmware, hardware, and appropriate combinations thereof. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, and the computer-readable medium can include computer storage media (or non-transitory media) and communication media (or temporary media). As known to those skilled in the art, the term computer storage media is included in any method or technology for storing data (such as computer-readable instructions, data structures, program modules, or other data) and is volatile and non-volatile, removable, and non-removable. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory, or other memory technology, CD-ROM, digital versatile disks (DVD), or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage, or other magnetic storage devices, or any other medium that can be used to store desired data and can be accessed by a computer. Furthermore, as is well known to those skilled in the art, communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any data delivery media.

[0137] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative uses of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0138] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

Claims

1. A multi-scale detail enhancement image segmentation method, characterized in that: The multi-scale detail enhancement image segmentation method comprises: Obtain the image to be segmented; Inputting the image to be segmented into a segmentation network composed of multiple scale convolution channels to obtain multiple output subsets corresponding one-to-one to the scale convolution channels; the scale convolution channels include depthwise separable convolution kernels composed of depthwise convolution and pointwise convolution, and the depthwise separable convolution kernels are composed of large convolution kernels larger than a preset size threshold; splicing all the output subsets to obtain an output set; Inputting the output set into a plurality of preset detail enhancement units respectively, and outputting a plurality of feature information corresponding to the detail enhancement units one by one; the plurality of detail enhancement units have a plurality of scale convolutions; Fusing the feature information to obtain a segmentation prediction map; The plurality of detail enhancement units are obtained by the following method: Screening the preset convolution kernels greater than the stacking threshold in the plurality of detail enhancement units to obtain the convolution kernels to be stacked; Superimposing the convolution kernel to be superimposed by a standard convolution kernel and an asymmetric convolution kernel to obtain a superimposed convolution kernel; Arranging the detail enhancement units from large to small according to the stacked convolution kernels to obtain the plurality of detail enhancement units; The method of superimposing the convolution kernels to be superimposed by a standard convolution kernel and an asymmetric convolution kernel to obtain a superimposed convolution kernel comprises: Setting both the standard convolution kernel and the asymmetric convolution kernel as the large convolution kernel; The large convolution kernels are superimposed by the additivity of the two-dimensional convolution kernels to obtain a superimposed convolution kernel; wherein the calculation formula for superimposing by the additivity of the two-dimensional convolution kernels includes: in, represents the input matrix, represents a two-dimensional convolution kernel of compatible size, Represents the addition operation of the corresponding position elements of the kernel parameters.

2. The multi-scale detail enhancement image segmentation method according to claim 1, characterized in that: Inputting the image to be segmented into a segmentation network composed of multiple scale convolution channels, obtaining multiple output subsets corresponding one-to-one to the scale convolution channels, including: Dividing the image to be segmented into multiple input subsets; The input subset is input into the corresponding scale convolution channel to obtain an output subset corresponding one-to-one to the scale convolution channel.

3. The multi-scale detail enhancement image segmentation method according to claim 2, characterized in that: The step of fusing the feature information to obtain a segmentation prediction map includes: Fusing the feature information through a decoder architecture to obtain fused feature information; The fused feature information is up-sampled to obtain the segmentation prediction map.

4. The multi-scale detail enhancement image segmentation method according to claim 2, characterized in that: Convolution operations are performed independently between each of the scale convolution channels.

5. The multi-scale detail enhancement image segmentation method according to claim 1, characterized in that: The large convolution kernel is obtained by compatibility with a small convolution kernel that is smaller than the preset size threshold.

6. A multi-scale detail-enhanced image segmentation system, characterized in that: The multi-scale detail enhancement image segmentation system comprises: An image acquisition module, used to acquire the image to be segmented; a channel multi-scale module, configured to input the image to be segmented into a segmentation network consisting of multiple scale convolution channels, and obtain multiple output subsets corresponding one-to-one to the scale convolution channels; the scale convolution channels include depthwise separable convolution kernels consisting of depthwise convolution and pointwise convolution, and the depthwise separable convolution kernels are composed of large convolution kernels larger than a preset size threshold; An output subset splicing module, used for splicing all the output subsets to obtain an output set; A multi-scale detail enhancement module, configured to input the output set into a plurality of preset detail enhancement units, and output a plurality of feature information corresponding to the detail enhancement units; the plurality of detail enhancement units having a plurality of scale convolutions; A feature fusion module is used to fuse the feature information to obtain a segmentation prediction map; The plurality of detail enhancement units are obtained by the following method: Screening the preset convolution kernels greater than the stacking threshold in the plurality of detail enhancement units to obtain the convolution kernels to be stacked; Superimposing the convolution kernel to be superimposed by a standard convolution kernel and an asymmetric convolution kernel to obtain a superimposed convolution kernel; Arranging the detail enhancement units from large to small according to the stacked convolution kernels to obtain the plurality of detail enhancement units; The method of superimposing the convolution kernels to be superimposed by a standard convolution kernel and an asymmetric convolution kernel to obtain a superimposed convolution kernel comprises: Setting both the standard convolution kernel and the asymmetric convolution kernel as the large convolution kernel; The large convolution kernels are superimposed by the additivity of the two-dimensional convolution kernels to obtain a superimposed convolution kernel; wherein the calculation formula for superimposing by the additivity of the two-dimensional convolution kernels includes: in, represents the input matrix, represents a two-dimensional convolution kernel of compatible size, Represents the addition operation of the corresponding position elements of the kernel parameters.

7. An electronic device, characterized in that: It includes at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor to enable the at least one control processor to perform the multi-scale detail enhancement image segmentation method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the multi-scale detail enhancement image segmentation method according to any one of claims 1 to 5.

Images