Image segmentation method and apparatus based on multi-scale feature extraction

Abstract

This invention discloses an image segmentation method and apparatus based on multi-scale feature extraction. The method includes: inputting the original image into a pre-trained U... 2 The data input layer of the Net++ network performs image resizing to obtain the target image; the target image is then input into U... 2 The feature extraction layer of the Net++ network performs multi-scale feature extraction to obtain a set of candidate feature maps. The feature extraction layer comprises multiple levels, each level including at least one multi-scale feature extraction module. When a level includes at least two multi-scale feature extraction modules, an attention module is set between each pair of adjacent multi-scale feature extraction modules in that level. The set of candidate feature maps is then input into U... 2 The multi-sided output fusion layer of the Net++ network performs feature fusion processing to obtain the target segmentation feature map. It is evident that implementing this invention can improve the extraction efficiency and accuracy of image feature extraction results, which is beneficial for improving the segmentation efficiency and accuracy of image segmentation results.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image segmentation method and apparatus based on multi-scale feature extraction. Background Technology

[0002] Currently, when analyzing the state of a body part, an image of that body part can be obtained by taking a photograph. The body part (target) and other unrelated parts (background) within the image can then be segmented to obtain regional and edge information, allowing for a more accurate analysis of its state. For example, when analyzing the state of teeth, X-ray imaging can be used to obtain X-ray images. By segmenting the tooth and non-tooth portions of the X-ray image, information about the tooth region and edges can be obtained, enabling a more accurate analysis of the teeth's state. Therefore, the accuracy of image segmentation plays a crucial role in analyzing the state of body parts.

[0003] However, practical experience has shown that if the image to be analyzed has characteristics such as low contrast and uneven intensity distribution, segmentation using existing techniques often results in problems such as objects sticking together and inaccurate object edge segmentation. Therefore, proposing a technical solution that can improve the accuracy of image segmentation is particularly important. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide an image segmentation method and apparatus based on multi-scale feature extraction, which can improve the accuracy of image segmentation.

[0005] To address the aforementioned technical problems, the first aspect of this invention discloses an image segmentation method based on multi-scale feature extraction, the method comprising:

[0006] The original image is input into the pre-trained U... 2 The data input layer of the Net++ network performs image size reduction processing to obtain a target image, the image size of which is the target size, and the data input layer is used to reduce the image size of the original image to the target size;

[0007] The target image is input into the U 2The feature extraction layer of the Net++ network performs multi-scale feature extraction to obtain a candidate feature map set. The feature extraction layer includes multiple levels, each level including at least one multi-scale feature extraction module. When a level includes at least two multi-scale feature extraction modules, an attention module is provided between each pair of adjacent multi-scale feature extraction modules in that level. Each attention module is used to identify the region of interest (ROI) of the feature map input to it and input the ROI to the next adjacent multi-scale feature extraction module. The candidate feature map set includes at least two candidate feature map subsets, and the feature maps contained in each candidate feature map subset are used for feature fusion to obtain a segmentation feature map.

[0008] The candidate feature map set is input into the U 2 The multi-sided output fusion layer of the Net++ network performs feature fusion processing to obtain the target segmentation feature map.

[0009] As an optional implementation, in the first aspect of the present invention, when a certain layer includes only one multi-scale feature extraction module, the multi-scale feature extraction module is used as an encoder; when a certain layer includes two multi-scale feature extraction modules, the first multi-scale feature extraction module at that level is used as an encoder, and the last multi-scale feature extraction module at that level is used as a decoder, and a skip path is provided between the encoder and the decoder of that level; when a certain layer includes two or more multi-scale feature extraction modules, the remaining multi-scale feature extraction modules of that level, excluding the encoder and the decoder, are used as dense residual modules, and skip paths are provided between the encoder and each dense residual module of that level, and between each dense residual module and the decoder of that level; wherein, the encoder is used to extract multi-scale features of the image input to the encoder; the decoder is used to restore the resolution of the image input to the decoder; the dense residual module is used to integrate information from deeper levels into the level where the dense residual module is located;

[0010] Wherein, if a certain level contains more multi-scale feature extraction modules, the level is shallower, and the shallowest level is the level that contains the most multi-scale feature extraction modules; if a certain level contains fewer multi-scale feature extraction modules, the level is deeper, and the deepest level is the level that contains only one multi-scale feature extraction module.

[0011] A downsampling path is provided between the encoder of each level and the encoder of the adjacent deep level; an upsampling path is provided between each multi-scale feature extraction module and the first adjacent module of the adjacent shallow level, wherein the position of the first adjacent module in its level is adjacent to the position of the multi-scale feature extraction module in its level.

[0012] For each attention module, the feature map input to the attention module includes a first input feature map and a second input feature map; wherein, the multi-scale feature extraction module that receives the region of interest output by the attention module is used as a location reference module, and the feature map output by the multi-scale feature extraction module that is the first input feature map is used; the feature map output by the second neighboring module of the adjacent deep layer of the location reference module is used as the second input feature map, and the position of the second neighboring module in its layer is the first neighboring module compared to the position of the location reference module in its layer.

[0013] As an optional implementation, in the first aspect of the invention, the target image is input to the U... 2 The feature extraction layer of the Net++ network performs multi-scale feature extraction to obtain a set of candidate feature maps, including:

[0014] The target image is input into the U 2 The shallowest level encoder in the feature extraction layer of the Net++ network performs downsampling operations on the target image based on the downsampling path until the adjacent shallow level of the deepest level, to obtain a multi-scale downsampled feature map.

[0015] Based on the deepest level encoder, dilated convolution processing is performed on the multi-scale downsampled feature maps output by the encoders of the adjacent shallow levels of the deepest level to obtain dilated feature maps.

[0016] For each of the multi-scale feature extraction modules other than all the encoders, based on the received region of interest, an upsampling operation is performed on all feature maps input to the multi-scale feature extraction module to obtain a multi-scale upsampled feature map;

[0017] All the multi-scale upsampled feature maps in the shallowest layer are determined as the first candidate feature map subset;

[0018] The dilated feature map and the multi-scale upsampled feature map output by the decoder of each of the remaining layers except the shallowest layer are determined as the second candidate feature map subset.

[0019] As an optional implementation, in the first aspect of the present invention, when the original image is an X-ray dental image, the target segmentation feature map is a segmentation feature map with teeth as the segmentation target; the type of the multi-scale feature extraction module includes a first feature extraction type and a second feature extraction type;

[0020] The multi-scale feature extraction module includes multiple sub-levels, and each sub-level includes at least one multi-scale feature extraction sub-module. When a sub-level includes a multi-scale feature extraction sub-module for receiving an input feature map, the sub-level is the shallowest sub-level. The multi-scale feature extraction sub-module for receiving the input feature map is an input convolution sub-module. The shallowest sub-level also includes a fusion sub-module, which is a multi-scale feature extraction sub-module used to fuse features and output a fused feature map. When a sub-level includes only one multi-scale feature extraction sub-module, the sub-level is the deepest sub-level.

[0021] When a certain sub-level includes at least two multi-scale feature extraction sub-modules, an attention sub-module is provided between each pair of adjacent multi-scale feature extraction sub-modules in the level, and a skip sub-path is provided between each multi-scale feature extraction sub-module in the level; for each attention sub-module, the attention sub-module receives the feature map output by the preceding adjacent multi-scale feature extraction sub-module and the feature map output by the preceding multi-scale feature extraction sub-module in the adjacent deep sub-level, and the subsequent adjacent multi-scale feature extraction sub-module of the attention sub-module receives the region of interest output by the attention sub-module;

[0022] A first feature extraction sub-path is provided between the first multi-scale feature extraction sub-module at each sub-level and the first multi-scale feature extraction sub-module at the adjacent deep level; a second feature extraction sub-path is provided between the last multi-scale feature extraction sub-module at each sub-level and the last multi-scale feature extraction sub-module at the adjacent shallow level.

[0023] As an optional implementation, in the first aspect of the present invention, when the type of the multi-scale feature extraction module is the first feature extraction type, the first feature extraction sub-path is a downsampling sub-path, and the multi-scale feature extraction sub-modules on the downsampling path, except for the shallowest sub-layer, are convolutional sub-modules or downsampling sub-modules; the first feature extraction sub-path is an upsampling sub-path, and the multi-scale feature extraction sub-modules on the upsampling path, except for the shallowest sub-layer, are convolutional sub-modules or upsampling sub-modules; the remaining multi-scale feature extraction sub-modules in each sub-layer, except for the upsampling sub-module and the downsampling sub-module, are dense convolutional sub-modules.

[0024] As an optional implementation, in the first aspect of the present invention, when the type of the multi-scale feature extraction module is the second feature extraction type, in the multi-scale feature extraction module, the multi-scale feature extraction sub-module in each of the sub-layers other than the shallowest sub-layer is a dilated convolution sub-module.

[0025] The deepest level encoder is of the second feature extraction type.

[0026] As an optional implementation, in the first aspect of the invention, the step of inputting the candidate feature map set into the U... 2 The multi-sided output fusion layer of the Net++ network performs feature fusion processing to obtain the target segmentation feature map, including:

[0027] Based on a preset activation function, feature fusion processing is performed on all candidate feature maps in the first subset of candidate feature maps to obtain a first segmentation feature map; wherein, the calculation formula for the first segmentation feature map is as follows:

[0028]

[0029] In the formula, Indicates a join operation, Y 0,5 For the first segmentation feature map, Y 0,1 Y 0,2 Y 0,3 and Y 0,4 All of them are feature maps from the first subset of candidate feature maps;

[0030] Based on the preset activation function, feature fusion processing is performed on all candidate feature maps in the second candidate feature map subset to obtain a second segmentation feature map; wherein, the calculation formula of the second segmentation feature map is as follows:

[0031]

[0032] In the formula, Y 5,0 Y is the second segmentation feature map. 1,0 Y 2,0 Y 3,0 and Y 4,0 All of them are feature maps from the second subset of candidate feature maps;

[0033] Based on the preset activation function, feature fusion processing is performed on the first segmentation feature map and the second segmentation feature map to obtain the target segmentation feature map; wherein, the calculation formula of the target segmentation feature map is as follows:

[0034]

[0035] In the formula, Y 5,5 This is the segmentation feature map of the target.

[0036] A second aspect of the present invention discloses an image segmentation apparatus based on multi-scale feature extraction, the apparatus comprising:

[0037] Image resizing unit, used to input the original image into a pre-trained U... 2 The data input layer of the Net++ network performs image size reduction processing to obtain a target image, the image size of which is the target size, and the data input layer is used to reduce the image size of the original image to the target size;

[0038] A multi-scale feature extraction unit is used to input the target image into the U. 2 The feature extraction layer of the Net++ network performs multi-scale feature extraction to obtain a candidate feature map set. The feature extraction layer includes multiple levels, each level including at least one multi-scale feature extraction module. When a level includes at least two multi-scale feature extraction modules, an attention module is provided between each pair of adjacent multi-scale feature extraction modules in that level. Each attention module is used to identify the region of interest (ROI) of the feature map input to it and input the ROI to the next adjacent multi-scale feature extraction module. The candidate feature map set includes at least two candidate feature map subsets, and the feature maps contained in each candidate feature map subset are used for feature fusion to obtain a segmentation feature map.

[0039] A multi-sided output fusion unit is used to input the candidate feature map set into the U. 2 The multi-sided output fusion layer of the Net++ network performs feature fusion processing to obtain the target segmentation feature map.

[0040] As an optional implementation, in a second aspect of the invention, when a certain layer includes only one multi-scale feature extraction module, the multi-scale feature extraction module is used as an encoder; when a certain layer includes two multi-scale feature extraction modules, the first multi-scale feature extraction module at that layer position is used as an encoder, and the last multi-scale feature extraction module at that layer position is used as a decoder, and a skip path is provided between the encoder and the decoder of that layer; when a certain layer includes two or more multi-scale feature extraction modules, the remaining multi-scale feature extraction modules of that layer other than the encoder and the decoder are used as dense residual modules, and skip paths are provided between the encoder and each dense residual module of that layer, and between each dense residual module of that layer and the decoder of that layer; wherein, the encoder is used to extract multi-scale features of the image input to the encoder; the decoder is used to restore the resolution of the image input to the decoder; the dense residual module is used to integrate information from deeper layers into the layer where the dense residual module is located;

[0041] Wherein, if a certain level contains more multi-scale feature extraction modules, the level is shallower, and the shallowest level is the level that contains the most multi-scale feature extraction modules; if a certain level contains fewer multi-scale feature extraction modules, the level is deeper, and the deepest level is the level that contains only one multi-scale feature extraction module.

[0042] A downsampling path is provided between the encoder of each level and the encoder of the adjacent deep level; an upsampling path is provided between each multi-scale feature extraction module and the first adjacent module of the adjacent shallow level, wherein the position of the first adjacent module in its level is adjacent to the position of the multi-scale feature extraction module in its level.

[0043] For each attention module, the feature map input to the attention module includes a first input feature map and a second input feature map; wherein, the multi-scale feature extraction module that receives the region of interest output by the attention module is used as a location reference module, and the feature map output by the multi-scale feature extraction module that is the first input feature map is used; the feature map output by the second neighboring module of the adjacent deep layer of the location reference module is used as the second input feature map, and the position of the second neighboring module in its layer is the first neighboring module compared to the position of the location reference module in its layer.

[0044] As an optional implementation, in a second aspect of the invention, the multi-scale feature extraction unit inputs the target image into the U... 2The Net++ network's feature extraction layer performs multi-scale feature extraction to obtain a candidate feature map set in the following ways:

[0045] The target image is input into the U 2 The shallowest level encoder in the feature extraction layer of the Net++ network performs downsampling operations on the target image based on the downsampling path until the adjacent shallow level of the deepest level, to obtain a multi-scale downsampled feature map.

[0046] Based on the deepest level encoder, dilated convolution processing is performed on the multi-scale downsampled feature maps output by the encoders of the adjacent shallow levels of the deepest level to obtain dilated feature maps.

[0047] For each of the multi-scale feature extraction modules other than all the encoders, based on the received region of interest, an upsampling operation is performed on all feature maps input to the multi-scale feature extraction module to obtain a multi-scale upsampled feature map;

[0048] All the multi-scale upsampled feature maps in the shallowest layer are determined as the first candidate feature map subset;

[0049] The dilated feature map and the multi-scale upsampled feature map output by the decoder of each of the remaining layers except the shallowest layer are determined as the second candidate feature map subset.

[0050] As an optional implementation, in a second aspect of the present invention, when the original image is an X-ray dental image, the target segmentation feature map is a segmentation feature map with teeth as the segmentation target; the type of the multi-scale feature extraction module includes a first feature extraction type and a second feature extraction type;

[0051] The multi-scale feature extraction module includes multiple sub-levels, and each sub-level includes at least one multi-scale feature extraction sub-module. When a sub-level includes a multi-scale feature extraction sub-module for receiving an input feature map, the sub-level is the shallowest sub-level. The multi-scale feature extraction sub-module for receiving the input feature map is an input convolution sub-module. The shallowest sub-level also includes a fusion sub-module, which is a multi-scale feature extraction sub-module used to fuse features and output a fused feature map. When a sub-level includes only one multi-scale feature extraction sub-module, the sub-level is the deepest sub-level.

[0052] When a certain sub-level includes at least two multi-scale feature extraction sub-modules, an attention sub-module is provided between each pair of adjacent multi-scale feature extraction sub-modules in the level, and a skip sub-path is provided between each multi-scale feature extraction sub-module in the level; for each attention sub-module, the attention sub-module receives the feature map output by the preceding adjacent multi-scale feature extraction sub-module and the feature map output by the preceding multi-scale feature extraction sub-module in the adjacent deep sub-level, and the subsequent adjacent multi-scale feature extraction sub-module of the attention sub-module receives the region of interest output by the attention sub-module;

[0053] A first feature extraction sub-path is provided between the first multi-scale feature extraction sub-module at each sub-level and the first multi-scale feature extraction sub-module at the adjacent deep level; a second feature extraction sub-path is provided between the last multi-scale feature extraction sub-module at each sub-level and the last multi-scale feature extraction sub-module at the adjacent shallow level.

[0054] As an optional implementation, in a second aspect of the present invention, when the type of the multi-scale feature extraction module is the first feature extraction type, the first feature extraction sub-path is a downsampling sub-path, and the multi-scale feature extraction sub-modules on the downsampling path, except for the shallowest sub-layer, are convolutional sub-modules or downsampling sub-modules; the first feature extraction sub-path is an upsampling sub-path, and the multi-scale feature extraction sub-modules on the upsampling path, except for the shallowest sub-layer, are convolutional sub-modules or upsampling sub-modules; the remaining multi-scale feature extraction sub-modules in each sub-layer, except for the upsampling sub-module and the downsampling sub-module, are dense convolutional sub-modules.

[0055] As an optional implementation, in a second aspect of the present invention, when the type of the multi-scale feature extraction module is the second feature extraction type, in the multi-scale feature extraction module, the multi-scale feature extraction sub-module in each of the sub-layers other than the shallowest sub-layer is a dilated convolution sub-module.

[0056] The deepest level encoder is of the second feature extraction type.

[0057] As an optional implementation, in a second aspect of the invention, the multi-side output fusion unit inputs the candidate feature map set into the U... 2 The specific methods for obtaining target segmentation feature maps by performing feature fusion processing on the multi-sided output fusion layer of the Net++ network include:

[0058] Based on a preset activation function, feature fusion processing is performed on all candidate feature maps in the first subset of candidate feature maps to obtain a first segmentation feature map; wherein, the calculation formula for the first segmentation feature map is as follows:

[0059]

[0060] In the formula, Indicates a join operation, Y 0,5 For the first segmentation feature map, Y 0,1 Y 0,2 Y 0,3 and Y 0,4 All of them are feature maps from the first subset of candidate feature maps;

[0061] Based on the preset activation function, feature fusion processing is performed on all candidate feature maps in the second candidate feature map subset to obtain a second segmentation feature map; wherein, the calculation formula of the second segmentation feature map is as follows:

[0062]

[0063] In the formula, Y 5,0 Y is the second segmentation feature map. 1,0 Y 2,0 Y 3,0 and Y 4,0 All of them are feature maps from the second subset of candidate feature maps;

[0064] Based on the preset activation function, feature fusion processing is performed on the first segmentation feature map and the second segmentation feature map to obtain the target segmentation feature map; wherein, the calculation formula of the target segmentation feature map is as follows:

[0065]

[0066] In the formula, Y 5,5 This is the segmentation feature map of the target.

[0067] A third aspect of the present invention discloses another image segmentation apparatus based on multi-scale feature extraction, the apparatus comprising:

[0068] Memory containing executable program code;

[0069] A processor coupled to the memory;

[0070] The processor calls the executable program code stored in the memory to execute the image segmentation method based on multi-scale feature extraction disclosed in the first aspect of the present invention.

[0071] The fourth aspect of the present invention discloses a computer storage medium storing computer instructions, which, when invoked, are used to execute the image segmentation method based on multi-scale feature extraction disclosed in the first aspect of the present invention.

[0072] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0073] In this embodiment of the invention, the original image is input into a pre-trained U... 2 The Net++ network's data input layer performs image resizing to obtain the target image, whose size is the target size. The data input layer is used to reduce the original image size to the target size; the target image is then input into U... 2 The feature extraction layer of the Net++ network performs multi-scale feature extraction to obtain a candidate feature map set. The feature extraction layer includes multiple levels, each level including at least one multi-scale feature extraction module. When a level includes at least two multi-scale feature extraction modules, an attention module is set between each pair of adjacent multi-scale feature extraction modules in that level. Each attention module is used to identify the region of interest (ROI) of the feature map input to it and input the ROI to the next adjacent multi-scale feature extraction module. The candidate feature map set includes at least two candidate feature map subsets. The feature maps contained in each candidate feature map subset are used for feature fusion to obtain a segmentation feature map. The candidate feature map set is input to U... 2 The multi-sided output fusion layer of the Net++ network performs feature fusion processing to obtain the target segmentation feature map. It is evident that implementing this invention can achieve this through U... 2 The data input layer of the Net++ network receives input to U... 2 The original image of the Net++ network is reduced to the target size to obtain the target image, and then based on U... 2 The feature extraction layer of the Net++ network performs multi-scale feature extraction on the target image, obtaining a set of candidate feature maps, and then uses U... 2 The multi-sided output fusion layer of the Net++ network performs feature fusion processing on the candidate feature map set to obtain the target segmentation feature map. This improves the extraction efficiency and accuracy of image feature extraction results. Based on the more accurate image feature extraction results, target detection is performed. Using the detected target as the basis for image segmentation improves the accuracy of determining the image segmentation basis, which is beneficial to improving the segmentation efficiency and accuracy of the image segmentation results. This, in turn, improves the accuracy of analysis based on the image segmentation results. Furthermore, by setting an attention module to identify key features in the image, the accuracy of extracting key features can be improved, thereby improving the reliability of determining the image segmentation basis and further improving the accuracy of the image segmentation results. Attached Figure Description

[0074] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0075] Figure 1 This is a flowchart illustrating an image segmentation method based on multi-scale feature extraction disclosed in an embodiment of the present invention;

[0076] Figure 2 This is a flowchart illustrating another image segmentation method based on multi-scale feature extraction disclosed in an embodiment of the present invention;

[0077] Figure 3 This is a U disclosed in the embodiments of the present invention. 2 A schematic diagram of the Net++ network architecture;

[0078] Figure 4 This is a U disclosed in an embodiment of the present invention. 2 A schematic diagram of the feature extraction layer of the Net++ network;

[0079] Figure 5 This is a U disclosed in the embodiments of the present invention. 2 A schematic diagram of the multi-scale feature extraction module in the feature extraction layer of the Net++ network;

[0080] Figure 6 This is another U disclosed in the embodiments of the present invention. 2 A schematic diagram of the multi-scale feature extraction module in the feature extraction layer of the Net++ network;

[0081] Figure 7 This is a U disclosed in the embodiments of the present invention. 2 A schematic diagram of the data input layer structure of a Net++ network;

[0082] Figure 8 This is an image segmentation result diagram of an image segmentation method based on multi-scale feature extraction disclosed in an embodiment of the present invention;

[0083] Figure 9 This is an image segmentation result diagram of another image segmentation method based on multi-scale feature extraction disclosed in an embodiment of the present invention;

[0084] Figure 10 This is a schematic diagram of the structure of an image segmentation device based on multi-scale feature extraction disclosed in an embodiment of the present invention;

[0085] Figure 11This is a schematic diagram of another image segmentation device based on multi-scale feature extraction disclosed in an embodiment of the present invention. Detailed Implementation

[0086] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0087] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or end that includes a series of steps or sub-modules is not limited to the listed steps or sub-modules, but may optionally include steps or sub-modules not listed, or may optionally include other steps or sub-modules inherent to these processes, methods, products, or ends.

[0088] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0089] This invention discloses an image segmentation method and apparatus based on multi-scale feature extraction, which can segment images using U... 2 The data input layer of the Net++ network receives input to U... 2 The original image of the Net++ network is reduced to the target size to obtain the target image, and then based on U... 2 The feature extraction layer of the Net++ network performs multi-scale feature extraction on the target image, obtaining a set of candidate feature maps, and then uses U... 2The multi-sided output fusion layer of the Net++ network performs feature fusion processing on the candidate feature map set to obtain the target segmentation feature map. This improves the extraction efficiency and accuracy of image features. Based on the more accurate image feature extraction results, target detection is performed. Using the detected target as the basis for image segmentation improves the accuracy of determining the segmentation criteria, thus enhancing the segmentation efficiency and accuracy, and consequently improving the accuracy of analysis based on the image segmentation results. Furthermore, by setting an attention module to identify key features in the image, the accuracy of key feature extraction is improved, thereby increasing the reliability of determining the segmentation criteria and further enhancing the accuracy of the image segmentation results. These will be explained in detail below.

[0090] Example 1

[0091] Please see Figure 1 , Figure 1 This is a flowchart illustrating an image segmentation method based on multi-scale feature extraction disclosed in an embodiment of the present invention. Figure 1 The described image segmentation method based on multi-scale feature extraction can be applied to an image segmentation device based on multi-scale feature extraction. This device may include one of a computing device, a computing terminal, a computing system, and a server, wherein the server may be a local server or a cloud server; this embodiment of the invention does not limit the scope. Figure 1 As shown, this image segmentation method based on multi-scale feature extraction can include the following operations:

[0092] 101. Input the original image into the pre-trained U... 2 The data input layer of the Net++ network performs image size reduction processing to obtain the target image.

[0093] In this embodiment of the invention, U 2 Net++ networks are end-to-end trainable image segmentation networks, and U 2 The Net++ network consists of a data input layer, a feature extraction layer, and a multi-sided output fusion layer. The original image can be a biomedical image, a natural image, or any other image requiring target detection and image segmentation; this embodiment of the invention does not limit the type of image. Specifically, the biomedical image can be an X-ray image, and further, the X-ray image can be a dental X-ray image or an X-ray image of other body parts. Dental X-ray images can be used to analyze dental problems; this embodiment of the invention does not limit the type of image. Furthermore, the original image can be an image directly acquired after shooting or an image obtained after enhancement processing; this embodiment of the invention does not limit the type of image. The target image has a target size, and the data input layer is used to reduce the image size of the original image to the target size, i.e., reduce the input to U. 2The resolution and complexity of the original image in the Net++ network are adjusted to improve the efficiency of subsequent feature extraction. Among them, U... 2 A schematic diagram of the Net++ network architecture can be seen as follows: Figure 3 As shown.

[0094] 102. Input the target image into U 2 The feature extraction layer of the Net++ network performs multi-scale feature extraction to obtain a set of candidate feature maps.

[0095] In this embodiment of the invention, the feature extraction layer includes multiple levels, each level including at least one multi-scale feature extraction module; when a level includes at least two multi-scale feature extraction modules, an attention module is provided between each two adjacent multi-scale feature extraction modules in that level, each attention module is used to identify the region of interest of the feature map input to the attention module and input the region of interest to the next adjacent multi-scale feature extraction module of the attention module; the candidate feature map set includes at least two candidate feature map subsets, each candidate feature map subset includes at least one feature map, and the feature maps contained in each candidate feature map subset are used for feature fusion to obtain a segmentation feature map; for example, the candidate feature map set includes two candidate feature map subsets, each candidate feature map subset includes four feature maps. The attention module is a module that introduces an attention mechanism. The attention mechanism can be a channel attention mechanism, a spatial attention mechanism, or a mechanism that combines channel attention and spatial attention. Further, the attention module can specifically be one of the following: SE (Squeeze-and-Excitation) module, CBAM (Convolutional Block Attention Module), ECA (Efficient Channel Attention) module, and other specific types of attention modules. This embodiment of the invention does not limit the scope of the attention module.

[0096] 103. Input the candidate feature map set into U 2 The multi-sided output fusion layer of the Net++ network performs feature fusion processing to obtain the target segmentation feature map.

[0097] In this embodiment of the invention, the multi-side output fusion layer employs the MSOF (Multiple Side-Output Fusion) strategy for deep supervision. When U 2 A schematic diagram of the Net++ network architecture is shown below. Figure 3 As shown, the target segmentation feature map is Figure 3 Y in 5,5 .

[0098] As can be seen, the method described in the embodiments of the present invention can be implemented through U 2 The data input layer of the Net++ network receives input to U... 2 The original image of the Net++ network is reduced to the target size to obtain the target image, and then based on U... 2 The feature extraction layer of the Net++ network performs multi-scale feature extraction on the target image, obtaining a set of candidate feature maps, and then uses U... 2 The multi-sided output fusion layer of the Net++ network performs feature fusion processing on the candidate feature map set to obtain the target segmentation feature map. This improves the extraction efficiency and accuracy of image feature extraction results. Based on the more accurate image feature extraction results, target detection is performed. Using the detected target as the basis for image segmentation improves the accuracy of determining the image segmentation basis, which is beneficial to improving the segmentation efficiency and accuracy of the image segmentation results. This, in turn, improves the accuracy of analysis based on the image segmentation results. Furthermore, by setting an attention module to identify key features in the image, the accuracy of extracting key features can be improved, thereby improving the reliability of determining the image segmentation basis and further improving the accuracy of the image segmentation results.

[0099] In an optional embodiment, when a layer includes only one multi-scale feature extraction module, the multi-scale feature extraction module is used as an encoder; when a layer includes two multi-scale feature extraction modules, the first multi-scale feature extraction module at that layer position is used as an encoder, and the last multi-scale feature extraction module at that layer position is used as a decoder, and a skip path is provided between the encoder and the decoder of that layer; when a layer includes two or more multi-scale feature extraction modules, the remaining multi-scale feature extraction modules of that layer other than the encoder and decoder are used as dense residual modules, and skip paths are provided between the encoder and each dense residual module of that layer, and between each dense residual module of that layer and the decoder of that layer; wherein, the encoder is used to extract multi-scale features of the image input to the encoder; the decoder is used to restore the resolution of the image input to the decoder; and the dense residual module is used to integrate information from deeper layers into the layer in which the dense residual module is located;

[0100] Among them, the more multi-scale feature extraction modules a certain level contains, the shallower the level is, and the shallowest level is the level that contains the most multi-scale feature extraction modules; the fewer multi-scale feature extraction modules a certain level contains, the deeper the level is, and the deepest level is the level that contains only one multi-scale feature extraction module.

[0101] Each level of encoder has a downsampling path between it and the encoder of the adjacent deep level; each multi-scale feature extraction module has an upsampling path between it and the first adjacent module of the adjacent shallow level, wherein the position of the first adjacent module in its level is adjacent to the position of the multi-scale feature extraction module in its level.

[0102] For each attention module, the feature map input to the attention module includes a first input feature map and a second input feature map; wherein, the multi-scale feature extraction module that receives the region of interest output by the attention module is used as the location reference module, and the feature map output by the multi-scale feature extraction module that is the first neighbor of the location reference module is used as the first input feature map; the feature map output by the second neighbor module of the adjacent deep layer of the location reference module is used as the second input feature map, and the position of the second neighbor module in its layer is the first neighbor of the location reference module in its layer.

[0103] It should be noted that when the multi-scale feature extraction module is used as an encoder, it can extract image features and capture and retain more contextual information. When the multi-scale feature extraction module is used as a decoder, each decoder cascades the upsampled feature maps of the previous level (deeper level) decoder and the feature maps of the encoder at the same level (same layer). This allows for the acquisition of more contextual information and low-level features from the encoder during decoding, and by cascading these feature maps, information at different scales and semantic levels can be captured. When the multi-scale feature extraction module is used as a dense residual module, it integrates deep-level information into the current layer, thereby capturing features at different scales and abstraction levels. This enhances the semantic expressiveness of the feature maps output by the encoder, making these feature maps closer to the feature maps required by the decoder, thus improving the network's performance and accuracy.

[0104] In an embodiment of the present invention, U is an example. 2 The feature extraction layer of a Net++ network can include a five-level encoder, a four-level decoder, and a series of nested dense residual modules. The structural diagram of the feature extraction layer can be shown as follows: Figure 4 As shown. The aforementioned multi-scale feature extraction module can be... Figure 4 The RSUP (ReSidual U-blocks Plus) module is used in this module. By fusing receptive domains at different scales, the RSUP module can capture more contextual information from various scales. Furthermore, compared to other residual modules, RSUP uses convolution based on the Unet++ architecture, while other residual modules use single-layer convolution; RSUP utilizes local features composed of weight layers to replace the original features, thus improving the U-blocks' receptive capabilities. 2The Net++ network can directly extract features from multiple scales within each RSUP module, thus capturing target features more accurately. When the multi-scale feature extraction module is used as the encoder, the RSUP-L module is used to extract image features. L represents the depth of the RSUP module, and L is determined based on the size of the feature map of the input feature extraction layer. The greater the depth L of the RSUP module, the deeper the network, and the larger the receptive area, enabling it to perceive a wider range of local and global features. The RSUP-L module can... Figure 4 The RSUP7, RSUP6, RSUP5, and RSUP4 modules are shown; the RSUP4F module is used to capture and retain more contextual information.

[0105] As can be seen, this optional embodiment, by setting up multi-level multi-scale feature extraction modules and using these modules as encoders, decoders, and dense residual modules, extracts features at multiple scales from continuously downsampled feature maps. Then, through progressive upsampling, merging, and convolution, it decodes these multi-scale features into high-resolution feature maps, simultaneously capturing global background information and local detail information. This improves the comprehensiveness and accuracy of image feature extraction, thereby enhancing the comprehensiveness and reliability of the image feature extraction results. Based on more reliable image feature extraction results, image segmentation is more accurate. Furthermore, by setting nested dense residual modules and skip paths, shallow-level and deep-level features can be fused, effectively reducing gradient vanishing and network degradation problems, and improving U... 2 The Net++ network offers stability while preserving features from the original image, improving the integrity and reliability of image feature extraction results and enhancing the accuracy of image segmentation.

[0106] In this optional embodiment, when the original image is an X-ray tooth image, the target segmentation feature map is a segmentation feature map with teeth as the segmentation target; the type of the multi-scale feature extraction module includes a first feature extraction type and a second feature extraction type.

[0107] The multi-scale feature extraction module includes multiple sub-levels, each of which includes at least one multi-scale feature extraction sub-module. When a sub-level includes a multi-scale feature extraction sub-module for receiving the input feature map, this sub-level is the shallowest sub-level. The multi-scale feature extraction sub-module for receiving the input feature map is an input convolution sub-module. The shallowest sub-level also includes a fusion sub-module, which is a multi-scale feature extraction sub-module used to fuse features and output a fused feature map. When a sub-level includes only one multi-scale feature extraction sub-module, this sub-level is the deepest sub-level.

[0108] When a sub-level includes at least two multi-scale feature extraction sub-modules, an attention sub-module is set between each pair of adjacent multi-scale feature extraction sub-modules in the level, and a skip sub-path is set between each multi-scale feature extraction sub-module in the level; for each attention sub-module, the attention sub-module receives the feature map output by the preceding adjacent multi-scale feature extraction sub-module and the feature map output by the preceding multi-scale feature extraction sub-module of the adjacent deep sub-level, and the subsequent adjacent multi-scale feature extraction sub-module of the attention sub-module receives the region of interest output by the attention sub-module;

[0109] A first feature extraction sub-path is set between the first multi-scale feature extraction sub-module at each sub-level and the first multi-scale feature extraction sub-module at the adjacent deep level; a second feature extraction sub-path is set between the last multi-scale feature extraction sub-module at each sub-level and the last multi-scale feature extraction sub-module at the adjacent shallow level.

[0110] It should be noted that the input convolution submodule performs convolution calculations on the input feature map and outputs a feature map with the same number of channels as the output feature map. The multi-scale feature extraction submodule can be used as one of the following: convolution submodule, upsampling submodule, downsampling submodule, dense convolution submodule, and dilated convolution submodule. The type of multi-scale feature extraction submodule can be determined according to requirements. Optionally, the convolution submodule may include Conv+BN (Batch Normalization) + ReLU (Linear Rectified Function, a type of activation function) operation; the upsampling submodule may include upsampling + convolution + BN + ReLU operation; the downsampling submodule may include downsampling + convolution + BN + ReLU operation; and the dilated convolution submodule may include Conv2d + BN + ReLU operation. This embodiment of the invention does not impose any limitations.

[0111] As can be seen, this optional embodiment can also increase the depth of the entire network by setting multiple sub-level multi-scale feature extraction sub-modules in the multi-scale feature extraction module, and capture more contextual information from different scales by fusing receptive fields of different scales, which can improve the expansion of the receptive field range, facilitate the extraction of more and richer image features, and thus improve the comprehensiveness of the image feature extraction results.

[0112] In this optional embodiment, further optionally, when the type of the multi-scale feature extraction module is a first feature extraction type, the first feature extraction sub-path is a downsampling sub-path, and the multi-scale feature extraction sub-modules on the downsampling path, except for the shallowest sub-layer, are convolutional sub-modules or downsampling sub-modules; the first feature extraction sub-path is an upsampling sub-path, and the multi-scale feature extraction sub-modules on the upsampling path, except for the shallowest sub-layer, are convolutional sub-modules or upsampling sub-modules; the remaining multi-scale feature extraction sub-modules in each sub-layer, except for the upsampling sub-modules and downsampling sub-modules, are dense convolutional sub-modules.

[0113] Except for the deepest encoder, all other multi-scale feature extraction modules in the feature extraction layer are of the first feature extraction type.

[0114] It should be noted that in the multi-scale feature extraction module of the first feature extraction type, the input convolution submodule will input features Figure X (Dimensions are H×W×C) in Perform convolution calculations to generate a channel with C channels. out The intermediate feature map F1(x) is used as input in the downsampling path. RB(F1(x)) is used to extract and encode multi-scale contextual information, and the local and multi-scale features are fused through a summation operation. The fusion formula is as follows:

[0115] HRSU(x)=F1(x)+RB(F1(x))

[0116] In the formula, F1(x) is the intermediate feature map (local feature), and RB(F1(x)) is the multi-scale feature.

[0117] The dense convolutional submodule can represent the relationship between the semantic information in the feature map output by the encoder and the feature map required by the decoder. The calculation formula for the feature map stack of the multi-scale feature extraction submodule is as follows:

[0118]

[0119] In the formula, H(-) represents a convolution operation with an activation function, u(-) represents upsampling, [-] represents a connection layer, and the output of the multi-scale feature extraction submodule is represented as X(i,j), where i represents the layer in which the multi-scale feature extraction submodule is located, and j represents the position of the multi-scale feature extraction submodule in its layer.

[0120] After introducing the attention submodule (which can be the attention submodule of the CBAM mechanism), the calculation formula for the feature map stack of the multi-scale feature extraction submodule is modified as follows:

[0121]

[0122] In the formula, H(-) represents a convolution operation with an activation function, u(-) represents upsampling, [-] represents a connection layer, and Ag represents an attention submodule.

[0123] In this embodiment of the invention, for example, the multi-scale feature extraction module of the first feature extraction type can be an RSUP-L module. In this case, the structural diagram of the multi-scale feature extraction module can be as follows: Figure 5 As shown, where: optionally, C is marked in the downsampling path. in ,3×3,C out and C out Both multi-scale feature extraction submodules of 3×3,M are convolutional submodules, and those marked C out The multi-scale feature extraction sub-modules marked M×2, 3×3, and M adjacent to the deepest sub-layer in the upsampling path are all convolutional sub-modules. The remaining multi-scale feature extraction sub-modules in the downsampling path are all downsampling sub-modules, and the remaining multi-scale feature extraction sub-modules in the upsampling path are all upsampling sub-modules.

[0124] As can be seen, this optional embodiment can also set downsampling and upsampling operations in the multi-scale feature extraction module of the first feature extraction type, providing a deeper network structure and more attention mechanisms, which can extract richer image features, thereby further improving the comprehensiveness of the image feature extraction results and helping to further improve the accuracy of the image segmentation results; and, by setting multi-scale feature extraction modules of different depths according to the size of the feature map, different scales of feature information corresponding to different image sizes can be obtained, which can improve the flexibility of image feature extraction, thereby improving the accuracy of image feature extraction results.

[0125] In this optional embodiment, it is further optional that when the type of the multi-scale feature extraction module is the second feature extraction type, in the multi-scale feature extraction module, the multi-scale feature extraction sub-module in each sub-level except the shallowest sub-level is a dilated convolution sub-module.

[0126] Among them, the deepest level encoder is of the second feature extraction type.

[0127] It should be noted that, compared to the multi-scale feature extraction module of the first feature extraction type, the multi-scale feature extraction module of the second feature extraction type uses dilated convolution instead of downsampling and upsampling operations to capture and retain a wider range of contextual information, and helps improve the segmentation accuracy of the model when the feature map resolution is relatively low. Specifically, dilated convolution can increase the receptive field and improve the performance of the convolutional neural network. The formula for calculating the size of the dilated convolution kernel is as follows:

[0128] w = k + (k-1)(a-1)

[0129] In the formula, k is the size of the convolution kernel, a is the convolution dilation rate, and w is the actual size of the dilated convolution kernel. The formula for calculating the image size after dilated convolution is shown below:

[0130]

[0131] In the formula, v is the size of the image obtained after dilation convolution, k is the size of the convolution kernel, a is the convolution dilation rate, x is the size of the input image, P is the padding, and S is the stride.

[0132] The intermediate feature map after dilated convolution can maintain the same spatial resolution as the input feature map, thus avoiding the resolution degradation problem during feature extraction, while retaining more contextual information in the multi-scale feature extraction module of the second feature extraction type.

[0133] In this embodiment of the invention, for example, the multi-scale feature extraction module of the second feature extraction type can be an RSUP4F module. In this case, the structural diagram of the multi-scale feature extraction module can be as follows: Figure 6 As shown.

[0134] As can be seen, this optional embodiment can also expand the receptive field while maintaining the resolution of the input feature map by setting a multi-scale feature extraction module of the second feature extraction type. This can improve the stability of the segmentation network while retaining more image detail information and capturing a wider range of contextual information, thereby improving the reliability of the image feature extraction results and improving the reliability of the image segmentation results.

[0135] Example 2

[0136] Please see Figure 2 , Figure 2 This is a flowchart illustrating an image segmentation method based on multi-scale feature extraction disclosed in an embodiment of the present invention. Figure 2The described image segmentation method based on multi-scale feature extraction can be applied to an image segmentation device based on multi-scale feature extraction. This device may include one of a computing device, a computing terminal, a computing system, and a server, wherein the server may be a local server or a cloud server; this embodiment of the invention does not limit the scope. Figure 2 As shown, this image segmentation method based on multi-scale feature extraction can include the following operations:

[0137] 201. Input the original image into the pre-trained U... 2 The data input layer of the Net++ network performs image size reduction processing to obtain the target image.

[0138] 202. Input the target image into U 2 The Net++ network uses the shallowest encoder in the feature extraction layer and performs downsampling operations on the target image based on the downsampling path until the adjacent shallowest layer of the deepest layer is reached, thus obtaining a multi-scale downsampled feature map.

[0139] 203. Based on the deepest level encoder, dilated convolution is performed on the multi-scale downsampled feature maps output by the adjacent shallow level encoders to obtain dilated feature maps.

[0140] 204. For each multi-scale feature extraction module other than all encoders, based on the received region of interest, perform upsampling on all feature maps input to the multi-scale feature extraction module to obtain a multi-scale upsampled feature map.

[0141] In this embodiment of the invention, all feature maps input to the multi-scale feature extraction module include feature maps input to the multi-scale feature extraction module based on jump paths and feature maps input to the multi-scale feature extraction module based on upsampling paths.

[0142] 205. All multi-scale upsampled feature maps in the shallowest layer are identified as the first candidate feature map subset.

[0143] In this embodiment of the invention, when U 2 A schematic diagram of the Net++ network architecture is shown below. Figure 3 As shown, the first candidate feature map subset includes the Y output of RSUP7(0,1). 0,1 The Y output of RSUP7(0,2) 0,2 The Y output of RSUP7(0,3) 0,3 Y output by RSUP7(0,1) 0,4 .

[0144] 206. The dilated feature map and the multi-scale upsampled feature map output by the decoder at each level except the shallowest level are determined as the second candidate feature map subset.

[0145] In this embodiment of the invention, when U 2 A schematic diagram of the Net++ network architecture is shown below. Figure 3 As shown, the second candidate feature map subset includes the Y output of RSUP6(1,3). 1,0 The Y output of RSUP5(2,2) 2,0 The Y output of RSUP4(3,1) 3,0 Y output by RSUP4F(4,0) 4,0 .

[0146] 207. Input the candidate feature map set into U 2 The multi-sided output fusion layer of the Net++ network performs feature fusion processing to obtain the target segmentation feature map.

[0147] In an embodiment of the present invention, for example, when U 2 A schematic diagram of the Net++ network architecture is shown below. Figure 3 As shown, the formula for calculating the feature mapping stack in the jump path is:

[0148]

[0149] In the formula, RSUP(i,j) represents the convolution operation within the multi-scale feature extraction module, u(-) represents upsampling, [-] represents the connection layer, i represents the layer in which the multi-scale feature extraction sub-module is located, and j represents the position of the multi-scale feature extraction sub-module within its layer.

[0150] And, for example, the calculation formula for the convolution operation with activation function within the shallowest level multi-scale feature extraction module is as follows:

[0151] X0,0 = Θ[Input]

[0152] X0,1 = Θ[Ag(X0,0),UP(X1,0)]

[0153] X0,2=Θ[Ag(X0,0),Ag(X0,1),UP(X1,1)]

[0154] X0,3=Θ[Ag(X0,0),Ag(X0,1),Ag(X0,2),UP(X1,2)]

[0155] X 0,4 =Θ[Ag(X 0,0 ),Ag(X 0,1 ),Ag(X 0,2 ),Ag(X 0,3 ),UP(X 1,3 )]

[0156] In the formula, X i,j The multi-scale feature extraction module is represented by i, which represents the level in which the multi-scale feature extraction sub-module is located, and j which represents the position of the multi-scale feature extraction sub-module in the level. Up(-) represents upsampling, Θ(-) represents convolution operation with activation function, and Ag represents attention module.

[0157] In this embodiment of the invention, for U 2 For further detailed description of the feature extraction layer of the Net++ network, please refer to the example in Example 1 for U. 2 The detailed description of the feature extraction layer of the Net++ network will not be repeated in the embodiments of this invention.

[0158] In this embodiment of the invention, other detailed descriptions of steps 201-207, as well as the detailed descriptions of steps 101-103 in Embodiment 1, will not be repeated here.

[0159] As can be seen, the method described in the embodiments of the present invention can be implemented through U 2 The data input layer of the Net++ network receives input to U... 2 The original image of the Net++ network is reduced to the target size to obtain the target image, and then based on U... 2 The feature extraction layer of the Net++ network performs multi-scale feature extraction on the target image, obtaining a set of candidate feature maps, and then uses U... 2 The multi-sided output fusion layer of the Net++ network performs feature fusion processing on the candidate feature map set to obtain the target segmentation feature map. This improves the efficiency and accuracy of image feature extraction, thereby improving the accuracy of target detection and image segmentation, which in turn enhances the accuracy of analysis based on image segmentation results. Furthermore, by setting an attention module, it can identify regions of interest to highlight the distinction between the segmented target and the background, improving the accuracy and efficiency of local feature extraction and thus increasing the reliability of feature extraction for the segmented target, ultimately improving the U... 2The Net++ network's ability to identify segmented targets enhances the accuracy of image segmentation. Furthermore, it can downsample the target image based on the downsampling path to obtain multi-scale downsampled feature maps. Based on the deepest encoder, it performs dilated convolution on adjacent multi-scale downsampled feature maps to obtain dilated feature maps. For each multi-scale feature extraction module (excluding all encoders), it upsamples all feature maps input to that module based on the received region of interest to obtain multi-scale upsampled feature maps. Then, all multi-scale upsampled feature maps in the shallowest layer are selected as the first candidate feature map subset, and the dilated feature maps and the multi-scale upsampled feature maps output by the decoders in each layer (excluding the shallowest layer) are selected as the second candidate feature map subset. This improves the accuracy and comprehensiveness of candidate feature map determination, thereby enhancing the accuracy of image feature fusion results and ultimately improving the accuracy and reliability of image segmentation.

[0160] In an optional embodiment, the candidate feature map set is input to U. 2 The multi-sided output fusion layer of the Net++ network performs feature fusion processing to obtain the target segmentation feature map, which may include the following operations:

[0161] Based on a preset activation function, feature fusion processing is performed on all candidate feature maps in the first candidate feature map subset to obtain the first segmentation feature map; wherein, the calculation formula of the first segmentation feature map is as follows:

[0162]

[0163] In the formula, Indicates a join operation, Y 0,5 Y is the first segmentation feature map. 0,1 Y 0,2 Y 0,3 and Y 0,4 All of them are feature maps from the first candidate feature map subset;

[0164] Based on a preset activation function, feature fusion processing is performed on all candidate feature maps in the second candidate feature map subset to obtain the second segmentation feature map; the calculation formula for the second segmentation feature map is as follows:

[0165]

[0166] In the formula, Y 5,0 For the second segmentation feature map, Y 1,0 Y 2,0 Y 3,0 and Y 4,0 All of them are feature maps from the second candidate feature map subset;

[0167] Based on a preset activation function, the first segmentation feature map and the second segmentation feature map are fused to obtain the target segmentation feature map; the calculation formula for the target segmentation feature map is as follows:

[0168]

[0169] In the formula, Y 5,5 Segment the feature map for the target.

[0170] The preset activation function can be the Sigmoid function or other activation functions, and this embodiment of the invention does not limit the specific activation function.

[0171] As can be seen, this optional embodiment can fuse all candidate feature maps in the first and second candidate feature map subsets respectively based on a preset activation function to obtain a first segmentation feature map and a second segmentation feature map, respectively. Then, the first and second segmentation feature maps are fused to obtain the target segmentation feature map. This helps to enrich the number of image feature fusion criteria, improves the reliability of image feature fusion results, thereby improving the accuracy of determining image segmentation criteria, and thus improving the accuracy of image segmentation results. At the same time, it can also improve U 2 Improve the convergence of Net++ networks to enhance U 2 The stability of the Net++ network helps improve the reliability of image segmentation results.

[0172] In another alternative embodiment, U 2 The Net++ network's data input layer consists of an input layer, five convolutional layers, four pooling layers, and two normalization layers. Following the input layer, the remaining layers are arranged as follows: two 7x7 convolutional layers, two 3x3 pooling layers, one normalization layer, one 1x1 convolutional layer, two 3x3 convolutional layers, one normalization layer, and two 3x3 pooling layers. The output image of the final pooling layer is the target image.

[0173] Optionally, the pooling layer can be a maximum pooling layer, and the normalization layer can be a local normalization layer.

[0174] In this embodiment of the invention, U is an example. 2 A schematic diagram of the data input layer structure of a Net++ network can be seen as follows: Figure 7 As shown, Input is the input layer, Conv is the convolutional layer, MaxPool is the max pooling layer, and LocalRespNorm is the local normalization layer.

[0175] As can be seen, this optional embodiment can convert the input raw image into a low-resolution feature map through convolution and pooling operations, which can reduce the image size and complexity of the input feature extraction layer, thereby improving the efficiency of image feature extraction and thus improving the efficiency of image segmentation.

[0176] In an embodiment of the present invention, for example, when U 2 A schematic diagram of the Net++ network architecture is shown below. Figure 3 As shown, based on U 2 After segmenting X-ray dental images using Net++ networks and other existing technologies, the resulting images can be categorized as follows: Figure 8 and Figure 9 The image segmentation results, where, Figure 8 The image segmentation result is for an X-ray image of teeth that is generally bright. Figure 9 The image segmentation result is for an X-ray image of teeth that is generally dark.

[0177] By comparison Figure 8 and Figure 9 The segmentation results show that in generally bright X-ray dental images, the UNet network segmentation method has several problems, such as incomplete segmentation, feature loss, and over-segmentation. For images where tooth features are not clearly defined, the segmentation results are even less ideal. In generally dark X-ray dental images, the UNet network segmentation results show that the network can roughly segment five teeth, but the edge of the second upper tooth overlaps with the first tooth, resulting in unclear boundaries. When using the UNet++ network to segment X-ray dental images, although the overall segmentation performance is better than the UNet network, U... 2 Net networks further improved the segmentation results. Although U 2 The U.S.Net network improves segmentation accuracy, but the segmentation results suffer from blurred edges. For example, in the first X-ray image of teeth, the boundary of the uppermost tooth is slightly blurred, and some areas are unclear. However, the segmentation effect is better than UNet++ and UNet. The segmentation result of the Deep Snake network for the four teeth from left to right is similar to the manually annotated image, but the fifth tooth is over-segmented. The U.S.Net proposed in this embodiment of the invention... 2 The Net++ network can effectively segment each tooth in X-ray dental images, and its segmentation results are closer to those of manually annotated mask images. Therefore, based on the observation of the segmentation experimental results, we can conclude that: U 2 The Net++ network improves the segmentation accuracy of X-ray dental images and produces clearer tooth edges.

[0178] To more objectively evaluate the segmentation results of X-ray teeth images, accuracy (ACC), dice similarity coefficient (DSC), and intersection-over-union ratio (IOU) were used. These metrics can more comprehensively reflect the segmentation performance of the model. The specific values ​​of each evaluation metric are listed in the table below, where the experimental values ​​are taken from the maximum values ​​of different test data.

[0179]

[0180] Analysis of the table above reveals that using the UNet network to segment X-ray dental images resulted in the lowest scores for ACC, DSC, and IOU, indicating that UNet's segmentation performance was the worst compared to other networks. In contrast, image segmentation using the UNet network generally yielded lower scores across all metrics, suggesting less than ideal segmentation results. Furthermore, compared to the Deep Mask segmentation results, the Unet++ network showed a decrease in accuracy, although its ACC, DSC, and IOU values ​​were slightly higher than UNet's. 2 When segmenting X-ray dental images, the UNet network outperformed other networks in all metrics. While the Deep Mask and Deep Snake networks slightly outperformed the UNet network in segmentation, they still lagged behind. 2 Net network segmentation results. The U proposed in this embodiment of the invention... 2 Net++ networks outperform other networks in all metrics. Therefore, U 2 The Net++ network achieves higher segmentation accuracy for X-ray dental images compared to other segmentation networks.

[0181] Example 3

[0182] Please see Figure 10 , Figure 10 This is a schematic diagram of the structure of an image segmentation device based on multi-scale feature extraction disclosed in an embodiment of the present invention. Figure 10 The described image segmentation apparatus based on multi-scale feature extraction may include one of a computing device, a computing terminal, a computing system, and a server, wherein the server includes a local server or a cloud server, and the embodiments of the present invention are not limited thereto. Figure 8 As shown, the image segmentation device based on multi-scale feature extraction may include:

[0183] Image resizing unit 301 is used to input the original image into a pre-trained U... 2 The data input layer of the Net++ network performs image size reduction processing to obtain the target image. The image size of the target image is the target size, and the data input layer is used to reduce the image size of the original image to the target size.

[0184] The multi-scale feature extraction unit 302 is used to input the target image into U. 2 The feature extraction layer of the Net++ network performs multi-scale feature extraction to obtain a candidate feature map set. The feature extraction layer includes multiple layers, and each layer includes at least one multi-scale feature extraction module. When a certain layer includes at least two multi-scale feature extraction modules, an attention module is set between each two adjacent multi-scale feature extraction modules in that layer. Each attention module is used to identify the region of interest in the feature map input to the attention module and input the region of interest to the next adjacent multi-scale feature extraction module of the attention module. The candidate feature map set includes at least two candidate feature map subsets. The feature maps contained in each candidate feature map subset are used for feature fusion to obtain a segmentation feature map.

[0185] The multi-sided output fusion unit 303 is used to input the candidate feature map set into U. 2 The multi-sided output fusion layer of the Net++ network performs feature fusion processing to obtain the target segmentation feature map.

[0186] As can be seen, the apparatus described in the embodiments of the present invention can be implemented via U 2 The data input layer of the Net++ network receives input to U... 2 The original image of the Net++ network is reduced to the target size to obtain the target image, and then based on U... 2 The feature extraction layer of the Net++ network performs multi-scale feature extraction on the target image, obtaining a set of candidate feature maps, and then uses U... 2 The multi-sided output fusion layer of the Net++ network performs feature fusion processing on the candidate feature map set to obtain the target segmentation feature map. This improves the extraction efficiency and accuracy of image feature extraction results. Based on the more accurate image feature extraction results, target detection is performed. Using the detected target as the basis for image segmentation improves the accuracy of determining the image segmentation basis, which is beneficial to improving the segmentation efficiency and accuracy of the image segmentation results. This, in turn, improves the accuracy of analysis based on the image segmentation results. Furthermore, by setting an attention module to identify key features in the image, the accuracy of extracting key features can be improved, thereby improving the reliability of determining the image segmentation basis and further improving the accuracy of the image segmentation results.

[0187] In an optional embodiment, when a layer includes only one multi-scale feature extraction module, the multi-scale feature extraction module is used as an encoder; when a layer includes two multi-scale feature extraction modules, the first multi-scale feature extraction module at that layer position is used as an encoder, and the last multi-scale feature extraction module at that layer position is used as a decoder, and a skip path is provided between the encoder and the decoder of that layer; when a layer includes two or more multi-scale feature extraction modules, the remaining multi-scale feature extraction modules of that layer other than the encoder and decoder are used as dense residual modules, and skip paths are provided between the encoder and each dense residual module of that layer, and between each dense residual module of that layer and the decoder of that layer; wherein, the encoder is used to extract multi-scale features of the image input to the encoder; the decoder is used to restore the resolution of the image input to the decoder; and the dense residual module is used to integrate information from deeper layers into the layer in which the dense residual module is located;

[0188] Among them, the more multi-scale feature extraction modules a certain level contains, the shallower the level is, and the shallowest level is the level that contains the most multi-scale feature extraction modules; the fewer multi-scale feature extraction modules a certain level contains, the deeper the level is, and the deepest level is the level that contains only one multi-scale feature extraction module.

[0189] Each level of encoder has a downsampling path between it and the encoder of the adjacent deep level; each multi-scale feature extraction module has an upsampling path between it and the first adjacent module of the adjacent shallow level, wherein the position of the first adjacent module in its level is adjacent to the position of the multi-scale feature extraction module in its level.

[0190] For each attention module, the feature map input to the attention module includes a first input feature map and a second input feature map; wherein, the multi-scale feature extraction module that receives the region of interest output by the attention module is used as the location reference module, and the feature map output by the multi-scale feature extraction module that is the first neighbor of the location reference module is used as the first input feature map; the feature map output by the second neighbor module of the adjacent deep layer of the location reference module is used as the second input feature map, and the position of the second neighbor module in its layer is the first neighbor of the location reference module in its layer.

[0191] As can be seen, the apparatus described in this optional embodiment can simultaneously capture global background information and local detail information by setting up multi-level multi-scale feature extraction modules and using these modules as encoders, decoders, and dense residual modules. It extracts features at multiple scales from continuously downsampled feature maps and then decodes these multi-scale features into high-resolution feature maps through progressive upsampling, merging, and convolution. This improves the comprehensiveness and accuracy of image feature extraction, thereby enhancing the comprehensiveness and reliability of the image feature extraction results. Based on more reliable image feature extraction results, image segmentation is more accurate. Furthermore, by setting nested dense residual modules and skip paths, shallow-level and deep-level features can be fused, effectively reducing gradient vanishing and network degradation problems, and improving U... 2 The Net++ network offers stability while preserving features from the original image, improving the integrity and reliability of image feature extraction results and enhancing the accuracy of image segmentation.

[0192] In this optional embodiment, optionally, the multi-scale feature extraction unit 302 inputs the target image into U. 2 The Net++ network's feature extraction layer performs multi-scale feature extraction to obtain a candidate feature map set in the following ways:

[0193] Input the target image into U 2 The Net++ network's feature extraction layer is the shallowest level encoder, and it performs downsampling operations on the target image based on the downsampling path until the deepest level's adjacent shallow level, thus obtaining a multi-scale downsampled feature map.

[0194] Based on the deepest level encoder, dilated convolution is performed on the multi-scale downsampled feature maps output by the adjacent shallow level encoders to obtain dilated feature maps.

[0195] For each multi-scale feature extraction module other than all encoders, based on the received region of interest, all feature maps input to that multi-scale feature extraction module are upsampled to obtain a multi-scale upsampled feature map.

[0196] All multi-scale upsampled feature maps in the shallowest layer are identified as the first candidate feature map subset;

[0197] The dilated feature map and the multi-scale upsampled feature map output by the decoder at each level except the shallowest level are determined as the second candidate feature map subset.

[0198] As can be seen, the apparatus described in this optional embodiment can also perform downsampling operations on the target image based on the downsampling path to obtain multi-scale downsampling feature maps, and perform dilated convolution processing on adjacent multi-scale downsampling feature maps based on the deepest level encoder to obtain dilated feature maps. For each multi-scale feature extraction module other than all encoders, upsampling operations are performed on all feature maps input to the multi-scale feature extraction module based on the received region of interest to obtain multi-scale upsampling feature maps. Then, all multi-scale upsampling feature maps in the shallowest level are determined as a first candidate feature map subset, and the dilated feature maps and the multi-scale upsampling feature maps output by the decoders of each level other than the shallowest level are determined as a second candidate feature map subset. This can improve the accuracy and comprehensiveness of candidate feature map determination, thereby improving the accuracy of image feature fusion results, and further improving the segmentation accuracy and reliability of image segmentation results.

[0199] In this optional embodiment, when the original image is an X-ray tooth image, the target segmentation feature map is a segmentation feature map with teeth as the segmentation target; the type of the multi-scale feature extraction module includes a first feature extraction type and a second feature extraction type.

[0200] The multi-scale feature extraction module includes multiple sub-levels, each of which includes at least one multi-scale feature extraction sub-module. When a sub-level includes a multi-scale feature extraction sub-module for receiving the input feature map, this sub-level is the shallowest sub-level. The multi-scale feature extraction sub-module for receiving the input feature map is an input convolution sub-module. The shallowest sub-level also includes a fusion sub-module, which is a multi-scale feature extraction sub-module used to fuse features and output a fused feature map. When a sub-level includes only one multi-scale feature extraction sub-module, this sub-level is the deepest sub-level.

[0201] When a sub-level includes at least two multi-scale feature extraction sub-modules, an attention sub-module is set between each pair of adjacent multi-scale feature extraction sub-modules in the level, and a skip sub-path is set between each multi-scale feature extraction sub-module in the level; for each attention sub-module, the attention sub-module receives the feature map output by the preceding adjacent multi-scale feature extraction sub-module and the feature map output by the preceding multi-scale feature extraction sub-module of the adjacent deep sub-level, and the subsequent adjacent multi-scale feature extraction sub-module of the attention sub-module receives the region of interest output by the attention sub-module;

[0202] A first feature extraction sub-path is set between the first multi-scale feature extraction sub-module at each sub-level and the first multi-scale feature extraction sub-module at the adjacent deep level; a second feature extraction sub-path is set between the last multi-scale feature extraction sub-module at each sub-level and the last multi-scale feature extraction sub-module at the adjacent shallow level.

[0203] As can be seen, the apparatus described in this optional embodiment can also increase the depth of the entire network by setting multiple sub-level multi-scale feature extraction sub-modules in the multi-scale feature extraction module, and capture more contextual information from different scales by fusing receptive fields of different scales, thereby improving the expansion of the receptive field range, which is conducive to extracting more and richer image features, and thus improving the comprehensiveness of the image feature extraction results.

[0204] In this optional embodiment, when the type of the multi-scale feature extraction module is the first feature extraction type, the first feature extraction sub-path is a downsampling sub-path, and the multi-scale feature extraction sub-modules on the downsampling path, except for the shallowest sub-layer, are convolutional sub-modules or downsampling sub-modules; the first feature extraction sub-path is an upsampling sub-path, and the multi-scale feature extraction sub-modules on the upsampling path, except for the shallowest sub-layer, are convolutional sub-modules or upsampling sub-modules; the remaining multi-scale feature extraction sub-modules in each sub-layer, except for the upsampling sub-modules and downsampling sub-modules, are dense convolutional sub-modules.

[0205] As can be seen, the apparatus described in this optional embodiment can also set downsampling and upsampling operations in the multi-scale feature extraction module of the first feature extraction type, providing a deeper network structure and more attention mechanisms, which can extract richer image features, thereby further improving the comprehensiveness of the image feature extraction results and helping to further improve the accuracy of the image segmentation results; and, by setting multi-scale feature extraction modules of different depths according to the size of the feature map, different scales of feature information corresponding to different image sizes can be obtained, which can improve the flexibility of image feature extraction, thereby improving the accuracy of image feature extraction results.

[0206] In this optional embodiment, when the type of the multi-scale feature extraction module is the second feature extraction type, in the multi-scale feature extraction module, the multi-scale feature extraction sub-module in each sub-level except the shallowest sub-level is a dilated convolution sub-module.

[0207] Among them, the deepest level encoder is of the second feature extraction type.

[0208] As can be seen, the apparatus described in this optional embodiment can also expand the receptive field while maintaining the resolution of the input feature map by setting a multi-scale feature extraction module of the second feature extraction type. This can improve the stability of the segmentation network while retaining more image detail information and capturing a wider range of contextual information, thereby improving the reliability of the image feature extraction results and improving the reliability of the image segmentation results.

[0209] In this optional embodiment, the multi-side output fusion unit 303 optionally inputs the candidate feature map set into U. 2 The specific methods for obtaining target segmentation feature maps by performing feature fusion processing on the multi-sided output fusion layer of the Net++ network include:

[0210] Based on a preset activation function, feature fusion processing is performed on all candidate feature maps in the first candidate feature map subset to obtain the first segmentation feature map; wherein, the calculation formula of the first segmentation feature map is as follows:

[0211]

[0212] In the formula, Indicates a join operation, Y 0,5 Y is the first segmentation feature map. 0,1 Y 0,2 Y 0,3 and Y 0,4 All of them are feature maps from the first candidate feature map subset;

[0213] Based on a preset activation function, feature fusion processing is performed on all candidate feature maps in the second candidate feature map subset to obtain the second segmentation feature map; the calculation formula for the second segmentation feature map is as follows:

[0214]

[0215] In the formula, Y 5,0 For the second segmentation feature map, Y 1,0 Y 2,0 Y 3,0 and Y 4,0 All of them are feature maps from the second candidate feature map subset;

[0216] Based on a preset activation function, the first segmentation feature map and the second segmentation feature map are fused to obtain the target segmentation feature map; the calculation formula for the target segmentation feature map is as follows:

[0217]

[0218] In the formula, Y 5,5 Segment the feature map for the target.

[0219] As can be seen, the apparatus described in this optional embodiment can also perform fusion processing on all candidate feature maps in the first candidate feature map subset and the second candidate feature map subset respectively based on a preset activation function to obtain a first segmentation feature map and a second segmentation feature map respectively. Then, the first segmentation feature map and the second segmentation feature map are fused to obtain the target segmentation feature map. This helps to enrich the number of image feature fusion criteria, improve the reliability of image feature fusion results, thereby improving the accuracy of determining the image segmentation criteria, and thus improving the accuracy of the image segmentation results. At the same time, it can also improve U 2 Improve the convergence of Net++ networks to enhance U 2 The stability of the Net++ network helps improve the reliability of image segmentation results.

[0220] Example 4

[0221] Please see Figure 11 , Figure 11 This is a schematic diagram of another image segmentation device based on multi-scale feature extraction disclosed in an embodiment of the present invention. For example... Figure 11 As shown, the image segmentation device based on multi-scale feature extraction may include:

[0222] Memory 401 storing executable program code;

[0223] Processor 402 coupled to memory 401;

[0224] The processor 402 calls the executable program code stored in the memory 401 to execute the steps in the image segmentation method based on multi-scale feature extraction described in Embodiment 1 or Embodiment 2 of the present invention.

[0225] Example 5

[0226] This invention discloses a computer storage medium storing computer instructions. When these computer instructions are invoked, they are used to execute the steps in the image segmentation method based on multi-scale feature extraction described in Embodiment 1 or Embodiment 2 of this invention.

[0227] Example 6

[0228] This invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform the steps in the image segmentation method based on multi-scale feature extraction described in Embodiment 1 or Embodiment 2.

[0229] The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0230] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0231] Finally, it should be noted that the image segmentation method and apparatus based on multi-scale feature extraction disclosed in the embodiments of the present invention are merely preferred embodiments of the present invention and are only used to illustrate the technical solutions of the present invention, not to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image segmentation method based on multi-scale feature extraction, characterized in that, The method includes: The original image is input into the pre-trained U... 2 The data input layer of the Net++ network performs image size reduction processing to obtain a target image, the image size of which is the target size, and the data input layer is used to reduce the image size of the original image to the target size; The target image is input into the U 2 The feature extraction layer of the Net++ network performs multi-scale feature extraction to obtain a candidate feature map set. The feature extraction layer includes multiple levels, each level including at least one multi-scale feature extraction module. When a level includes at least two multi-scale feature extraction modules, an attention module is provided between each pair of adjacent multi-scale feature extraction modules in that level. Each attention module is used to identify the region of interest (ROI) of the feature map input to it and input the ROI to the next adjacent multi-scale feature extraction module. The candidate feature map set includes at least two candidate feature map subsets, and the feature maps contained in each candidate feature map subset are used for feature fusion to obtain a segmentation feature map. The candidate feature map set is input into the U 2 The multi-sided output fusion layer of the Net++ network performs feature fusion processing to obtain the target segmentation feature map.

2. The image segmentation method based on multi-scale feature extraction according to claim 1, characterized in that, When a certain layer includes only one multi-scale feature extraction module, the multi-scale feature extraction module is used as an encoder; when a certain layer includes two multi-scale feature extraction modules, the first multi-scale feature extraction module at that level is used as an encoder, and the last multi-scale feature extraction module at that level is used as a decoder, and a skip path is provided between the encoder and the decoder at that level; when a certain layer includes two or more multi-scale feature extraction modules, the remaining multi-scale feature extraction modules at that level, excluding the encoder and the decoder, are used as dense residual modules, and skip paths are provided between the encoder and each dense residual module at that level, and between each dense residual module and the decoder at that level; wherein, the encoder is used to extract multi-scale features of the image input to the encoder; the decoder is used to restore the resolution of the image input to the decoder; the dense residual module is used to integrate information from deeper levels into the level where the dense residual module is located; Wherein, if a certain level contains more multi-scale feature extraction modules, the level is shallower, and the shallowest level is the level that contains the most multi-scale feature extraction modules; if a certain level contains fewer multi-scale feature extraction modules, the level is deeper, and the deepest level is the level that contains only one multi-scale feature extraction module. A downsampling path is provided between the encoder of each level and the encoder of the adjacent deep level; an upsampling path is provided between each multi-scale feature extraction module and the first adjacent module of the adjacent shallow level, wherein the position of the first adjacent module in its level is adjacent to the position of the multi-scale feature extraction module in its level. For each attention module, the feature map input to the attention module includes a first input feature map and a second input feature map; wherein, the multi-scale feature extraction module that receives the region of interest output by the attention module is used as a location reference module, and the feature map output by the multi-scale feature extraction module that is the first input feature map is used; the feature map output by the second neighboring module of the adjacent deep layer of the location reference module is used as the second input feature map, and the position of the second neighboring module in its layer is the first neighboring module compared to the position of the location reference module in its layer.

3. The image segmentation method based on multi-scale feature extraction according to claim 2, characterized in that, The target image is input to the U 2 The feature extraction layer of the Net++ network performs multi-scale feature extraction to obtain a set of candidate feature maps, including: The target image is input into the U 2 The shallowest level encoder in the feature extraction layer of the Net++ network performs downsampling operations on the target image based on the downsampling path until the adjacent shallow level of the deepest level, to obtain a multi-scale downsampled feature map. Based on the deepest level encoder, dilated convolution processing is performed on the multi-scale downsampled feature maps output by the encoders of the adjacent shallow levels of the deepest level to obtain dilated feature maps. For each of the multi-scale feature extraction modules other than all the encoders, based on the received region of interest, an upsampling operation is performed on all feature maps input to the multi-scale feature extraction module to obtain a multi-scale upsampled feature map; All the multi-scale upsampled feature maps in the shallowest layer are determined as the first candidate feature map subset; The dilated feature map and the multi-scale upsampled feature map output by the decoder of each of the remaining layers except the shallowest layer are determined as the second candidate feature map subset.

4. The image segmentation method based on multi-scale feature extraction according to claim 2 or 3, characterized in that, When the original image is an X-ray image of teeth, the target segmentation feature map is a segmentation feature map with teeth as the segmentation target; the multi-scale feature extraction module includes a first feature extraction type and a second feature extraction type; The multi-scale feature extraction module includes multiple sub-levels, and each sub-level includes at least one multi-scale feature extraction sub-module. When a sub-level includes a multi-scale feature extraction sub-module for receiving an input feature map, the sub-level is the shallowest sub-level. The multi-scale feature extraction sub-module for receiving the input feature map is an input convolution sub-module. The shallowest sub-level also includes a fusion sub-module, which is a multi-scale feature extraction sub-module used to fuse features and output a fused feature map. When a certain sub-level includes only one multi-scale feature extraction sub-module, the sub-level is the deepest sub-level; When a certain sub-level includes at least two multi-scale feature extraction sub-modules, an attention sub-module is provided between each pair of adjacent multi-scale feature extraction sub-modules in the level, and a skip sub-path is provided between each multi-scale feature extraction sub-module in the level; for each attention sub-module, the attention sub-module receives the feature map output by the preceding adjacent multi-scale feature extraction sub-module and the feature map output by the preceding multi-scale feature extraction sub-module in the adjacent deep sub-level, and the subsequent adjacent multi-scale feature extraction sub-module of the attention sub-module receives the region of interest output by the attention sub-module; A first feature extraction sub-path is provided between the first multi-scale feature extraction sub-module at each sub-level and the first multi-scale feature extraction sub-module at the adjacent deep level; a second feature extraction sub-path is provided between the last multi-scale feature extraction sub-module at each sub-level and the last multi-scale feature extraction sub-module at the adjacent shallow level.

5. The image segmentation method based on multi-scale feature extraction according to claim 4, characterized in that, When the type of the multi-scale feature extraction module is the first feature extraction type, the first feature extraction sub-path is a downsampling sub-path, and the multi-scale feature extraction sub-modules on the downsampling path, except for the shallowest sub-layer, are convolutional sub-modules or downsampling sub-modules; when the first feature extraction sub-path is an upsampling sub-path, and the multi-scale feature extraction sub-modules on the upsampling path, except for the shallowest sub-layer, are convolutional sub-modules or upsampling sub-modules; the remaining multi-scale feature extraction sub-modules in each sub-layer, except for the upsampling sub-module and the downsampling sub-module, are dense convolutional sub-modules.

6. The image segmentation method based on multi-scale feature extraction according to claim 4, characterized in that, When the type of the multi-scale feature extraction module is the second feature extraction type, in the multi-scale feature extraction module, the multi-scale feature extraction sub-module in each of the sub-levels other than the shallowest sub-level is a dilated convolution sub-module; The deepest level encoder is of the second feature extraction type.

7. The image segmentation method based on multi-scale feature extraction according to claim 3, characterized in that, The candidate feature map set is input into the U 2 The multi-sided output fusion layer of the Net++ network performs feature fusion processing to obtain the target segmentation feature map, including: Based on a preset activation function, feature fusion processing is performed on all candidate feature maps in the first subset of candidate feature maps to obtain a first segmentation feature map; wherein, the calculation formula for the first segmentation feature map is as follows: In the formula, ⊕ represents the join operation, Y 0,5 For the first segmentation feature map, Y 0,1 Y 0,2 Y 0,3 and Y 0,4 All of them are feature maps from the first subset of candidate feature maps; Based on the preset activation function, feature fusion processing is performed on all candidate feature maps in the second candidate feature map subset to obtain a second segmentation feature map; wherein, the calculation formula of the second segmentation feature map is as follows: In the formula, Y 5,0 Y is the second segmentation feature map. 1,0 Y 2,0 Y 3,0 and Y 4,0 All of them are feature maps from the second subset of candidate feature maps; Based on the preset activation function, feature fusion processing is performed on the first segmentation feature map and the second segmentation feature map to obtain the target segmentation feature map; wherein, the calculation formula of the target segmentation feature map is as follows: In the formula, Y 5,5 This is the segmentation feature map of the target.

8. An image segmentation device based on multi-scale feature extraction, characterized in that, The device includes: Image resizing unit, used to input the original image into a pre-trained U... 2 The data input layer of the Net++ network performs image size reduction processing to obtain a target image, the image size of which is the target size, and the data input layer is used to reduce the image size of the original image to the target size; A multi-scale feature extraction unit is used to input the target image into the U. 2 The feature extraction layer of the Net++ network performs multi-scale feature extraction to obtain a candidate feature map set. The feature extraction layer includes multiple levels, each level including at least one multi-scale feature extraction module. When a level includes at least two multi-scale feature extraction modules, an attention module is provided between each pair of adjacent multi-scale feature extraction modules in that level. Each attention module is used to identify the region of interest (ROI) of the feature map input to it and input the ROI to the next adjacent multi-scale feature extraction module. The candidate feature map set includes at least two candidate feature map subsets, and the feature maps contained in each candidate feature map subset are used for feature fusion to obtain a segmentation feature map. A multi-sided output fusion unit is used to input the candidate feature map set into the U. 2 The multi-sided output fusion layer of the Net++ network performs feature fusion processing to obtain the target segmentation feature map.

9. An image segmentation device based on multi-scale feature extraction, characterized in that, The device includes: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the image segmentation method based on multi-scale feature extraction as described in any one of claims 1-7.

10. A computer storage medium, characterized in that, The computer storage medium stores computer instructions, which, when invoked, are used to execute the image segmentation method based on multi-scale feature extraction as described in any one of claims 1-7.

Images