Image segmentation method and apparatus, and electronic device and storage medium

Abstract

An image segmentation method and apparatus, an electronic device, and a storage medium. The method comprises: acquiring an image to be segmented, and loading an image segmentation network model matching said image, wherein the image segmentation network model comprises a backbone network, a plurality of full-scale connection modules and a parallel residual network module, the backbone network comprises an encoder and a decoder, the encoder and the decoder are in a skip connection by means of the plurality of full-scale connection modules so as to perform feature fusion, the parallel residual network module comprises a plurality of levels of residual blocks sequentially connected, and each residual block in the parallel residual network module is connected to a corresponding decoder layer in the decoder (S110); inputting said image into the image segmentation network model to perform image segmentation, so as to obtain a first segmentation mask and a second segmentation mask (S120); and on the basis of the first segmentation mask and the second segmentation mask, determining a segmentation result of said image (S130).

Description

Image segmentation method and device, electronic device, and storage medium

[0001] The present application claims priority to the Chinese patent application No. 202410882275.6, filed on July 3, 2024, with the Chinese Patent Office, the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of image segmentation, for example, to an image segmentation method and device, an electronic device, and a storage medium. BACKGROUND

[0003] The main goal of image segmentation technology is to accurately and meaningfully divide the image content, that is, to divide the image into multiple non-overlapping regions, each of which represents a different object or a specific background part in the image. The progress of image segmentation technology has greatly promoted the understanding of image content and subsequent image analysis operations, including object detection, tracking, behavior recognition, and three-dimensional modeling, which have important application prospects in the fields of industry, military, medicine, and public security.

[0004] With the development of X-ray (XRAY) technology, there are more and more processing tasks for industrial X-ray images, such as defect detection tasks for precision products such as semiconductors, integrated circuit (ic) elements, castings, and new energy batteries, such as virtual welding, bubbles, and cracks, or the need to detect and identify the position of chips on a circuit board, measure the distance, etc.

[0005] In the industrial X-ray non-destructive testing task, due to the irregular geometry of the laminated battery, and the small and uneven distribution of the positive and negative electrode tips of the battery, the traditional edge detection and threshold segmentation method has poor effect, and it is often difficult to accurately segment the positive and negative electrode points and other regions. SUMMARY

[0006] The present application provides an image segmentation method, device, electronic device, and storage medium to improve the image segmentation quality.

[0007] The present application provides an image segmentation method, device, electronic device, and storage medium to improve the image segmentation quality.

[0008] An image to be segmented is acquired, and an image segmentation network model matched with the image to be segmented is loaded; the image segmentation network model comprises a backbone network, a plurality of full-scale connection modules, and a parallel residual network module; the backbone network comprises an encoder and a decoder; the encoder and the decoder are connected by a plurality of full-scale connection modules for feature fusion; the parallel residual network module comprises a plurality of hierarchical residual blocks connected in sequence; each residual block in the parallel residual network module is connected with a corresponding decoder layer in the decoder;

[0009] The image to be segmented is input into the image segmentation network model for image segmentation, to obtain a first segmentation mask and a second segmentation mask; the first segmentation mask is output by the decoder, and the second segmentation mask is output by the parallel residual network module;

[0010] A segmentation result of the image to be segmented is determined based on the first segmentation mask and the second segmentation mask.

[0011] Embodiments of the present application provide an image segmentation device, comprising:

[0012] An image segmentation network model loading module is configured to acquire an image to be segmented, and load an image segmentation network model matched with the image to be segmented; the image segmentation network model comprises a backbone network, a plurality of full-scale connection modules, and a parallel residual network module; the backbone network comprises an encoder and a decoder; the encoder and the decoder are connected by a plurality of full-scale connection modules for feature fusion; the parallel residual network module comprises a plurality of hierarchical residual blocks connected in sequence; each residual block in the parallel residual network module is connected with a corresponding decoder layer in the decoder;

[0013] An image segmentation module is configured to input the image to be segmented into the image segmentation network model for image segmentation, to obtain a first segmentation mask and a second segmentation mask; the first segmentation mask is output by the decoder, and the second segmentation mask is output by the parallel residual network module;

[0014] An image segmentation result determination module is configured to determine a segmentation result of the image to be segmented based on the first segmentation mask and the second segmentation mask.

[0015] Embodiments of the present application provide an electronic device, comprising:

[0016] at least one processor; and

[0017] a memory in communication with the at least one processor; wherein

[0018] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the image segmentation method described in any of the embodiments of the present application.

[0019] The embodiment of the present application provides a computer readable storage medium, the computer readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to perform the image segmentation method described in any of the embodiments of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0020] Fig. 1 is a flow chart of an image segmentation method provided by an embodiment of the present application;

[0021] Fig. 2 is a structural schematic diagram of an image segmentation network model provided by the embodiment of the present application;

[0022] Fig. 3 is a structural schematic diagram of a full-scale connection module provided by the embodiment of the present application;

[0023] Fig. 4 is a structural schematic diagram of a residual block provided by the embodiment of the present application;

[0024] Fig. 5 is a flow chart of a training method of an image segmentation network model provided by the embodiment of the present application;

[0025] Fig. 6 is a structural schematic diagram of an image segmentation device provided by the embodiment of the present application;

[0026] Fig. 7 is a structural schematic diagram of an electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor should be within the scope of protection of the present application.

[0028] It should be noted that the terms "first", "second", and the like in the description and in the claims of the present application and the above-described accompanying drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] Embodiment one

[0030] FIG. 1 is a flowchart of an image segmentation method provided by an embodiment of the present application. The present embodiment can be applied to the case of image segmentation of an industrial X-ray laminated battery image. The method can be executed by an image segmentation device, which can be realized in the form of hardware and / or software, and can be configured in a server or a personal computer or other electronic device for image segmentation. As shown in FIG. 1, the method comprises:

[0031] S110, obtaining an image to be segmented, and loading an image segmentation network model matched with the image to be segmented; the image segmentation network model comprises a backbone network, a plurality of full-scale connection modules and a parallel residual network module, the backbone network comprises an encoder and a decoder; the encoder and the decoder are connected by a plurality of full-scale connection modules for feature fusion; the parallel residual network module comprises a plurality of hierarchical residual blocks connected in sequence; each residual block in the parallel residual network module is connected with a corresponding decoder layer in the decoder.

[0032] The image to be segmented refers to an image to be segmented, which includes but is not limited to an X-ray laminated battery image, which is not limited herein. In the present embodiment, the image to be segmented is obtained, and an image segmentation network model matched with the image to be segmented is loaded. The image segmentation network model is a convolutional neural network model trained from the same type of image as the image to be segmented.

[0033] The image segmentation network model comprises a backbone network, a plurality of full-scale connection modules, and a parallel residual network module; FIG. 2 is a structural schematic diagram of an image segmentation network model provided by an embodiment of the present application, as shown in FIG. 2, the backbone network is a convolutional neural network with a U-shaped structure, and the backbone network comprises an encoder and a decoder; the encoder and the decoder are connected through a plurality of full-scale connection modules for feature fusion; the parallel residual network module comprises a plurality of hierarchical residual blocks connected in sequence, and each residual block in the parallel residual network module is connected with a corresponding decoder layer in the decoder. Exemplarily, the convolution kernel size of the backbone network is 3x3.

[0034] On the basis of the above-mentioned embodiments, optionally, any full-scale connection module is connected with at least part of the encoder layers in the encoder and connected with a corresponding decoder layer in the decoder; wherein the i-th full-scale connection module is connected with i+n encoder layers in the encoder and connected with the i+m decoder layer in the encoder, wherein i is an integer greater than or equal to 1; the full-scale connection module is used for performing scale transformation on the feature maps of different scales output by at least part of the encoder layers connected with the full-scale connection module, to obtain feature maps of a uniform scale, performing feature fusion on the feature maps of the uniform scale based on a feature fusion mechanism, generating new feature maps, and inputting the new feature maps into the corresponding decoder layer of the full-scale connection module.

[0035] As shown in FIG. 2, any full-scale connection module is connected with at least part of the encoder layers in the encoder and connected with a corresponding decoder layer in the decoder; wherein the full-scale connection module 1 is connected with the encoder layer 1, the encoder layer 2, and the encoder layer 3, and connected with the decoder layer 2; the full-scale connection module 2 is connected with the encoder layer 1, the encoder layer 2, the encoder layer 3, and the encoder layer 4, and connected with the decoder layer 3; the full-scale connection module 3 is connected with the encoder layer 1, the encoder layer 2, the encoder layer 3, the encoder layer 4, and the encoder layer 5, and connected with the decoder layer 4. As shown in FIG. 2, in the connection relationship between the full-scale connection module and the decoder layer and the encoder in the image segmentation network model, n is 2 and m is 1. It should be noted that n and m are set by a person skilled in the art according to requirements, which are not limited here.

[0036] In this embodiment, the full-scale connection module performs size transformation on the feature maps of different scales output by at least part of the encoder layers connected with the full-scale connection module, to obtain feature maps of a uniform size. The feature maps of different scales are reduced through maximum pooling or enlarged through bilinear interpolation, so as to adjust the feature maps to the same size as the input of the decoder layer. As shown in FIG. 2, the input of the full-scale connection module 1 is the feature maps XE1 , X E2 and X E3 , in order to integrate multiple hierarchical features and realize cross-level information interaction, it is necessary to adjust the sizes of the above three feature maps of different scales to be consistent, X E1 needs to be reduced by 2 times, X E3 needs to be enlarged by 2 times, so that all feature maps have the same size; the input of the full-scale connection module 2 is composed of feature maps X E1 , X E2 , X E3 and X E4 , X E1 needs to be reduced by 4 times, X E2 needs to be reduced by 2 times, X E4 needs to be enlarged by 2 times, so that all feature maps have the same size; the input of the full-scale connection module 3 is composed of feature maps X E1 , X E2 , X E3 , X E4 and X E5 , X E1 needs to be reduced by 8 times, X E2 needs to be reduced by 4 times, X E3 needs to be reduced by 2 times, X E5 needs to be enlarged by 2 times, so that all feature maps have the same size. X E1 , X E2 , X E3 , X E4 and X E5 are feature maps output by the encoder layer 1, the encoder layer 2, the encoder layer 3, the encoder layer 4 and the encoder layer 5 respectively.

[0037] For example, FIG. 3 is a structural schematic diagram of a full-scale connection module provided by the embodiment one of the present application, as shown in FIG. 3, X E1 , X E2 , X E3 , X E4 and X E5 feature maps of different scales are transformed into feature maps of uniform scale.

[0038] Based on the feature fusion mechanism, the feature fusion is performed on the feature maps of uniform scale to generate new feature maps, and the new feature maps are input into the corresponding decoder layers of the full-scale connection module.

[0039] The scale of the feature map after the full-scale connection module unifies the scale is the same as the size of the input feature map of the corresponding decoder layer of the full-scale connection module.

[0040] The full-scale connection module is applied in the image segmentation network model, high-level semantic information and low-level semantic information are combined through a jump connection mode, features from different network layers can be effectively combined, the recognition ability of a target region and edge details is enhanced, the accuracy of a segmentation result is improved, and the network parameter quantity is reduced to improve the calculation efficiency.

[0041] On the basis of the above embodiment, optionally, the feature fusion mechanism includes a plurality of 3x3 convolution kernels, a 3x3 convolution layer and an activation function. The feature maps of the same scale are subjected to element-by-element convolution operation through a plurality of 3x3 convolution kernels, the features of different network depths are fused, and a new feature map is generated by using a 3x3 convolution layer and an activation function. Exemplarily, the number of 3x3 convolution kernels can be 64, and the number is set by a person skilled in the art according to requirements, which is not limited here; the activation function includes but is not limited to ReLU (Rectified Linear Unit) function, Leaky ReLU, Sigmoid function, etc., which is not limited here.

[0042] In some embodiments, optionally, the full-scale connection module is further used for channel rearrangement operation on the feature maps after the scale transformation. As shown in FIG. 3, after each feature map is converted into a feature map of a uniform scale through scale transformation, the channel rearrangement operation is performed on the feature maps after the scale transformation, the channel order of the feature map is disturbed, the feature interaction capability is increased, the model can extract more rich global context information without significantly increasing the calculation cost, and thus the accuracy of image segmentation is improved.

[0043] On the basis of the above embodiment, optionally, each residual block includes a plurality of dilated convolution branches, a residual connection branch and a merging layer; the residual block is used for fusing the feature map output by the decoder layer of the current level and the feature map output by the residual block of the next level to obtain a to-be-processed feature map, inputting the to-be-processed feature map into the plurality of dilated convolution branches and the residual connection branch, merging the feature maps output by the plurality of dilated convolution branches based on the merging layer, and performing element-by-element addition on the feature maps output by the plurality of dilated convolution branches and the residual connection branch to obtain a target feature map, and outputting the target feature map to the residual block of the previous level.

[0044] The to-be-processed feature map is obtained by element-wise adding the feature map output by the decoder layer of the current level and the feature map output by the residual block of the next level, and the to-be-processed feature map is obtained by element-wise adding the feature map output by the decoder layer of the current level and the feature map output by the residual block of the next level; it should be noted that if the residual block has no residual block of the next level, the to-be-processed feature map is the feature map output by the decoder layer of the current level; as shown in FIG. 2, the residual block 4 has no residual block of the next level, and the input of the residual block 4 is only the feature map output by the decoder layer of the current level, so the to-be-processed feature map of the residual block 4 is the feature map output by the decoder layer of the current level, that is, the feature map output by the decoder layer 4.

[0045] In this embodiment, the to-be-processed feature map is input into the plurality of hollow convolution branches and the residual connection branch, the feature maps output by the plurality of hollow convolution branches are merged based on the merging layer, and the feature map output by the residual connection branch is element-wise added to obtain a target feature map, and the target feature map is output to the residual block of the previous level. The target feature map is the feature map input by the residual block to the residual block of the previous level.

[0046] For example, FIG. 4 is a structural schematic diagram of a residual block provided by the first embodiment of the present application, as shown in FIG. 4, the to-be-processed feature map passes through three hollow convolution branches respectively, the first branch is composed of 3x3 hollow convolutions with hollow rates of 2 and 3 respectively; the second branch is composed of 3x3 hollow convolution and 1x1 standard convolution with a hollow rate of 3; the third branch is composed of a hollow convolution with a hollow rate of 5, the feature maps output by the above three hollow convolution branches are merged through the Concat layer (that is, the merging layer), and then element-wise added to the feature map output by the residual connection branch to obtain a target feature map, and finally the target feature map is output to the residual block of the previous level.

[0047] In this embodiment, a parallel residual network module is introduced into an image segmentation network model, multi-scale hollow convolution and residual learning mechanism are used, the ability of the network to extract and fuse multi-scale features is effectively improved, and parallel convolution layers with different expansion rates can expand the receptive field of the network and extract features in different spatial ranges.

[0048] S120, input the to-be-segmented image into the image segmentation network model for image segmentation to obtain a first segmentation mask and a second segmentation mask; the first segmentation mask is output by the decoder, and the second segmentation mask is output by the parallel residual network module.

[0049] The first segmentation mask refers to a segmentation mask output by a decoder using a deconvolution operation to realize multi-level up-sampling, and the second segmentation mask refers to a segmentation mask output by a parallel residual network module. Specifically, the segmentation mask can be a segmentation image or a probability map with the same size as the image to be segmented. In this embodiment, the image to be segmented is input into the image segmentation network model to perform image segmentation, to obtain the first segmentation mask output by the decoder and the second segmentation mask output by the parallel residual network module.

[0050] In S130, a segmentation result of the image to be segmented is determined based on the first segmentation mask and the second segmentation mask.

[0051] In this embodiment, the first segmentation mask and the second segmentation mask are combined to obtain a combined mask. Specifically, the first segmentation mask and the second segmentation mask can be combined by a Boolean algorithm to obtain the combined mask. Alternatively, the average value of each corresponding pixel in the first segmentation mask and the second segmentation mask can be calculated to obtain the combined mask. The segmentation region in the image to be segmented is determined according to the combined mask, and the threshold value of the pixel value in the combined mask is processed to obtain the segmentation region in the image to be segmented. The segmentation result of the image to be segmented is determined according to the segmentation region in the image to be segmented. The image segmentation result can be a new image file, or can be directly marked or modified on the image to be segmented.

[0052] The technical scheme of this embodiment introduces a full-scale connection module into the image segmentation network model. Through the full-scale connection module, the model can learn the hierarchical representation at different scales, enhance the recognition ability of the target region and its edge details, improve the image segmentation quality, and also reduce the network parameter quantity to improve the calculation efficiency. In addition, a parallel residual network module is designed in the network structure of the image segmentation network model. The multi-scale hollow convolution and residual learning mechanism are used to effectively improve the ability of the network to extract and fuse multi-scale features, and to expand the effective receptive field area of the model to extract features in different spatial ranges. The image segmentation result is determined based on the first segmentation mask output by the decoder of the image segmentation network model and the second segmentation mask output by the parallel residual network module, to further improve the image segmentation quality.

[0053] Embodiment Two

[0054] FIG. 5 is a flowchart of a training method of an image segmentation network model according to Embodiment Two of the present application. This embodiment provides a training method of an X-ray slice battery image segmentation network model based on the above-mentioned embodiments. The explanations of the same or corresponding terms as in the above-mentioned embodiments are not repeated here.

[0055] As shown in FIG. 5, the method comprises the following steps.

[0056] S510, acquire the X-ray laminated battery image, and form a data set based on the X-ray laminated battery image.

[0057] In this embodiment, the X-ray laminated battery image is acquired, and a data set is formed based on the X-ray laminated battery image. The data set includes a training set and a test set. For example, the division ratio of the training set and the test set is 4:1, which is not limited here.

[0058] S520, bias correction and gray scale normalization are performed on each X-ray laminated battery image in the data set to obtain a standard data set.

[0059] Before model training, image preprocessing operation is required for the images in the data set. Affine transformation technology is used to correct the bias of each X-ray laminated battery image in the data set to maintain the flatness and parallelism of the image and reduce the non-uniformity or noise in the image. Gray scale normalization method is used to normalize each X-ray laminated battery image in the bias-corrected data set to obtain a standard data set. The standard data set refers to a data set in which each X-ray laminated battery image in the data set is converted into a standard form of X-ray laminated battery image with similar contrast and brightness.

[0060] The gray scale normalization method is to divide the gray scale value of the image according to a preset percentage, and set the pixel value exceeding the preset percentile to the maximum gray scale value, and set the pixel value below the preset percentile to the minimum gray scale value, and map the remaining part in proportion, so that the image has a uniform gray scale range and better contrast, which is convenient for subsequent image analysis and feature extraction. It can be understood that the preset percentage and the preset percentile are set by the person skilled in the art according to the demand, which is not limited here.

[0061] In some embodiments, the images in the labeled standard data set can be optionally standardized and normalized. The standardization and normalization method is:

[0062] Wherein, μ is the mean of the picture tensor, X represents the picture tensor, σ represents the standard deviation of the picture tensor, max represents the maximum value of the picture tensor, min represents the minimum value of the picture tensor, X1 represents the standardized picture tensor, x 0 represents the normalized picture tensor.

[0063] S530, a convolutional neural network model is established, and the convolutional neural network model is iteratively trained based on the labeled standard data set to obtain an image segmentation network model of the X-ray laminated battery image.

[0064] The established convolutional neural network model comprises a backbone network, a plurality of full-scale connection modules and a parallel residual network module, the backbone network comprises an encoder and a decoder; the encoder and the decoder are connected through a plurality of full-scale connection modules for feature fusion; the parallel residual network module comprises a plurality of hierarchical residual blocks connected in sequence; each residual block in the parallel residual network module is connected with a corresponding decoder layer in the decoder. Each convolutional layer in the convolutional neural network model comprises a series of nonlinear transformations, and the nonlinear transformations comprise convolution operation, batch normalization and activation function (ReLU, Rectified Linear Unit).

[0065] The convolution operation formula is as follows:

[0066] Wherein, N is the size of the original image, S is the stride size, and M is the size of the convolution kernel.

[0067] Batch normalization: transform the input image of each layer into a normal distribution with a mean of 0 and a variance of 1 to obtain a standardized result, and the formula is as follows:

[0068] Wherein, x i represents the i-th input image, μ B is the average value of the feature, σ B 2 is the variance of the feature, γ and β are introduced parameters, and ε is a constant added to the minimum batch variance.

[0069] It can be understood that the batch normalization operation can accelerate the convergence speed of the network and greatly accelerate the training speed; the ReLU activation function is used to add a nonlinear factor, which can enhance the learning ability of the model and will not have the problem of gradient disappearance.

[0070] The technical scheme of the embodiment trains a convolutional neural network model introducing a full-scale connection module and a parallel residual network module by using a data set of X-ray laminated battery images to obtain an image segmentation network model of the X-ray laminated battery images, wherein a full-scale connection module is introduced in the image segmentation network model, the model can learn hierarchical representation at different scales through the full-scale connection module, the recognition capability of a target region and edge details is enhanced, the image segmentation quality is improved, and the network parameter quantity is reduced to improve the calculation efficiency. Since the sizes and shapes of the positive and negative pole points in the X-ray laminated battery images may change significantly in different image volumes, the image segmentation network model of the X-ray laminated battery images can fully utilize the spatial detail information of the low layer and the semantic information of the high layer to accurately segment the X-ray laminated battery images. In addition, a parallel residual network module is designed in the network structure of the image segmentation network model, multi-scale hollow convolution and residual learning mechanism are used to effectively improve the ability of the network to extract and fuse multi-scale features, and the effective receptive field area of the model is expanded to extract features in different spatial ranges, further improving the image segmentation quality.

[0071] Embodiment three

[0072] FIG. 6 is a structural schematic diagram of an image segmentation device provided by embodiment three of the present application. As shown in FIG. 6, the device comprises:

[0073] An image segmentation network model loading module 610 is configured to obtain a to-be-segmented image and load an image segmentation network model matched with the to-be-segmented image; the image segmentation network model comprises a backbone network, a plurality of full-scale connection modules and a parallel residual network module, the backbone network comprises an encoder and a decoder; the encoder and the decoder are connected by a plurality of full-scale connection modules for feature fusion; the parallel residual network module comprises a plurality of hierarchical residual blocks connected in sequence; each residual block in the parallel residual network module is connected with a corresponding decoder layer in the decoder;

[0074] An image segmentation module 620 is configured to input the to-be-segmented image into the image segmentation network model for image segmentation to obtain a first segmentation mask and a second segmentation mask; the first segmentation mask is output by the decoder, and the second segmentation mask is output by the parallel residual network module;

[0075] An image segmentation result determination module 630 is configured to determine a segmentation result of the to-be-segmented image based on the first segmentation mask and the second segmentation mask.

[0076] The technical scheme of the embodiment introduces a full-scale connection module into the image segmentation network model, and through the full-scale connection module, the model can learn hierarchical representation at different scales, enhance the recognition ability of the target region and its edge details, improve the image segmentation quality, and also reduce the network parameter quantity to improve the calculation efficiency. In addition, a parallel residual network module is designed in the network structure of the image segmentation network model, and the multi-scale hollow convolution and the residual learning mechanism are used to effectively improve the ability of the network to extract and fuse multi-scale features, and to expand the effective receptive field area of the model to extract features in different spatial ranges. The first segmentation mask output by the decoder of the image segmentation network model and the second segmentation mask output by the parallel residual network module are used to determine the image segmentation result, thereby further improving the image segmentation quality.

[0077] On the basis of the above embodiment, optionally, any full-scale connection module is connected with at least part of the encoder layers in the encoder and is connected with a corresponding decoder layer in the decoder; wherein the i-th full-scale connection module is connected with i+n encoder layers in the encoder and is connected with the i+m decoder layer in the encoder, wherein i is an integer greater than or equal to 1; the full-scale connection module is used for performing scale transformation on the feature maps of different scales output by at least part of the encoder layers connected with the full-scale connection module to obtain feature maps of a uniform scale, performing feature fusion on the feature maps of the uniform scale based on a feature fusion mechanism, generating new feature maps, and inputting the new feature maps into the decoder layer corresponding to the full-scale connection module.

[0078] On the basis of the above embodiment, optionally, the feature fusion mechanism includes a plurality of 3x3 convolution kernels, a 3x3 convolution layer and an activation function.

[0079] On the basis of the above embodiment, optionally, the full-scale connection module is also used for performing a channel rearrangement operation on the feature maps after scale transformation.

[0080] On the basis of the above embodiment, optionally, each residual block includes a plurality of hollow convolution branches, a residual connection branch and a merging layer; the residual block is used for fusing the feature maps output by the decoder layer at the current level and the feature maps output by the residual block at the next level to obtain a to-be-processed feature map, inputting the to-be-processed feature map into the plurality of hollow convolution branches and the residual connection branch, merging the feature maps output by the plurality of hollow convolution branches based on the merging layer, and performing element-by-element addition with the feature maps output by the residual connection branch to obtain a target feature map, and outputting the target feature map to the residual block at the previous level.

[0081] On the basis of the above-mentioned embodiment, optionally, the image to be segmented includes an X-ray slice battery image.

[0082] On the basis of the above-mentioned embodiment, optionally, the device further includes a training module of the image segmentation network model, configured to obtain the X-ray slice battery image, and form a data set based on the X-ray slice battery image; perform bias correction and gray scale normalization on each X-ray slice battery image in the data set to obtain a standard data set; establish a convolutional neural network model, and perform iterative training on the convolutional neural network model based on the labeled standard data set to obtain the image segmentation network model.

[0083] The image segmentation device provided in the embodiments of the present application can execute the image segmentation method provided in any of the embodiments of the present application, and has the function modules corresponding to the execution method.

[0084] Embodiment four

[0085] FIG. 7 is a structural schematic diagram of an electronic device provided in Embodiment Four of the present application. The electronic device 10 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present application described and / or claimed in this document.

[0086] As shown in FIG. 7, the electronic device 10 includes at least one processor 11, and a memory, such as a Read-Only Memory (ROM) 12, a Random Access Memory (RAM) 13, etc., which are in communication with the at least one processor 11, wherein the memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the Read-Only Memory (ROM) 12 or loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An Input / Output (I / O) interface 15 is also connected to the bus 14.

[0087] A plurality of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, a speaker, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0088] The processor 11 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various special-purpose Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, a Digital Signal Processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as the image segmentation method.

[0089] In some embodiments, the image segmentation method can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded onto the RAM 13 and executed by the processor 11, one or more steps of the image segmentation method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the image segmentation method by any other appropriate means, such as by means of firmware.

[0090] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a Field-Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), a Complex Programmable Logic Device (CPLD), a System on Chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0091] Computer programs implementing the image segmentation method of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program running on the processor implements the functions / operations specified in the flow diagrams and / or block diagrams. The computer program can execute entirely on a machine, partly on a machine, partly on a machine as a stand-alone software package, partly on a machine and partly on a remote machine or entirely on a remote machine or server.

[0092] Embodiment five

[0093] The embodiment five of the present application also provides a computer readable storage medium, which stores computer instructions for causing a processor to execute an image segmentation method, the method comprising:

[0094] An image to be segmented is obtained, and an image segmentation network model matched with the image to be segmented is loaded; the image segmentation network model comprises a backbone network, a plurality of full-scale connection modules, and a parallel residual network module; the backbone network comprises an encoder and a decoder; the encoder and the decoder are connected by a plurality of full-scale connection modules for feature fusion; the parallel residual network module comprises a plurality of hierarchical residual blocks connected in sequence; each residual block in the parallel residual network module is connected with a corresponding decoder layer in the decoder;

[0095] inputting the image to be segmented into the image segmentation network model to perform image segmentation, to obtain a first segmentation mask and a second segmentation mask; the first segmentation mask is output by the decoder, and the second segmentation mask is output by the parallel residual network module;

[0096] determining a segmentation result of the image to be segmented based on the first segmentation mask and the second segmentation mask.

[0097] In the context of the present application, the computer-readable storage medium can be a tangible medium that can contain or store the computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. Examples of the machine-readable storage medium can include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM) or a flash memory, an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0098] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a Cathode Ray Tube (CRT) or a Liquid Crystal Display (LCD) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0099] The systems and techniques described herein can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described herein), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), blockchain networks, and the Internet.

[0100] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and virtual private server (VPS) services.

[0101] It should be understood that the steps shown in the above forms can be reordered, added, or deleted. For example, the steps described in the present application can be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions of the present application can be achieved, and the present application is not limited herein.

Claims

1. An image segmentation method, comprising: obtaining an image to be segmented, and loading an image segmentation network model matched with the image to be segmented; the image segmentation network model comprises a backbone network, a plurality of full-scale connection modules and a parallel residual network module, the backbone network comprises an encoder and a decoder; the encoder and the decoder are connected by a plurality of full-scale connection modules for feature fusion; the parallel residual network module comprises a plurality of hierarchical residual blocks connected in sequence; each residual block in the parallel residual network module is connected with a corresponding decoder layer in the decoder; inputting the image to be segmented into the image segmentation network model for image segmentation to obtain a first segmentation mask and a second segmentation mask; the first segmentation mask is output by the decoder, and the second segmentation mask is output by the parallel residual network module; determining a segmentation result of the image to be segmented based on the first segmentation mask and the second segmentation mask.

2. The method of claim 1, wherein, any full-scale connection module is connected with at least part of the encoder layers in the encoder and with a corresponding decoder layer in the decoder; wherein the i-th full-scale connection module is connected with i+n encoder layers in the encoder and with the i+m decoder layer in the encoder, wherein i is an integer greater than or equal to 1; the full-scale connection module is used for performing scale transformation on feature maps of different scales output by at least part of the encoder layers connected with the full-scale connection module to obtain feature maps of a uniform scale, performing feature fusion on the feature maps of the uniform scale based on a feature fusion mechanism to generate new feature maps, and inputting the new feature maps into the corresponding decoder layer of the full-scale connection module.

3. The method of claim 2, wherein, the feature fusion mechanism comprises a plurality of 3x3 convolution kernels, a 3x3 convolution layer and an activation function.

4. The method of claim 2, wherein, each full-scale connection module is also used for performing channel rearrangement operation on the feature maps after scale transformation.

5. The method of claim 1, wherein, each residual block comprises a plurality of hollow convolution branches, a residual connection branch and a merging layer; in the case that the residual block is not the last hierarchical residual block in the plurality of hierarchical residual blocks, the residual block is used for fusing feature maps output by a decoder layer of a current hierarchical level where the residual block is located and feature maps output by a residual block of a next hierarchical level of the current hierarchical level to obtain a to-be-processed feature map, inputting the to-be-processed feature map into the plurality of hollow convolution branches and the residual connection branch, merging feature maps output by the plurality of hollow convolution branches based on the merging layer, and performing element-wise addition with feature maps output by the residual connection branch to obtain a target feature map, and outputting the target feature map to a residual block of a previous hierarchical level of the current hierarchical level. In a case where the residual block is a residual block of a last level of the multiple levels of residual blocks, a feature map output by a decoder layer of a current level where the residual block is located is a to-be-processed feature map, the to-be-processed feature map is input to the multiple hollow convolution branches and the residual connection branch, feature maps output by the multiple hollow convolution branches are merged based on the merging layer, and the feature maps output by the multiple hollow convolution branches are element-wise added with a feature map output by the residual connection branch to obtain a target feature map, and the target feature map is output to a residual block of a previous level of the current level.

6. The method of claim 1, wherein, The to-be-segmented image includes an X-ray slice cell image.

7. The method of claim 6, wherein, The training method of the image segmentation network model includes: An X-ray slice cell image is acquired, and a data set is formed based on the X-ray slice cell image; Each X-ray slice cell image in the data set is subjected to bias correction and gray scale normalization to obtain a standard data set; A convolutional neural network model is established, and the convolutional neural network model is iteratively trained based on the labeled standard data set to obtain an image segmentation network model of the X-ray slice cell image.

8. An image segmentation device, comprising: An image segmentation network model loading module configured to acquire a to-be-segmented image and load an image segmentation network model matched with the to-be-segmented image; the image segmentation network model comprises a backbone network, multiple full-scale connection modules, and a parallel residual network module; the backbone network comprises an encoder and a decoder; the encoder and the decoder are connected by multiple full-scale connection modules for feature fusion; the parallel residual network module comprises multiple levels of residual blocks connected in sequence; Each residual block in the parallel residual network module is connected with a corresponding decoder layer in the decoder; An image segmentation module configured to input the to-be-segmented image to the image segmentation network model for image segmentation to obtain a first segmentation mask and a second segmentation mask; the first segmentation mask is output by the decoder, and the second segmentation mask is output by the parallel residual network module; An image segmentation result determination module configured to determine a segmentation result of the to-be-segmented image based on the first segmentation mask and the second segmentation mask.

9. An electronic device, comprising: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the image segmentation method of any one of claims 1-7.

10. A computer-readable storage medium storing computer instructions for enabling a processor to execute the image segmentation method of any one of claims 1-7 when executed.

Images