Image segmentation method, device and equipment for detecting industrial defects and storage medium

Abstract

The application provides an image segmentation method, device and equipment and a storage medium for detecting industrial defects. The method comprises the following steps: performing feature extraction on a to-be-detected image by using a convolutional neural network to obtain a first feature map; performing multiple times of down-sampling on the first feature map by using a down-sampling module of an encoder to obtain a second feature map and a basic feature corresponding to the first feature map; performing up-sampling on the second feature map by using an up-sampling module of a decoder to obtain a deep feature corresponding to the second feature map; and fusing the basic feature and the deep feature to obtain an image segmentation result. By using the method, the stability and accuracy of the segmentation image obtained when detecting industrial defects can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of detecting industrial products, in particular, to an image segmentation method, device and equipment for detecting industrial defects and a storage medium. BACKGROUND

[0002] With the development of modern machine learning and artificial intelligence technology, more and more related technologies are applied to industrial scenes, and the quality detection technology of industrial products, especially the detection of scratches and cracks caused by some misoperations, is particularly important.

[0003] Traditional industrial defect detection is often completed by manual operation or by using a neural network to detect product defects. When a neural network is used to detect product defects, the region image of the product defect is placed in the neural network for defect detection.

[0004] However, during the placement process, the relative translation of the defect caused by the translation of the image leads to unstable and inaccurate segmentation results when detecting industrial defects.

[0005] Therefore, how to improve the stability and accuracy of the segmentation image obtained when detecting industrial defects is a technical problem to be solved. SUMMARY

[0006] The purpose of the embodiments of the present application is to provide an image segmentation method for detecting industrial defects. The technical solutions of the embodiments of the present application can achieve the effect of improving the stability and accuracy of the segmentation image obtained when detecting industrial defects.

[0007] In a first aspect, the embodiments of the present application provide an image segmentation method for detecting industrial defects, comprising: performing feature extraction on a to-be-detected image by using a convolutional neural network to obtain a first feature map; performing multiple times of down-sampling on the first feature map by using a down-sampling module of an encoder to obtain a second feature map and a basic feature corresponding to the first feature map; performing up-sampling on the second feature map by using an up-sampling module of a decoder to obtain a deep feature corresponding to the second feature map; and fusing the basic feature and the deep feature to obtain an image segmentation result.

[0008] In the above embodiments, in the image segmentation process based on deep learning, the down-sampling and up-sampling methods are redesigned, and the image is extracted by using the down-sampling of the encoder and the up-sampling of the decoder. Finally, the final image segmentation result is obtained by using the feature fusion method. This method can avoid the problem that the segmentation result of the target detection object is unstable due to the translation of the input image in the prior art. Therefore, the method of the present application can improve the stability of the segmentation result while ensuring that the segmentation result has high precision, and meet the requirements of industrial defect detection.

[0009] In some embodiments, the first feature map is down-sampled multiple times by a down-sampling module of the encoder to obtain a second feature map and a base feature corresponding to the first feature map, including:

[0010] The first feature map is sampled a preset number of times by a down-sampling module of the encoder to obtain a plurality of feature slices, wherein the plurality of feature slices are of the same size;

[0011] The plurality of feature slices are extracted by a preset convolutional layer to obtain a plurality of feature components, wherein the plurality of feature components and the plurality of feature slices have the same spatial resolution;

[0012] The features of the plurality of feature slices are extracted by the down-sampling module of the encoder to obtain the base feature and the second feature map.

[0013] In the above embodiments, the second feature map and the base feature can be accurately obtained through feature screening by sampling the first feature map and extracting features of the feature components.

[0014] In some embodiments, the preset convolutional layer includes:

[0015] a plurality of convolutional layers;

[0016] The first convolutional layer of the plurality of convolutional layers includes 64 convolutional kernels, and the second convolutional layer includes 32 convolutional kernels;

[0017] Each convolutional layer of the plurality of convolutional layers is followed by a batch normalization layer and a linear rectifier function.

[0018] In the above embodiments, the convolutional layer, the batch normalization layer, and the function can improve the accuracy of feature extraction when extracting features of the feature slices.

[0019] In some embodiments, the features of the plurality of feature slices are extracted by the down-sampling module of the encoder to obtain the base feature and the second feature map, including:

[0020] The plurality of feature components are cropped in the spatial dimension according to a preset proportion coefficient to obtain a plurality of cropped feature components, wherein the cropping operation includes cropping edge features of the plurality of feature components;

[0021] The mean of the plurality of cropped feature components is calculated to obtain the weight of each feature component in the plurality of cropped feature components;

[0022] The second feature map and the base feature are determined according to the weight of each feature component in the plurality of cropped feature components.

[0023] In the above embodiment, the application can avoid the influence of irrelevant features such as image edges on the final segmentation result through the clipping of the feature components, ensure that the network can select the same feature slices in the downsampling process before and after the translation of the input image, increase the translation equivalence of the segmentation network, and make the output segmentation result more stable.

[0024] In some embodiments, the second feature map and the basic feature are determined according to the weight of each feature component in the plurality of clipped feature components, comprising:

[0025] The feature slice corresponding to the maximum weight of each feature component in the plurality of clipped feature components is selected as the second feature map, or the plurality of feature slices are weighted and fused according to the weight of each feature component in the plurality of clipped feature components to obtain the second feature map;

[0026] The features of the second feature map are extracted through the downsampling module of the encoder to obtain the basic feature.

[0027] In the above embodiment, the application can ensure that the representative feature slice is selected during feature extraction through the weighted fusion method or the selection of the feature slice with the maximum weight, so that the accurate basic feature can be obtained during the final feature extraction.

[0028] In some embodiments, the mean of the plurality of clipped feature components is calculated to obtain the weight of each feature component in the plurality of clipped feature components, comprising:

[0029] The mean of the plurality of clipped feature components is calculated to obtain the initial weight of each feature component in the plurality of clipped feature components;

[0030] The initial weight of each feature component in the plurality of clipped feature components is normalized through the normalization exponential function to obtain the weight of each feature component in the plurality of clipped feature components.

[0031] In the above embodiment, the application can amplify the weight of the feature component through the calculation of the normalization exponential function to obtain the weight of the feature component that is more in line with the actual demand.

[0032] In some embodiments, the second feature map is upsampled through the upsampling module of the decoder to obtain the deep feature corresponding to the second feature map, comprising:

[0033] The second feature map is upsampled through the upsampling module of the decoder to obtain the third feature map;

[0034] The third feature map is restored to the position of the second feature map through the upsampling module of the decoder, and the fourth feature map is obtained through the zero padding method, wherein the fourth feature map and the second feature map have the same size;

[0035] extracting features of the fourth feature map to obtain deep features.

[0036] In the above embodiment, the fourth feature map with the same size as the second feature map is obtained through sampling and image restoration of the second feature map and through zero padding, and the deep features of the image can be directly extracted.

[0037] In some embodiments, the feature extraction of the to-be-detected image is performed through the convolutional neural network to obtain the first feature map, including:

[0038] The low-pass filtering processing of the to-be-detected image is performed through the preset Gaussian kernel in the convolutional neural network to obtain the first feature map.

[0039] In the above embodiment, after the low-pass filtering processing of the to-be-detected image through the Gaussian kernel, the frequency spectrum aliasing caused by violating the Nyquist sampling theorem can be alleviated.

[0040] In some embodiments, after the fusion of the basic features and the deep features to obtain the image segmentation result, further including:

[0041] Obtaining a plurality of to-be-detected images and a plurality of image segmentation results corresponding to the plurality of to-be-detected images;

[0042] Training the convolutional neural network through the plurality of to-be-detected images and the plurality of image segmentation results to obtain an image segmentation model;

[0043] Inputting the to-be-segmented image corresponding to the product with defects in the industry into the image segmentation model to obtain a product defect segmentation result.

[0044] In the above embodiment, when the image segmentation is performed through the image segmentation model trained by the method of the present application, the to-be-segmented image can be directly input into the image segmentation model, and an accurate image segmentation result can be obtained.

[0045] In a second aspect, the embodiments of the present application provide an image segmentation device for detecting industrial defects, including:

[0046] The extraction module is configured to perform feature extraction on the to-be-detected image through the convolutional neural network to obtain the first feature map.

[0047] The first sampling module is configured to perform multiple down-sampling on the first feature map through the down-sampling module of the encoder to obtain the second feature map and the basic features corresponding to the first feature map.

[0048] The second sampling module is configured to perform up-sampling on the second feature map through the up-sampling module of the decoder to obtain the deep features corresponding to the second feature map.

[0049] A fusion module is configured to fuse the base feature and the deep feature to obtain the image segmentation result.

[0050] Optionally, the first sampling module is specifically configured to:

[0051] The first feature map is sampled by a preset number of feature sampling modules of the encoder to obtain a plurality of feature slices, wherein the plurality of feature slices have the same size.

[0052] The plurality of feature slices are extracted by a plurality of preset convolutional layers to obtain a plurality of feature components, wherein the plurality of feature components and the plurality of feature slices have the same spatial resolution.

[0053] The plurality of feature slices are extracted by a down-sampling module of the encoder to obtain the base feature and the second feature map.

[0054] Optionally, the preset convolutional layer comprises:

[0055] a plurality of convolutional layers.

[0056] The first convolutional layer of the plurality of convolutional layers comprises 64 convolutional kernels, and the second convolutional layer comprises 32 convolutional kernels.

[0057] Each convolutional layer of the plurality of convolutional layers comprises a batch normalization layer and a linear rectifier function.

[0058] Optionally, the first sampling module is specifically configured to:

[0059] The plurality of feature components are cropped in the spatial dimension according to a preset proportion coefficient to obtain a plurality of cropped feature components, wherein the cropping operation comprises cropping edge features of the plurality of feature components.

[0060] The mean of the plurality of cropped feature components is calculated to obtain the weight of each feature component in the plurality of cropped feature components.

[0061] The second feature map and the base feature are determined according to the weight of each feature component in the plurality of cropped feature components.

[0062] Optionally, the first sampling module is specifically configured to:

[0063] The feature slice corresponding to the maximum weight of each feature component in the plurality of cropped feature components is selected as the second feature map, or the plurality of feature slices are weighted and fused according to the weight of each feature component in the plurality of cropped feature components to obtain the second feature map.

[0064] The feature of the second feature map is extracted by the down-sampling module of the encoder to obtain the base feature.

[0065] Optionally, the first sampling module is specifically configured to:

[0066] obtaining a mean value of the plurality of cropped feature components to obtain an initial weight of each feature component in the plurality of cropped feature components;

[0067] normalizing the initial weight of each feature component in the plurality of cropped feature components by a normalization exponential function to obtain a weight of each feature component in the plurality of cropped feature components.

[0068] Optionally, the second sampling module is specifically configured to:

[0069] up-sampling the second feature map by an up-sampling module of the decoder to obtain a third feature map;

[0070] restoring the third feature map to a position of the second feature map by the up-sampling module of the decoder, and obtaining a fourth feature map by a zero padding method, wherein the fourth feature map and the second feature map have the same size;

[0071] extracting features of the fourth feature map to obtain deep features.

[0072] Optionally, the extraction module is specifically configured to:

[0073] performing low-pass filtering processing on the to-be-detected image by a preset Gaussian kernel in the convolutional neural network to obtain a first feature map.

[0074] Optionally, the apparatus further includes:

[0075] a training module configured to, after the fusion module fuses the base features and the deep features to obtain the image segmentation result, acquire a plurality of to-be-detected images and a plurality of image segmentation results corresponding to the plurality of to-be-detected images;

[0076] training the convolutional neural network by the plurality of to-be-detected images and the plurality of image segmentation results to obtain an image segmentation model;

[0077] inputting a to-be-segmented image corresponding to a product with a defect in the industry into the image segmentation model to obtain a product defect segmentation result.

[0078] In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, the memory stores computer readable instructions, when the computer readable instructions are executed by the processor, the steps in the method provided in the first aspect are executed.

[0079] In a fourth aspect, an embodiment of the present application provides a readable storage medium, which stores a computer program, when the computer program is executed by a processor, the steps in the method provided in the first aspect are executed.

[0080] Other features and advantages of the present application will be set forth in the following description, and in part will be apparent from the description, or can be learned by practice of the application. The objects and other advantages of the present application will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings. BRIEF DESCRIPTION OF DRAWINGS

[0081] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments of the present application will be briefly introduced as follows. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation to the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0082] Figure 1 A flow chart of an image segmentation method for detecting industrial defects provided by the embodiments of the present application;

[0083] Figure 2 A schematic diagram of down-sampling by a down-sampling module provided by the embodiments of the present application;

[0084] Figure 3 A structural schematic diagram of an image segmentation network provided by the embodiments of the present application;

[0085] Figure 4 A schematic block diagram of an image segmentation device for detecting industrial defects provided by the embodiments of the present application;

[0086] Figure 5 A structural schematic diagram of an image segmentation device for detecting industrial defects provided by the embodiments of the present application. DETAILED DESCRIPTION

[0087] The technical solutions in the embodiments of the present application will be described clearly and completely in the embodiments of the present application in combination with the drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. The components of the embodiments of the present application described and shown in the drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of the present application.

[0088] It should be noted that similar reference numerals and letters refer to like items in the accompanying drawings, and that, as used in this patent description, the terms "first", "second", and so on, merely denote important instead of their sequence.

[0089] First, some of the terms involved in the embodiments of the present application are explained to facilitate understanding by those skilled in the art.

[0090] Terminal device: can be a mobile terminal, a fixed terminal or a portable terminal, such as a mobile phone, a station, a unit, a device, a multimedia computer, a multimedia tablet, an Internet node, a communicator, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a personal communication system device, a personal navigation device, a personal digital assistant, an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an electronic book device, a game device, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. It is also foreseeable that the terminal device can support any type of interface for the user (such as wearable devices) and the like.

[0091] Server: can be a stand-alone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, and basic cloud computing services such as big data and artificial intelligence platforms.

[0092] CSPS module: down-sampling module of the common shift polyphase sampling (CSPS) encoder. Down-sampling, also known as decimation, is one of the basic contents of multi-rate signal processing. A sample value sequence is sampled once every few sample values, so that the new sequence is the down-sampling of the original sequence. Down-sampling is relative to the original continuous-time signal, and also needs to meet the sampling theorem, otherwise such down-sampling will cause signal component aliasing. Therefore, sampling is also called the discretization process of the waveform. It is used for down-sampling and feature extraction of images to obtain feature slices.

[0093] U-CSPS module: up-sampling module. Up-sampling is to collect samples of an analog signal. Sampling is to convert a signal that is continuous in time and amplitude into a signal that is discrete in time and amplitude under the action of a sampling pulse. It is used for feature slice restoration and feature extraction.

[0094] Cropping: Cropping is used to render part of the data for some volume rendering with large size and complex result to show the internal details. Two cropping techniques are provided in the vtkVolumeMapper class, which are Cropping and Clipping.

[0095] Batch Normalization: Batch Normalization, or BN for short, is a special layer in neural networks. In neural networks, consecutive multiple linear transformation layers are equivalent to a linear transformation layer (x = WU + B), so a non-linear transformation layer (activation function layer) is often connected after each linear layer.

[0096] ReLu: ReLu, or ReLU, is a common activation function in artificial neural networks, usually referring to a non-linear function represented by a ramp function and its variants.

[0097] Low-pass filter: Low-pass filter is a filtering method that allows low-frequency signals to pass normally, while high-frequency signals exceeding the set threshold are blocked or weakened. However, the degree of blocking or weakening will change depending on different frequencies and different filtering programs (purposes). It is sometimes also called high-cut filter or treble-cut filter. Low-pass filter is the opposite of high-pass filter.

[0098] Encoder: Encoder is a device that encodes signals (such as bit streams) or data into a signal form that can be used for communication, transmission and storage.

[0099] Decoder: Decoder is a device that converts signals transmitted and stored into analog signals and outputs.

[0100] argmax is a function that takes the argument (set) of a function. When we have another function y = f(x), if there is a result x0 = argmax(f(x)), it means that when the function f(x) takes x = x0, it gets the maximum value of the range of f(x); if there are multiple points that make f(x) take the same maximum value, then argmax(f(x)) is a point set. In other words, argmax(f(x)) is the variable point x (or set of x) that makes f(x) take the maximum value. Arg means argument, which here means "independent variable".

[0101] Choose: The function returns, from a list of arguments, and returns a value. The syntax of the Choose function has two parts, namely the index required argument and the choice required argument. Past tense: chose Past participle: chosen.

[0102] The application is applied to the scene of detecting industrial products, and a specific scene is to realize segmentation of an image in a target detection area in an image by fusing an encoder and a decoder through deep learning.

[0103] At present, traditional industrial defect detection is often completed by manual work, or product defect detection is performed through a neural network. When product defect detection is performed through the neural network, a region image of a product defect is placed in the neural network for defect detection. However, during the placing process, the defect is relatively translated due to the translation of the image, so that the segmentation result obtained when the industrial defect is detected is unstable and inaccurate. For example, some scholars mentioned that only one pixel movement can cause a confidence fluctuation of about 30% on the classification result. The LPS network pursues complete translation invariance of the segmentation result, and proves that the complete translation of the segmentation result can be ensured when circular shifts are performed on the input image. However, in the actual industrial field, the circular shift is not used, and the actual application scene needs to make the segmentation network have certain translation invariance in the common shift scene. However, due to the common shift, extra features are introduced at the edge of the input image or a part of the features is lost, the change of the edge distribution makes it difficult to model the translation invariance of the structure, and the increase of the number of down-sampling times makes the influence of the edge feature tend to the center of the feature map, causing large fluctuation of the segmentation result.

[0104] Therefore, the application extracts features of a to-be-detected image through a convolutional neural network to obtain a first feature map; performs multiple down-sampling on the first feature map through a down-sampling module of an encoder to obtain a second feature map and a basic feature corresponding to the first feature map; performs up-sampling on the second feature map through an up-sampling module of a decoder to obtain a deep feature corresponding to the second feature map; and fuses the basic feature and the deep feature to obtain an image segmentation result. In the image segmentation process based on deep learning, the application redesigns the down-sampling and up-sampling modes, extracts features of the image through the down-sampling of the encoder and the up-sampling of the decoder, and finally obtains the final image segmentation result through the feature fusion mode. The method can avoid the problem that the segmentation result of the target detection object is unstable due to the translation of the input image in the prior art. Therefore, the method of the application can improve the stability of the segmentation result while ensuring that the segmentation result has high precision, and meets the requirements of industrial defect detection.

[0105] In the embodiments of the present application, the execution subject can be an image segmentation device for detecting industrial defects in an image segmentation system for detecting industrial defects. In actual applications, the image segmentation device for detecting industrial defects can be an electronic device such as a terminal device and a server, which is not limited herein.

[0106] The embodiments of the present application will be described in detail below. Figure 1 The image segmentation method for detecting industrial defects of the embodiments of the present application will be described in detail.

[0107] Please refer to Figure 1 , Figure 1 A flowchart of an image segmentation method for detecting industrial defects provided by the embodiments of the present application is shown in FIG. 1. Figure 1 The image segmentation method for detecting industrial defects includes the following steps.

[0108] Step 110: performing feature extraction on the to-be-detected image through a convolutional neural network to obtain a first feature map.

[0109] The convolutional neural network can be a preset convolutional neural network, which includes a segmentation network of a U-shaped decoder and an encoder and is a symmetrical structure. The to-be-detected image can be an image having product defects in some industry, for example, scratches and cracks on the surface of a table body, and can also be a hole and the like photographed by a screen camera. The first feature map can be a target region containing a target detection object, or can be an entire region containing the target detection object. The target detection object can be scratches, holes, or cracks and the like.

[0110] In some embodiments of the present application, the feature extraction on the to-be-detected image through the convolutional neural network to obtain the first feature map includes: performing low-pass filtering processing on the to-be-detected image through a preset Gaussian kernel in the convolutional neural network to obtain the first feature map.

[0111] In the above process, after the low-pass filtering processing on the to-be-detected image through the Gaussian kernel, the frequency spectrum aliasing caused by violating the Nyquist sampling theorem can be alleviated.

[0112] The Gaussian kernel includes a Gaussian kernel function. The low-pass filtering processing on the to-be-detected image can filter some features other than the features related to the target detection object.

[0113] Step 120: performing multiple down-sampling on the first feature map through a down-sampling module of the encoder to obtain a second feature map and a basic feature corresponding to the first feature map.

[0114] The second feature map obtained through the down-sampling can be a thumbnail containing the target detection object. The second feature map can be a feature map obtained through multiple down-sampling. The basic feature can be some shape, size, and color and the like. The encoder includes a convolution module and a down-sampling module.

[0115] In some embodiments of the present application, the first feature map is down-sampled multiple times by the down-sampling module of the encoder to obtain the second feature map and the base feature corresponding to the first feature map, comprising: the first feature map is sampled a preset number of times by the down-sampling module of the encoder to obtain a plurality of feature slices, wherein the plurality of feature slices are of the same size; a plurality of feature components are obtained by extracting features of the plurality of feature slices through a preset convolutional layer, wherein the spatial resolution of the plurality of feature components and the plurality of feature slices is the same; and the base feature and the second feature map are obtained by extracting features of the plurality of feature slices through the down-sampling module of the encoder.

[0116] In the above embodiments of the present application, the second feature map and the base feature can be accurately obtained through feature screening by sampling the first feature map and extracting features of the feature components.

[0117] The preset number can be set according to requirements, for example, 4 times. The number of the plurality of feature slices is the same as the preset number.

[0118] In some embodiments, the base feature and the second feature map are obtained by extracting features of the plurality of feature slices through the down-sampling module of the encoder, comprising: a plurality of cropped feature components are obtained by performing a cropping operation on the plurality of feature components in a spatial dimension according to a preset proportion coefficient, wherein the cropping operation includes cropping edge features of the plurality of feature components; the weights of each feature component in the plurality of cropped feature components are obtained by calculating the mean of the plurality of cropped feature components; and the second feature map and the base feature are determined according to the weights of each feature component in the plurality of cropped feature components.

[0119] In the above embodiments of the present application, the cropping of the feature components can avoid the influence of some irrelevant features such as image edges on the final segmentation result, and ensure that the same feature slices can be selected in the down-sampling process of the network before and after the input image is translated, thereby increasing the translation invariance of the segmentation network and making the output segmentation result more stable.

[0120] The preset proportion coefficient can be set according to requirements, for example, the preset proportion coefficient k = 1 / 8. The cropping operation includes cropping the feature components corresponding to the image information extracted from the edges of the feature slices.

[0121] In some embodiments, the second feature map and the base feature are determined according to the weight of each of the plurality of cropped feature components, including: selecting a feature slice corresponding to the maximum weight of each of the plurality of cropped feature components as the second feature map or fusing the plurality of feature slices according to the weight of each of the plurality of cropped feature components to obtain the second feature map; and extracting features of the second feature map through the downsampling module of the encoder to obtain the base feature.

[0122] In the above embodiments, the application can ensure that the representative feature slice is selected during feature extraction by means of weighted fusion or selecting the feature slice with the maximum weight, so that the accurate base feature can be obtained during final feature extraction.

[0123] In some embodiments of the application, the preset convolutional layer includes: a plurality of convolutional layers; the first convolutional layer of the plurality of convolutional layers includes 64 convolutional kernels, and the second convolutional layer includes 32 convolutional kernels; and each convolutional layer of the plurality of convolutional layers includes a batch normalization layer and a linear rectifier function.

[0124] In the above process, the application can improve the accuracy of feature extraction during feature extraction of the feature slice by using the above-mentioned convolutional layer, batch normalization layer and function.

[0125] The batch normalization layer includes a plurality of normalization functions or a plurality of normalization exponential functions, which are used for image feature extraction or conversion.

[0126] In some embodiments of the application, the mean of the plurality of cropped feature components is calculated to obtain the weight of each of the plurality of cropped feature components, including: calculating the mean of the plurality of cropped feature components to obtain the initial weight of each of the plurality of cropped feature components; and performing normalization processing on the initial weight of each of the plurality of cropped feature components by using a normalization exponential function to obtain the weight of each of the plurality of cropped feature components.

[0127] In the above process, the application can amplify the weight of the feature component by using the calculation of the normalization exponential function to obtain the weight of the feature component that is more in line with the actual demand.

[0128] The image segmentation method for detecting industrial defects of the embodiments of the application will be described in detail below in combination with the downsampling module in Figure 2 .

[0129] Please refer to Figure 2 , Figure 2 for a schematic diagram of the downsampling performed by the downsampling module provided in the embodiments of the application, as shown in Figure 2The method of downsampling by the downsampling module includes:

[0130] The intermediate layer feature (the first feature map with a size of 10*10) is downsampled by a downsampling (CSPS) module in the encoder, for example, the input intermediate layer feature is:

[0131] X∈R W×H×C ;

[0132] wherein W, H and C respectively represent the width, height and channel number of the intermediate layer feature after the convolution module, X represents the input feature, R represents the feature component, and the input feature can be sampled into four feature slices with the same size by the downsampling module:

[0133]

[0134] wherein X i represents the i-th feature slice, R represents the feature component, W, H and C respectively represent the width, height and channel number of the intermediate layer feature after the convolution module. The four feature slices (for example, the size can be 5*5) obtained are input into two convolution modules (Conv(3*3, 64)) for feature extraction, each convolution module includes a convolution layer with a convolution kernel size of 3, a Batch Normalization (BN) layer and a Relu activation function. The first convolution module includes 64 convolution kernels, the second convolution module includes 32 convolution kernels, and the convolution layer in the module is set to a mirror mode (reflect), and the size of the feature map is not changed after convolution. Assuming that the overall mapping relationship of the two convolution modules is:

[0135] F1:X i →Y i ;

[0136] wherein, F1 represents the mapping relationship, X i represents the i-th feature slice, Y i represents the i-th feature slice after mapping, W, H and C respectively represent the width, height and channel number of the intermediate layer feature after the convolution module, that is, the feature component obtained after feature extraction, for example, four feature components (for example, the size can be 5*5).

[0137] It should be noted that: affected by the ordinary translation edge, although the background characteristics of the defect and its neighborhood do not change, the image edge position characteristics will be very different, if the corresponding weight is obtained by directly averaging each feature component at this time, the randomness of the edge feature will lead to the randomness of the feature slice selected by CSPS after ordinary translation, which is contrary to our expectation to realize the translation invariance of the segmentation network.

[0138] After the feature components are extracted from the feature slices in the application, a clipping operation is designed to adaptively cut the edges of the feature components according to their spatial dimensions, and the clipping operation is represented as a mapping:

[0139] F cr :Y i →Z i ;

[0140] Wherein, k∈(0,1],F cr represents the mapping relationship before and after clipping, Y i represents the i-th feature slice, Z i represents the i-th feature slice after clipping, W and H represent the width and height of the intermediate layer feature after the convolution module, R represents the feature component, and k represents the proportion coefficient of cutting the feature component, represents the down rounding of the element, and the mapping relationship can be realized by matrix slicing in practice, that is, only the middle region of the feature component is retained and the edges are uniformly cut. The best proportion coefficient set by the real data comparison in the application can be 0.8. The mean of the clipped feature component (for example, the size can be 3x3, which can be clipped by the calculation method of (int(5k)xint(5k))) can obtain the corresponding weight of each feature component:

[0141] mean:Z i →z i ;

[0142] Wherein k∈(0,1] same above, z i ∈R, i=1,2,3,4, Z i represents the i-th feature slice after clipping, z i represents the weight of the i-th feature slice after clipping, W and H represent the width and height of the intermediate layer feature after the convolution module, R represents the feature component, and z i ∈R, i=1,2,3,4 is input into the T-softmax function for feature fusion:

[0143]

[0144] Wherein pi represents the probability of the CSPS module selecting the i-th feature slice, T represents a temperature control coefficient, and can also be processed by means of the argmax+choose function to obtain a second feature map.

[0145] It should be noted that: in the test process, the p-th feature slice is used as the down-sampled feature map, z i represents the weight of the i-th feature slice, z j represents the weight of the j-th feature slice, and finally the probabilities of multiple feature slices are weighted and summed according to the weight of each feature component in the corresponding multiple cropped feature components to obtain a second feature map, and the basic feature and the deep feature are fused to obtain the final image segmentation result (for example, a 5x5 feature image).

[0146] The cropping operation is supplemented as follows: assuming that the feature dimension obtained after feature extraction on the feature slice is: 128x128x32, then the feature dimension after cropping is The cropping operation only retains the central feature, so the edge feature is completely cropped.

[0147] The up-sampling module U-CSPS in the decoder is the inverse operation of the CSPS sampling operation, which only needs to restore the selected feature slice of the down-sampling to the corresponding position of the up-sampling feature map, and zero-fill the other positions, which will not be repeated here.

[0148] Step 130: up-sampling the second feature map by the up-sampling module of the decoder to obtain the deep feature corresponding to the second feature map.

[0149] The deep feature includes texture, color, shape, size, and spatial relationship of the image. The decoder includes a convolution module and an up-sampling module.

[0150] In some embodiments of the present application, the second feature map is up-sampled by the up-sampling module of the decoder to obtain the deep feature corresponding to the second feature map, including: up-sampling the second feature map by the up-sampling module of the decoder to obtain a third feature map; restoring the third feature map to the position of the second feature map by the up-sampling module of the decoder, and obtaining a fourth feature map by zero-filling, wherein the fourth feature map and the second feature map are of the same size; extracting the features of the fourth feature map to obtain the deep feature.

[0151] In the above process, the second feature map is sampled and the image is restored, and the fourth feature map of the same size as the second feature map is obtained by zero-filling, so that the deep feature of the image can be directly extracted.

[0152] The third feature map is restored to the position of the second feature map by an upsampling module of the decoder, including selecting the center coordinates of the third feature map and the fourth feature map, taking the center coordinates as a restoration reference, and restoring the center of the third feature map to the position of the second feature map. The fourth feature map is obtained by zero padding, which is equivalent to supplementing the third feature map to the same size as the second feature map with no features, or supplementing the image in a blank manner to make the third feature map and the fourth feature map the same size.

[0153] Step 140: fusing the basic features and the deep features to obtain an image segmentation result.

[0154] The image segmentation result includes a target object, for example, a scratch, a crack, or a hole, etc. The basic features and the deep features can be fused to obtain all features of the entire target object, and finally the target object is formed according to the features, that is, the effect of separating the target object from the image is achieved. The basic features include some low-level features in the image, and the deep features include some high-level features in the image. The fusion of the features can be performed by transverse connection, that is, the basic features and the deep features can be directly spliced into the image segmentation result, or the key features of the target region in the basic features and the deep features can be extracted and spliced into the image segmentation result. For example, for a crack of an industrial part, the crack region can be preferentially demarcated, and only the basic features and the deep features of the crack region need to be extracted and spliced to obtain the image segmentation result.

[0155] In some embodiments of the present application, after fusing the basic features and the deep features to obtain the image segmentation result, Figure 1 The method also includes: obtaining a plurality of to-be-detected images and a plurality of image segmentation results corresponding to the plurality of to-be-detected images; training the convolutional neural network through the plurality of to-be-detected images and the plurality of image segmentation results to obtain an image segmentation model; inputting a to-be-segmented image corresponding to a product with a defect in the industry into the image segmentation model to obtain a product defect segmentation result.

[0156] In the above process, when the image segmentation model trained by the model of the present application is used for image segmentation, the to-be-segmented image can be directly input into the image segmentation model to obtain an accurate image segmentation result.

[0157] The image segmentation model comprises a decoder and an encoder of the application. A plurality of image segmentation results obtained by performing the method of the application multiple times and a plurality of original images (to-be-detected images) can be used as training samples of the image segmentation model. The image segmentation model can use tensorflow, caffe or pytorch in a deep learning framework. In application, an image containing a target object, for example, a crack, is input into the image segmentation model, and an accurate image segmentation result, that is, an image containing only the target object, can be obtained. The image segmentation model of the application can ensure the translation invariance of the segmentation network when the input image is circularly translated, and can also reduce or even change the fluctuation of the segmentation result when the input image is normally translated.

[0158] In the above Figure 1 process, the application extracts features of a to-be-detected image by using a convolutional neural network to obtain a first feature map; performs multiple down-sampling on the first feature map by using a down-sampling module of an encoder to obtain a second feature map and a basic feature corresponding to the first feature map; performs up-sampling on the second feature map by using an up-sampling module of a decoder to obtain a deep feature corresponding to the second feature map; and fuses the basic feature and the deep feature to obtain an image segmentation result. In the image segmentation process based on deep learning, the application redesigns the down-sampling and up-sampling modes, extracts features of an image by using the down-sampling of the encoder and the up-sampling of the decoder, and finally obtains a final image segmentation result by using the feature fusion mode. The method can avoid the problem that the segmentation result of a target detection object is unstable due to the translation of an input image in the prior art. Therefore, the method of the application can improve the stability of the segmentation result while ensuring that the segmentation result has high precision, thereby meeting the requirements of industrial defect detection.

[0159] The following will be described in detail Figure 3 The structure of the image segmentation network in the image segmentation model of the embodiment of the application will be described in detail.

[0160] Please refer to Figure 3 , Figure 3 A structure diagram of an image segmentation network provided by the embodiment of the application is shown in FIG. 1. Figure 2 The structure of the image segmentation network comprises:

[0161] 3 CSPS modules, 3 U-CSPS modules, a plurality of feature maps, a channel, a rectangle representing a feature map, and numbers on the upper and lower sides of the rectangle representing the number of channels of the feature map. For example, 3, 64, 96, 128, 192 and 256 in the figure represent the number of channels. The labels on the left and right sides represent the spatial dimensions, that is, the width and height, of the feature map. For example, 64x64, 128x128, 32x32 and 16x16 in the figure represent the resolution. All the arrows represent operation operations, which are as follows:

[0162] First, the image to be detected is input into the segmentation network, and after passing through the convolution kernel (size 3) number of 64, 96, 96 convolution modules, the Blur (blurring operation) + CSPS (downsampling) operation is used for downsampling, and the spatial resolution is 64x64, and the channel number is 96 Figure 1 ; down-sampling feature Figure 1 After passing through the convolution kernel (size 3) number of 128, 128 convolution modules, the Blur + CSPS operation is used for downsampling, and the spatial resolution is 32x32, and the channel number is 128 Figure 2 ; down-sampling feature Figure 2 After passing through the convolution kernel (size 3) number of 192, 192 convolution modules, the Blur + CSPS operation is used for downsampling, and the spatial resolution is 16x16, and the channel number is 192 Figure 3 ; down-sampling feature Figure 3 After passing through the convolution kernel (size 3) number of 256, 256, 128 convolution modules, the U-CSPS (up-sampling) + Blur operation is used to obtain the result, and the channel dimension of the symmetric low-level feature in the network is concatenated to obtain the spatial resolution 32x32, and the channel number is 320 Figure 1 ; up-sampling feature Figure 1 After passing through the convolution kernel (size 3) number of 128 convolution modules, the U-CSPS + Blur operation is used to obtain the result, and the channel dimension of the symmetric low-level feature in the neural network is concatenated to obtain the spatial resolution 64x64, and the channel number is 256 Figure 2 ; up-sampling feature map After passing through the convolution kernel (size 3) number of 128 convolution modules, the U-CSPS + Blur operation is used to obtain the result, and the channel dimension of the symmetric low-level feature in the network is concatenated to obtain the spatial resolution 128x128, and the channel number is 224 Figure 3 ; up-sampling feature Figure 3 After passing through the convolution kernel (size 3) number of 128 convolution modules, the convolution layer with convolution kernel (size 1) number of c is used to map the channel dimension to c, where c represents the number of possible categories of each pixel, and finally the softmax function + Conv1x1 (1x1 convolution) can be used to obtain the probability of each pixel belonging to each category, and finally the target object is extracted according to the classification of the pixel, wherein each convolution module above includes a Conv3x3 (3x3 convolution), a BN (batch normalization layer) and a ReLu (activation function).

[0163] It should be noted that all other convolution operations of non-down sampling and up sampling operations in the network keep the size of the spatial resolution unchanged, and the padding mode used is the mirror symmetry mode, so as to reduce the influence of the edge caused by translation in the convolution process.

[0164] The image segmentation device for detecting industrial defects will be described below. Figures 4-5 The image segmentation device for detecting industrial defects will be described below.

[0165] Please refer to Figure 4 , a schematic block diagram of an image segmentation device 400 for detecting industrial defects provided in the embodiment of the present application, which can be a module, a program segment or code on an electronic device. The image segmentation device 400 corresponds to the method embodiment described above, and can execute each step involved in the method embodiment. The specific functions of the image segmentation device 400 can be referred to the description below, and the detailed description is appropriately omitted here to avoid repetition. Figure 1 The image segmentation device 400 corresponds to the method embodiment described above, and can execute each step involved in the method embodiment. The specific functions of the image segmentation device 400 can be referred to the description below, and the detailed description is appropriately omitted here to avoid repetition. Figure 1 The image segmentation device 400 corresponds to the method embodiment described above, and can execute each step involved in the method embodiment. The specific functions of the image segmentation device 400 can be referred to the description below, and the detailed description is appropriately omitted here to avoid repetition.

[0166] Optionally, the image segmentation device 400 comprises:

[0167] The extraction module 410 is configured to perform feature extraction on the to-be-detected image by using the convolutional neural network to obtain a first feature map.

[0168] The first sampling module 420 is configured to perform multiple times of down sampling on the first feature map by using the down sampling module of the encoder to obtain a second feature map and a basic feature corresponding to the first feature map.

[0169] The second sampling module 430 is configured to perform up sampling on the second feature map by using the up sampling module of the decoder to obtain a deep feature corresponding to the second feature map.

[0170] The fusion module 440 is configured to fuse the basic feature and the deep feature to obtain an image segmentation result.

[0171] Optionally, the first sampling module is specifically configured to:

[0172] perform feature sampling on the first feature map by using the down sampling module of the encoder for a preset number of times to obtain a plurality of feature slices, wherein the plurality of feature slices are of the same size; perform feature extraction on the plurality of feature slices by using a preset convolutional layer to obtain a plurality of feature components, wherein the plurality of feature components and the plurality of feature slices have the same spatial resolution; and extract features of the plurality of feature slices by using the down sampling module of the encoder to obtain the basic feature and the second feature map.

[0173] Optionally, the preset convolutional layer comprises:

[0174] The plurality of convolutional layers comprises a first convolutional layer with 64 convolutional kernels and a second convolutional layer with 32 convolutional kernels; each convolutional layer in the plurality of convolutional layers comprises a batch normalization layer and a linear rectifier function.

[0175] Optionally, the first sampling module is specifically configured to:

[0176] The plurality of feature components are subjected to a clipping operation in a spatial dimension according to a preset proportion coefficient, to obtain a plurality of clipped feature components, wherein the clipping operation comprises clipping edge features of the plurality of feature components; the mean of the plurality of clipped feature components is calculated to obtain the weight of each feature component in the plurality of clipped feature components; and the second feature map and the basic feature are determined according to the weight of each feature component in the plurality of clipped feature components.

[0177] Optionally, the first sampling module is specifically configured to:

[0178] The feature slice corresponding to the maximum weight of each feature component in the plurality of clipped feature components is selected as the second feature map, or the plurality of feature slices are weighted and fused according to the weight of each feature component in the plurality of clipped feature components to obtain the second feature map; and the feature of the second feature map is extracted through the downsampling module of the encoder to obtain the basic feature.

[0179] Optionally, the first sampling module is specifically configured to:

[0180] The mean of the plurality of clipped feature components is calculated to obtain the initial weight of each feature component in the plurality of clipped feature components; and the initial weight of each feature component in the plurality of clipped feature components is normalized by a normalization exponential function to obtain the weight of each feature component in the plurality of clipped feature components.

[0181] Optionally, the second sampling module is specifically configured to:

[0182] The second feature map is upsampled through the upsampling module of the decoder to obtain a third feature map;

[0183] The third feature map is restored to the position of the second feature map through the upsampling module of the decoder, and a fourth feature map is obtained through zero padding, wherein the fourth feature map and the second feature map have the same size; and the feature of the fourth feature map is extracted to obtain a deep feature.

[0184] Optionally, the extraction module is specifically configured to:

[0185] The first feature map is obtained by performing low-pass filtering on the to-be-detected image through a preset Gaussian kernel in the convolutional neural network.

[0186] Optionally, the device further comprises:

[0187] The training module is configured to, after the fusion module fuses the basic features and the deep features to obtain the image segmentation result, acquire a plurality of to-be-detected images and a plurality of image segmentation results corresponding to the plurality of to-be-detected images; train the convolutional neural network by using the plurality of to-be-detected images and the plurality of image segmentation results, to obtain an image segmentation model; and input a to-be-detected image corresponding to a product with a defect in the industry into the image segmentation model, to obtain a product defect segmentation result.

[0188] Please refer to Figure 5 The device for detecting industrial defects provided in the embodiments of the present application can include a memory 510 and a processor 520. Optionally, the device can further include a communication interface 530 and a communication bus 540. The device corresponds to the method embodiments described above, and can perform the steps involved in the method embodiments. The specific functions of the device can be referred to the description below. Figure 1 The device corresponds to the method embodiments described above, and can perform the steps involved in the method embodiments. The specific functions of the device can be referred to the description below. Figure 1 The device corresponds to the method embodiments described above, and can perform the steps involved in the method embodiments. The specific functions of the device can be referred to the description below.

[0189] Specifically, the memory 510 is configured to store computer readable instructions.

[0190] The processor 520 is configured to process the computer readable instructions stored in the memory, and can perform the steps in the method. Figure 1 The processor 520 is configured to process the computer readable instructions stored in the memory, and can perform the steps in the method.

[0191] The communication interface 530 is configured to communicate with other node devices in signaling or data. For example, it is used for communication with a server or a terminal, or communication with other device nodes, and the embodiments of the present application are not limited thereto.

[0192] The communication bus 540 is configured to realize direct connection communication of the above-mentioned components.

[0193] In the embodiments of the present application, the communication interface 530 of the device is configured to communicate with other node devices in signaling or data. The memory 510 can be a high-speed RAM memory, or a non-volatile memory such as at least one disk memory. The memory 510 can also be at least one storage device located away from the aforementioned processor. The memory 510 stores computer readable instructions, and when the computer readable instructions are executed by the processor 520, the electronic device performs the above-mentioned Figure 1The processor 520 can be used on the image segmentation device 400 and used to perform the functions in the present application. For example, the processor 520 described above can be a general processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and the embodiments of the present application are not limited thereto.

[0194] The embodiments of the present application further provide a readable storage medium, when the computer program is executed by the processor, the method is executed as Figure 1 The processor 520 can be used on the image segmentation device 400 and used to perform the functions in the present application. For example, the processor 520 described above can be a general processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and the embodiments of the present application are not limited thereto.

[0195] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the device described above can refer to the corresponding process in the foregoing method, and will not be described in detail here.

[0196] In summary, the embodiments of the present application provide an image segmentation method, device, equipment and storage medium for detecting industrial defects, the method comprising: extracting features of a to-be-detected image through a convolutional neural network to obtain a first feature map; performing multiple times of down-sampling on the first feature map through a down-sampling module of an encoder to obtain a second feature map and a basic feature corresponding to the first feature map; performing up-sampling on the second feature map through an up-sampling module of a decoder to obtain a deep feature corresponding to the second feature map; and fusing the basic feature and the deep feature to obtain an image segmentation result. Through the method, the stability and accuracy of the segmentation image obtained when detecting industrial defects can be improved.

[0197] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can also be implemented by other means. The apparatus embodiments described above are only illustrative, for example, the flowcharts and block diagrams in the drawings show the possible implementation architecture, function and operation of the apparatus, method and computer program product according to the embodiments of the present application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment or a part of code, which contains one or more executable instructions for implementing the specified logic function. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different order from that shown in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and sometimes they can be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0198] In addition, the functional modules in the embodiments of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0199] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0200] The above merely provides an example of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, and thus, once an item is defined in one drawing, it need not be further defined and explained in subsequent drawings.

[0201] The above merely provides an example of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, and thus, once an item is defined in one drawing, it need not be further defined and explained in subsequent drawings.

[0202] It should be noted that the relational terms herein such as first and second and the like are used solely to distinguish one from another entity or action without necessarily requiring or implying any actual relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

Claims

1. An image segmentation method for detecting industrial defects, characterized in that, include: The first feature map is obtained by extracting features from the image to be detected using a convolutional neural network; The first feature map is downsampled multiple times by the encoder's downsampling module to obtain the second feature map and the basic features corresponding to the first feature map; The second feature map is upsampled by the upsampling module of the decoder to obtain the deep features corresponding to the second feature map; By fusing the basic features and the deep features, the image segmentation result is obtained; The encoder's downsampling module performs multiple downsampling operations on the first feature map to obtain the second feature map and the basic features corresponding to the first feature map, including: The encoder's downsampling module performs a preset number of feature samples on the first feature map to obtain multiple feature slices, wherein the multiple feature slices are of the same size. Features are extracted from the multiple feature slices by a preset convolutional layer to obtain multiple feature components, wherein the multiple feature components and the multiple feature slices have the same spatial resolution; The encoder's downsampling module extracts features from the multiple feature slices to obtain the basic features and the second feature map; The step of extracting features from the multiple feature slices through the downsampling module of the encoder to obtain the basic features and the second feature map includes: The multiple feature components are clipped in the spatial dimension according to a preset scaling factor to obtain multiple clipped feature components. The average value of the multiple cropped feature components is calculated to obtain the weight of each feature component among the multiple cropped feature components. The second feature map and the basic feature are determined based on the weight of each feature component among the plurality of cropped feature components; The step of determining the second feature map and the basic feature based on the weight of each feature component among the plurality of cropped feature components includes: The feature slice corresponding to the largest weight of each feature component among the multiple cropped feature components is selected as the second feature map, or the multiple feature slices are weighted and fused according to the weight of each feature component among the multiple cropped feature components to obtain the second feature map. The basic features are obtained by extracting features from the second feature map through the downsampling module of the encoder.

2. The method according to claim 1, characterized in that, The preset convolutional layer includes: Multiple convolutional layers; The first convolutional layer in the plurality of convolutional layers includes 64 convolutional kernels, and the second convolutional layer includes 32 convolutional kernels; Each of the plurality of convolutional layers is followed by a batch normalization layer and a linear rectified function.

3. The method according to claim 1, characterized in that, The step of calculating the average of the multiple cropped feature components to obtain the weight of each feature component in the multiple cropped feature components includes: The mean of the multiple cropped feature components is calculated to obtain the initial weight of each feature component among the multiple cropped feature components. The initial weights of each feature component in the plurality of cropped feature components are normalized by using a normalized exponential function to obtain the weights of each feature component in the plurality of cropped feature components.

4. The method according to any one of claims 1-3, characterized in that, The upsampling module of the decoder upsamples the second feature map to obtain the deep features corresponding to the second feature map, including: The second feature map is upsampled by the upsampling module of the decoder to obtain the third feature map; The third feature map is restored to the position of the second feature map by the upsampling module of the decoder, and the fourth feature map is obtained by padding with zeros. The features of the fourth feature map are extracted to obtain the deep features.

5. The method according to any one of claims 1-3, characterized in that, The step of extracting features from the image to be detected using a convolutional neural network to obtain a first feature map includes: The first feature map is obtained by performing low-pass filtering on the image to be detected using a pre-set Gaussian kernel in a convolutional neural network.

6. The method according to any one of claims 1-3, characterized in that, After fusing the basic features and the deep features to obtain the image segmentation result, the method further includes: Obtain multiple images to be detected and multiple image segmentation results corresponding to the multiple images to be detected; The convolutional neural network is trained using the multiple images to be detected and the multiple image segmentation results to obtain an image segmentation model; The image to be segmented corresponding to a defective product in the industry is input into the image segmentation model to obtain the product defect segmentation result.

7. An image segmentation device for detecting industrial defects, characterized in that, include: The extraction module is used to extract features from the image to be detected using a convolutional neural network to obtain a first feature map; The first sampling module is used to downsample the first feature map multiple times through the downsampling module of the encoder to obtain the second feature map and the basic features corresponding to the first feature map; The second sampling module is used to upsample the second feature map through the upsampling module of the decoder to obtain the deep features corresponding to the second feature map; The fusion module is used to fuse the basic features and the deep features to obtain the image segmentation result; The first sampling module is specifically used for: The encoder's downsampling module performs a preset number of feature samples on the first feature map to obtain multiple feature slices, wherein the multiple feature slices are of the same size. Features are extracted from the multiple feature slices by a preset convolutional layer to obtain multiple feature components, wherein the multiple feature components and the multiple feature slices have the same spatial resolution; The encoder's downsampling module extracts features from the multiple feature slices to obtain the basic features and the second feature map; The first sampling module is further configured to: The multiple feature components are clipped in the spatial dimension according to a preset scaling factor to obtain multiple clipped feature components. The average value of the multiple cropped feature components is calculated to obtain the weight of each feature component among the multiple cropped feature components. The second feature map and the basic feature are determined based on the weight of each feature component among the plurality of cropped feature components; The first sampling module is further configured to: The feature slice corresponding to the largest weight of each feature component among the multiple cropped feature components is selected as the second feature map, or the multiple feature slices are weighted and fused according to the weight of each feature component among the multiple cropped feature components to obtain the second feature map. The basic features are obtained by extracting features from the second feature map through the downsampling module of the encoder.

8. An electronic device, characterized in that, include: A memory and a processor, the memory storing computer-readable instructions that, when executed by the processor, perform the steps of the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, include: A computer program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1-6.

Images