Image Segmentation Method, Device and Terminal Device Based on Structural Similarity Measurement

Abstract

The present invention discloses an image segmentation method, apparatus, and terminal device based on structural similarity measurement. The method comprises: obtaining a training set, performing preprocessing and block operations to obtain original image subgraphs of the training set and annotated image subgraphs of the training set; constructing a U-shaped segmentation network including an encoding path and a decoding path; training the U-shaped segmentation network using the original image subgraphs of the training set and the annotated image subgraphs of the training set; splicing the training set segmented image subgraphs predicted and output by the U-shaped segmentation network to obtain a complete training set segmented image; minimizing the difference between the training set segmented image subgraphs and the annotated image subgraphs of the training set using a cross-entropy loss function; inputting the preprocessed and block-parted original image subgraphs to be segmented into the trained U-shaped segmentation network, and splicing the segmentation results to obtain a final segmented image. The present invention significantly improves the accuracy and visual effect of image segmentation by expressing the structural similarity between feature points.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and more particularly to an image segmentation method, apparatus and terminal device based on structural similarity measurement. Background Art

[0002] In recent years, as industrial robot technology has matured, it has been widely adopted in automated production lines, significantly improving production efficiency, reducing labor workload, and significantly increasing economic benefits. However, traditional industrial robots suffer from significant deficiencies in flexibility, multitasking, and real-time performance. Changes in product shape, size, or process flow require re-teaching and re-programming, significantly impacting the efficiency of automated production lines. Machine vision image segmentation can provide industrial robots with the correct location and boundaries of objects, significantly improving the accuracy of their grasping and placement, and enhancing their environmental perception capabilities in real-world applications.

[0003] Image segmentation algorithms are mainly divided into three categories: threshold-based segmentation methods, edge-based segmentation methods, and region-based segmentation methods. Threshold-based segmentation methods are very sensitive to noise and are prone to mis-segmentation of overlapping locations when using the histogram peak-valley method to segment foreground and background grayscale. Edge-based segmentation methods are suitable for application scenarios with clear target edges, because commonly used edge detection operators are prone to false edges and loss of edges. Region-based segmentation methods are prone to over-segmentation and under-segmentation of images and are easily affected by continuous features.

[0004] Currently, there are three major challenges in deep learning-based machine vision image segmentation tasks:

[0005] (1) The target images captured by industrial robots are easily affected by lighting, shadows, and ambient temperature. The quality of image acquisition determines the reliability and accuracy of the segmentation model.

[0006] (2) The target and background colors in the industrial robot environment are consistent, making segmentation difficult.

[0007] (3) The shape and size of the target to be grasped vary greatly, and the segmentation model needs to correctly segment the diverse and changing target images.

[0008] Therefore, how to provide an image segmentation method, apparatus and terminal device based on structural similarity measurement that can improve segmentation accuracy and segmentation visual effect is a problem that those skilled in the art urgently need to solve. Summary of the Invention

[0009] In view of this, the present invention provides an image segmentation method, apparatus and terminal device based on structural similarity measurement, which significantly improves the accuracy and visual effect of image segmentation by expressing the features of structural similarity between feature points.

[0010] In order to achieve the above object, the present invention adopts the following technical solutions:

[0011] An image segmentation method based on structural similarity measurement includes a training phase and a segmentation phase;

[0012] The training phase includes:

[0013] Obtaining a machine vision-based training set and performing preprocessing; the training set includes original training set images and corresponding annotated training set images;

[0014] Performing a block operation on the training set original image and the training set annotated image to obtain a training set original image sub-image and a training set annotated image sub-image;

[0015] Construct a U-shaped segmentation network including an encoding path and a decoding path;

[0016] The U-shaped segmentation network is trained using the original image subgraphs of the training set and the annotated image subgraphs of the training set, the training set segmentation image subgraphs predicted and output by the U-shaped segmentation network are spliced to obtain a complete training set segmentation image, and the cross entropy loss function is used to minimize the difference between the training set segmentation image subgraphs and the training set annotated image subgraphs to optimize the parameters of the U-shaped segmentation network;

[0017] The segmentation phase includes:

[0018] Obtain the original image to be segmented based on machine vision, and perform preprocessing and block operations to obtain the sub-images of the original image to be segmented;

[0019] The original image sub-image to be segmented is input into the trained U-shaped segmentation network to obtain a segmented image sub-image, and the segmented image sub-images belonging to the same original image to be segmented are spliced in a block order to obtain a final segmented image.

[0020] Furthermore, in the above-mentioned image segmentation method based on structural similarity measurement, the expression of the cross entropy loss function is:

[0021]

[0022] Among them, N represents the number of pixels, Represents the prediction of pixel i, y (i) Represents the true label corresponding to pixel i.

[0023] Furthermore, in the above-mentioned image segmentation method based on structural similarity measurement, preprocessing is performed on the original image to be segmented or the training set, including: enhancing the image contrast by using an HSV stretching method.

[0024] Furthermore, in the above-mentioned image segmentation method based on structural similarity measurement, the encoding path includes four repeated structural similarity measurement encoding units; the decoding path includes three repeated decoding units with the same feature depth as the encoding path; the structural similarity measurement encoding units and the decoding units of corresponding feature depths are connected in series;

[0025] Each of the structural similarity measurement encoding units includes 2 convolutional layers, 1 pooling layer and 1 structural similarity weight calculation module;

[0026] Each of the decoding units includes 2 convolutional layers and 1 transposed convolutional layer.

[0027] Furthermore, in the above-mentioned image segmentation method based on structural similarity measurement, during the training phase, the structural similarity weight calculation module performs the following steps:

[0028] Traversing the feature point vector at each feature point position of the original image sub-image of the training set;

[0029] Calculating the global feature vector of the original image sub-image of the training set using a global pooling layer;

[0030] Calculating the structural similarity weight between the global feature vector and the feature point vector of the original image sub-image of the training set;

[0031] Adding the structural similarity weight to the corresponding feature point vector to form a weighted feature map;

[0032] The training set original image sub-image and the weighted feature map are stacked along the direction of the feature channel.

[0033] Furthermore, in the above-mentioned image segmentation method based on structural similarity measurement, the calculation formula of the structural similarity weight is:

[0034] w mn =l(m,n)c(m,n)u(m,n)

[0035]

[0036]

[0037]

[0038] Among them, w mnRepresents the structural similarity weight; l(m,n), c(m,n) and u(m,n) represent the brightness, contrast and structural similarity between m and n respectively; m represents the global feature vector of the original image sub-image, which is achieved by global average pooling, and n represents the feature point vector at (x,y) in the original image sub-image; μ m Represents the mean of the global eigenvector m, μ n represents the mean of the feature point vector n; σ m represents the standard deviation of the global eigenvector m, σ n Represents the standard deviation of the feature point vector n, σ mn represents the covariance between m and n; C1, C2, C3 are constant terms;

[0039] The calculation formula of the weighted feature map is:

[0040] n′=w mn n,

[0041] Where n′ represents the weighted feature point vector.

[0042] The present invention also provides an image segmentation device based on structural similarity measurement, comprising:

[0043] An acquisition module, used to acquire the original image to be segmented based on machine vision;

[0044] A preprocessing module, used for preprocessing the original image to be segmented;

[0045] The image segmentation module is used to segment the pre-processed original image to be segmented into blocks to obtain sub-images of the original image to be segmented;

[0046] The segmentation prediction module is used to segment the original image sub-image to be segmented using the above-mentioned trained U-shaped segmentation network to obtain segmented image sub-images, and splice the segmented image sub-images belonging to the same original image to be segmented in a block order to obtain a final segmented image.

[0047] Furthermore, in the above-mentioned image segmentation device based on structural similarity measurement, the preprocessing module includes:

[0048] RGB-HSV mapping unit, used to map the original image to be segmented from RGB color space to HSV color space;

[0049] A normalization unit, used for performing minimum-maximum normalization processing on the S channel and V channel of the original image to be segmented in the HSV color space;

[0050] The mapping unit is used to remap the original image to be segmented from the HSV color space back to the RGB color space.

[0051] Furthermore, in the above-mentioned image segmentation device based on structural similarity measurement, the segmentation prediction module includes a U-shaped network unit and a sub-image splicing unit; wherein the U-shaped network unit includes: an encoding path layer, a decoding path layer, a supervision unit, and a loss function unit:

[0052] The coding path layer includes 4 repeated structural similarity measurement coding units; each of the structural similarity measurement coding units includes 2 convolutional layers, 1 pooling layer and 1 structural similarity weight calculation module;

[0053] The decoding path layer includes three repeated decoding units with the same feature depth as the encoding path layer; the structural similarity measurement encoding unit and the decoding unit corresponding to the feature depth are connected in series; each of the decoding units includes two convolutional layers and one transposed convolutional layer;

[0054] The supervision unit is used to use the original image to be segmented as supervision information;

[0055] The sub-image stitching unit is used to stitch the segmented sub-images belonging to the same original image to be segmented in a block order to obtain a final segmented image;

[0056] The loss function unit is used to optimize the parameters of the U-shaped segmentation network during the training phase.

[0057] The present invention also provides a terminal device, comprising at least one processor and at least one memory communicatively connected to the processor; the memory stores program instructions that can be executed by the processor; when the processor runs the program instructions, it executes the steps of the image segmentation method based on structural similarity measurement as described above.

[0058] It can be seen from the above technical solution that compared with the prior art, the present invention discloses an image segmentation method, apparatus and terminal device based on structural similarity measurement, which has the following beneficial effects: the present invention performs structural similarity measurement on the global feature vector and feature point vector of the block image, and introduces structural similarity weights of global semantics and single feature point semantics into the block image, so that the feature points are weighted with contextual information. The higher the structural similarity, the greater the contribution of the feature point to the segmentation target, which improves the feature expression ability of the segmentation model, thereby improving the edge segmentation accuracy of the segmented object. It can be widely used in the assembly line operation of industrial robots with machine vision, and at the same time, it can improve the accuracy of segmentation and the visual effect of segmentation when the size and shape of the target to be segmented vary greatly. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0060] Figure 1 A flowchart of the image segmentation method based on structural similarity measurement provided by the present invention;

[0061] Figure 2 A schematic diagram of the structure of the U-shaped segmentation network provided by the present invention;

[0062] Figure 3 The flow chart of the present invention for measuring the structural similarity of the sub-images of the original image to be segmented;

[0063] Figure 4 A schematic diagram of data flow during the training and testing phases of the U-shaped segmentation network provided by the present invention;

[0064] Figure 5 This is a structural schematic diagram of the image segmentation device based on structural similarity measurement provided by the present invention. DETAILED DESCRIPTION

[0065] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0066] like Figure 1 、 Figure 4 As shown, the embodiment of the present invention discloses an image segmentation method based on structural similarity measurement, which includes a training stage and a segmentation stage;

[0067] The training phase includes:

[0068] S1. Obtain a machine vision-based training set and perform preprocessing; the training set includes the original training set images and the corresponding training set annotated images;

[0069] S2. Perform a block operation on the original image of the training set and the annotated image of the training set to obtain a sub-image of the original image of the training set and a sub-image of the annotated image of the training set;

[0070] S3, constructing a U-shaped segmentation network including an encoding path and a decoding path;

[0071] S4. Use the training set original image subgraphs and the training set annotated image subgraphs to train the U-shaped segmentation network, splice the training set segmentation image subgraphs predicted and output by the U-shaped segmentation network to obtain the complete training set segmentation image, and use the cross entropy loss function to minimize the difference between the training set segmentation image subgraphs and the training set annotated image subgraphs to optimize the U-shaped segmentation network parameters;

[0072] The segmentation phase includes:

[0073] S5. Obtain the original image to be segmented based on machine vision, and perform preprocessing and block operations to obtain sub-images of the original image to be segmented;

[0074] S6. Input the original image sub-image to be segmented into the trained U-shaped segmentation network to obtain the segmented image sub-image, and splice the segmented image sub-images belonging to the same original image to be segmented in the order of blocks to obtain the final segmented image.

[0075] The above steps are further explained below.

[0076] In S1, data augmentation is performed on the images in the machine vision-based training set. Specifically, the HSV stretching method can be used to enhance the image contrast and make the image boundaries clearer. This is achieved through the following process:

[0077] 1) Map the original image of the training set from RGB color space to HSV color space;

[0078] 2) Perform min-max normalization on the S and V channels of the original images of the training set; the normalization function is as follows:

[0079]

[0080]

[0081] Where S(x,y) represents the value of the S color channel at (x,y), V(x,y) represents the value of the V color channel at (x,y), Max(x,y) represents the maximum value of the S or V color channel, and Min(x,y) represents the minimum value of the S or V color channel.

[0082] 3) Remap the original images of the training set back to the RGB color space.

[0083] In S2, the specific steps for performing block operations on the original images of the training set are as follows:

[0084] Use a sliding window of uniform size to traverse the original images of the training set from top to bottom and from left to right, and there should be no overlap between two adjacent sliding windows. The size of the original images of the training set must be an integer multiple of the sliding window size. When the size of the original images of the training set is 512×1024, the sliding window size can be 128×256 or 64×128; when the size of the original images of the training set is 256×256, the sliding window size can be 32×32 or 16×16.

[0085] In a specific embodiment, in S3, as Figure 2 As shown, the U-shaped segmentation network consists of an encoding path and a decoding path. The encoding path reduces the image resolution while extracting semantic features from the image, while the decoding path restores the image resolution to its original size. To preserve the pixel position information of each feature depth feature map during the encoding phase while extracting feature information, the U-shaped segmentation network concatenates feature maps from encoding units to decoding units of the same feature depth. Finally, the U-shaped segmentation network uses a softmax activation function to discriminate the category of each pixel in the feature map after the resolution is restored, thereby obtaining the final segmented image sub-image.

[0086] Specifically, the encoding path includes 4 repeated structural similarity measurement encoding units; the decoding path includes 3 repeated decoding units with the same feature depth as the encoding path; the structural similarity measurement encoding units and decoding units of corresponding feature depths are connected in series; the connection method is to stack the feature maps along the feature channel direction.

[0087] Each structural similarity measurement encoding unit includes 2 convolutional layers, 1 pooling layer and 1 structural similarity weight calculation module; the structural similarity weight calculation module is used to achieve the purpose of measuring the structural similarity weight between the global feature vector and the feature point vector.

[0088] Each decoding unit consists of 2 convolutional layers and 1 transposed convolutional layer.

[0089] In S4, during the training phase of the U-shaped segmentation network, the U-shaped segmentation network is trained using the original image subgraphs of the training set and the annotated image subgraphs of the training set, and the training set segmentation image subgraphs predicted and output by the U-shaped segmentation network are spliced to obtain a complete training set segmentation image.

[0090] The stitching operation is to stitch the segmented image sub-images belonging to the same segmented image together to obtain a machine vision image with the original size. The cross-entropy loss function is used to minimize the difference between the segmented image sub-image and the annotated image sub-image to optimize the segmentation effect of the segmentation network.

[0091] Among them, the expression of the cross entropy loss function is:

[0092]

[0093] Among them, N represents the number of pixels, Represents the prediction of pixel i, y (i) Represents the true label corresponding to pixel i.

[0094] Specifically, during the training phase, the structural similarity weight calculation module performs the following steps:

[0095] The feature point vector at each feature point position of the original image sub-image of the training set is traversed; the feature point vector of each feature point on the original image sub-image to be segmented can be traversed using a sliding window with a size of 1×1 and a step size of 1.

[0096] Use the global pooling layer to calculate the global feature vector of the original image sub-image of the training set;

[0097] Calculate the structural similarity weight between the global feature vector and feature point vector of the training set image sub-image;

[0098] Add the structural similarity weight to the corresponding feature point vector to form a weighted feature map;

[0099] The original image sub-image of the training set and the weighted feature map are stacked along the direction of the feature channel.

[0100] The stacked feature maps pass through the very end of the network, that is, the activation function Softmax, to obtain the segmented image sub-image. Then, the sub-images belonging to the same original image are spliced from top to bottom and from left to right to restore the original size of the image.

[0101] The calculation formula of the structural similarity weight is:

[0102] w mn =l(m,n)c(m,n)u(m,n)

[0103]

[0104]

[0105]

[0106] Among them, w mn Represents the structural similarity weight; l(m,n), c(m,n) and u(m,n) represent the brightness, contrast and structural similarity between m and n respectively; m represents the global feature vector of the original image sub-image, which is achieved by global average pooling, and n represents the feature point vector at (x,y) in the original image sub-image; μ m Represents the mean of the global eigenvector m, μ n represents the mean of the feature point vector n; σm represents the standard deviation of the global eigenvector m, σ n Represents the standard deviation of the feature point vector n, σ mn represents the covariance between m and n; C1, C2, C3 are constant terms;

[0107] The calculation formula of the weighted feature map is:

[0108] n′=w mn n,

[0109] Where n′ represents the weighted feature point vector.

[0110] Specifically, in S5, the original image to be segmented based on machine vision is obtained, and preprocessed and segmented to obtain sub-images of the original image to be segmented. In this embodiment, the original image to be segmented is subjected to the same preprocessing operation as in step S1 and the same segmentation operation as in step S2.

[0111] In other embodiments, Figure 5 As shown, the present invention also provides an image segmentation device based on structural similarity measurement, comprising:

[0112] An acquisition module, used to acquire the original image to be segmented based on machine vision;

[0113] A preprocessing module, used to preprocess the original image to be segmented;

[0114] The image segmentation module is used to segment the pre-processed original image to be segmented into blocks to obtain sub-images of the original image to be segmented;

[0115] The segmentation prediction module is used to segment the original image sub-image to be segmented using the trained U-shaped segmentation network to obtain segmented image sub-images, and to splice the segmented image sub-images belonging to the same original image to be segmented in block order to obtain the final segmented image.

[0116] In one embodiment, the pre-processing module includes:

[0117] RGB-HSV mapping unit, used to map the original image to be segmented from RGB color space to HSV color space;

[0118] A normalization unit, used for performing minimum-maximum normalization processing on the S channel and V channel of the original image to be segmented in the HSV color space;

[0119] The mapping unit is used to remap the original image to be segmented from the HSV color space back to the RGB color space.

[0120] In one embodiment, the segmentation prediction module includes: a U-shaped network unit and a subgraph splicing unit, wherein the U-shaped network unit includes: an encoding path layer, a decoding path layer, a supervision unit, and a loss function unit; the encoding path layer includes four repeated structural similarity measurement encoding units; each structural similarity measurement encoding unit includes two convolutional layers, one pooling layer, and one structural similarity weight calculation module;

[0121] The decoding path layer includes three repeated decoding units with the same feature depth as the encoding path layer; the structural similarity measurement encoding unit and decoding unit corresponding to the feature depth are connected in series; each decoding unit includes two convolutional layers and one transposed convolutional layer;

[0122] The supervision unit is used to use the original image to be segmented as supervision information;

[0123] The sub-image stitching unit is used to stitch the segmented sub-images belonging to the same original image to be segmented in a block order to obtain a final segmented image;

[0124] Loss function unit,The loss function unit is used to optimize the parameters of the U-shaped segmentation network during the training phase.

[0125] In a specific embodiment, the structural similarity weight calculation module includes:

[0126] A pixel scanning unit, used for traversing the feature point vector at each feature point position of the original image sub-image to be segmented;

[0127] A global feature vector unit, used to calculate the global feature vector of the original image sub-image to be segmented using a global pooling layer;

[0128] A structural similarity weight calculation unit, used to calculate the structural similarity weight between the global feature vector and the feature point vector of the original image sub-image to be segmented;

[0129] A feature map weighting unit is used to add the structural similarity weight to the corresponding feature point vector to form a weighted feature map;

[0130] A skip-layer connection unit is used to stack the original image sub-image to be segmented with the weighted feature map along the feature channel. By setting up a skip-layer connection unit between the encoding path layer and the decoding path layer, the embodiment of the present invention can prevent the loss of added structural similarity features during the training phase, significantly improving the accuracy of pixel classification and thus more accurately segmenting the target.

[0131] The present invention also provides a terminal device, comprising at least one processor and at least one memory communicatively connected to the processor; the memory stores program instructions that can be executed by the processor; when the processor runs the program instructions, it executes the steps of the above-mentioned image segmentation method based on structural similarity measurement.

[0132] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0133] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image segmentation method based on structural similarity measurement, characterized in that: It includes training phase and segmentation phase; The training phase includes: Obtaining a machine vision-based training set and performing preprocessing; the training set includes original training set images and corresponding annotated training set images; Performing a block operation on the training set original image and the training set annotated image to obtain a training set original image sub-image and a training set annotated image sub-image; Construct a U-shaped segmentation network including an encoding path and a decoding path; The U-shaped segmentation network is trained using the original image subgraphs of the training set and the annotated image subgraphs of the training set, the training set segmentation image subgraphs predicted and output by the U-shaped segmentation network are spliced to obtain a complete training set segmentation image, and the cross entropy loss function is used to minimize the difference between the training set segmentation image subgraphs and the training set annotated image subgraphs to optimize the parameters of the U-shaped segmentation network; The segmentation phase includes: Obtain the original image to be segmented based on machine vision, and perform preprocessing and block operations to obtain the sub-images of the original image to be segmented; The original image sub-image to be segmented is input into the trained U-shaped segmentation network to obtain a segmented image sub-image, and the segmented image sub-images belonging to the same original image to be segmented are spliced in a block order to obtain a final segmented image.

2. The image segmentation method based on structural similarity measurement according to claim 1, characterized in that: The expression of the cross entropy loss function is: Among them, N represents the number of pixels, Represents the prediction of pixel i, y (i) Represents the true label corresponding to pixel i.

3. The image segmentation method based on structural similarity measurement according to claim 2, characterized in that: The original image to be segmented or the training set is preprocessed, including: enhancing image contrast using an HSV stretching method.

4. The image segmentation method based on structural similarity measurement according to claim 1, characterized in that: The encoding path includes four repeated structural similarity measurement encoding units; the decoding path includes three repeated decoding units with the same characteristic depth as the encoding path; the structural similarity measurement encoding units and the decoding units of corresponding characteristic depths are connected in series; Each of the structural similarity measurement encoding units includes 2 convolutional layers, 1 pooling layer and 1 structural similarity weight calculation module; Each of the decoding units includes 2 convolutional layers and 1 transposed convolutional layer.

5. The image segmentation method based on structural similarity measurement according to claim 4, characterized in that: During the training phase, the structural similarity weight calculation module performs the following steps: Traversing the feature point vector at each feature point position of the original image sub-image of the training set; Calculating the global feature vector of the original image sub-image of the training set using a global pooling layer; Calculating the structural similarity weights between the global feature vectors and feature point vectors of the training set image sub-images; Adding the structural similarity weight to the corresponding feature point vector to form a weighted feature map; The training set original image sub-image and the weighted feature map are stacked along the direction of the feature channel.

6. The image segmentation method based on structural similarity measurement according to claim 5, characterized in that: The calculation formula of the structural similarity weight is: w mn =l(m,n)c(m,n)u(m,n) Among them, w mn Represents the structural similarity weight; l(m,n), c(m,n) and u(m,n) represent the brightness, contrast and structural similarity between m and n respectively; m represents the global feature vector of the original image sub-image, which is achieved by global average pooling, and n represents the feature point vector at (x,y) in the original image sub-image; μ m Represents the mean of the global eigenvector m, μ n represents the mean of the feature point vector n; σ m represents the standard deviation of the global eigenvector m, σ n Represents the standard deviation of the feature point vector n, σ mn represents the covariance between m and n; C1, C2, C3 are constant terms; The calculation formula of the weighted feature map is: n′=w mn n, Where n′ represents the weighted feature point vector.

7. An image segmentation device based on structural similarity measurement, characterized in that: include: An acquisition module, used to acquire the original image to be segmented based on machine vision; A preprocessing module, used for preprocessing the original image to be segmented; The image segmentation module is used to segment the pre-processed original image to be segmented into blocks to obtain sub-images of the original image to be segmented; A segmentation prediction module is used to segment the original image sub-image to be segmented using the trained U-shaped segmentation network as described in any one of claims 1 to 6 to obtain segmented image sub-images, and to splice the segmented image sub-images belonging to the same original image to be segmented in a block order to obtain a final segmented image.

8. The image segmentation device based on structural similarity measurement according to claim 7, characterized in that: The pre-processing module comprises: RGB-HSV mapping unit, used to map the original image to be segmented from RGB color space to HSV color space; A normalization unit, used for performing minimum-maximum normalization processing on the S channel and V channel of the original image to be segmented in the HSV color space; The mapping unit is used to remap the original image to be segmented from the HSV color space back to the RGB color space.

9. The image segmentation device based on structural similarity measurement according to claim 7, characterized in that: The segmentation prediction module includes a U-shaped network unit and a sub-graph splicing unit; the U-shaped network unit includes: an encoding path layer, a decoding path layer, a supervision unit and a loss function unit; The coding path layer includes four repeated structural similarity measurement coding units; each structural similarity measurement coding unit includes two convolutional layers, one pooling layer and one structural similarity weight calculation module; The decoding path layer includes three repeated decoding units with the same feature depth as the encoding path layer; the structural similarity measurement encoding unit and the decoding unit corresponding to the feature depth are connected in series; each of the decoding units includes two convolutional layers and one transposed convolutional layer; The supervision unit is used to use the original image to be segmented as supervision information; The sub-image stitching unit is used to stitch the segmented sub-images belonging to the same original image to be segmented in a block order to obtain a final segmented image; The loss function unit is used to optimize the parameters of the U-shaped segmentation network during the training phase.

10. A terminal device, characterized in that: The method comprises at least one processor and at least one memory in communication with the processor; the memory stores program instructions executable by the processor; and when the processor runs the program instructions, the method executes the steps of the image segmentation method based on structural similarity measurement according to any one of claims 1 to 6.

Images