A repairing method and device for saliency detection uncertain edge points

Abstract

The application discloses a kind of saliency detection uncertain edge point repair method and device, including the following steps: A, boundary renderer is integrated into codec style saliency detection network, model training is carried out, and optimal model parameter is obtained;B, using the trained model, the threshold segmentation processing is carried out to the edge detection branch output result, and the target class object boundary region is obtained;C, the position generator of uncertain boundary point is used to obtain the predetermined number of saliency uncertain points at target object boundary region;D, based on the position of saliency uncertain point, the rough features and strip features of each saliency uncertain point are fused;E, using multilayer perception, the reclassification is carried out using the fused rough features and strip features.The repair method and device of the application use multilayer perception to reclassify, and replace the reclassification result into initial result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent image processing, and particularly relates to an edge refinement method for saliency detection. BACKGROUND

[0002] In the saliency detection task, saliency detection predicts each pixel. Convolutional neural networks can easily provide encodings of segmentation information for various methods. Fully convolution network (FCN) uses a fully convolutional structure for saliency detection and establishes a skip structure to connect saliency information at different depths in the convolutional layer. In FCN, the mask result is predicted by using 8x, 16x, 32x smaller features than the input through transposed convolution. UNet is also a fully convolutional network with a symmetric architecture on the encoding and decoding of feature maps. UNet uses the same resolution to connect feature maps in the encoder and decoder and uses transposed convolution to restore these features to output the mask result at a higher resolution.

[0003] Although these methods take advantage of the excellent feature extraction capability of the convolution operator, the feature maps that are 8 times or 16 times smaller than the input are too rough to perform segmentation. When these coarse masks are upsampled or resized to the same size as the input image, there is a blur problem on the target class boundary of the mask result, which affects the accuracy of saliency detection. In order to avoid the down-sampling effect in the mask-based model, scholars have proposed some contour-based segmentation models that distinguish objects by contours formed by contour vertices. These models can obtain clear contours with the same resolution as the input image by determining the coordinates of the contour vertices. The Polygon-RNN model and the Polygon-RNN++ model use RNN to find contour vertices one by one. Curve-GCN implements a graph convolution network (GCN) to obtain the coordinates of the contour vertices by regression. Curve-GCN can simultaneously adjust the coordinates of a fixed number of vertices from the initial contour to the target.

[0004] Although the above contour-based segmentation methods avoid the influence by down-sampling and directly restore the resolution, due to the limitation of the fixed number of contour vertices, they cannot provide complex edges, so the segmentation effect is not good for uncertain boundaries. SUMMARY

[0005] In order to solve the technical problems that the prior art cannot provide complex edges due to the limitation of the fixed number of contour vertexes, and the segmentation effect is poor for uncertain boundaries, the present application provides a repairing method and device for saliency detection of uncertain edge points, which improves the edge segmentation quality by adding a boundary renderer of uncertain points of a target class object. In the work of PointRend, the rendering of the segmentation result is learned, which uses the mask score to select the uncertain points around the contour. Unlike PointRend, the boundary renderer of uncertain points of the method obtains a certain number of uncertain points by focusing on the edge area of the target object, combines the coarse feature and strip feature of each point, uses a multilayer perceptron for reclassification, and replaces the reclassification result into the initial result.

[0006] In order to achieve this goal, the present application adopts the following technical solutions.

[0007] A repairing method for saliency detection of uncertain edge points, the method comprising the steps of:

[0008] A, integrating a boundary renderer into a codec style saliency detection network, outputting a high-dimensional feature map obtained by a decoding module to a saliency detection branch and an edge detection branch, performing model training to obtain optimal model parameters;

[0009] B, using the trained model to perform threshold segmentation processing on the output result of the edge detection branch to obtain a target class object boundary region;

[0010] C, using an uncertain boundary point position generator to obtain a predetermined number of saliency uncertain points in the target object boundary region;

[0011] D, based on the position of the saliency uncertain points, fusing the coarse feature and strip feature of each saliency uncertain point;

[0012] E, using a multilayer perceptron to reclassify the fused coarse feature and strip feature.

[0013] In the repairing method for saliency detection of uncertain edge points in the present application, the step of integrating the boundary renderer into the codec style saliency detection network, outputting the high-dimensional feature map obtained by the decoding module to the saliency detection branch and the edge detection branch, and performing model training to obtain the optimal model parameters comprises:

[0014] A1, obtaining the high-dimensional feature map output of the last stage of the decoding module;

[0015] A2, taking the high-dimensional feature map as the input of the saliency detection branch, outputting a saliency detection mask, and calculating a loss value for model parameter updating;

[0016] A3, taking the high-dimensional feature map as an input of the edge detection branch, outputting an edge detection mask, and calculating a loss value for model parameter updating;

[0017] A4, using a multi-layer perception to classify points obtained in a training stage, outputting a saliency score of each point, and calculating a loss value for model parameter updating;

[0018] A5, when the loss values in A2, A3 and A4 converge, obtaining optimal model parameters.

[0019] In the repairing method for saliency detection of uncertain edge points in the application, the loss value in steps A2, A3 and A4 is binary cross entropy, and the calculation method is as follows:

[0020]

[0021] Wherein, y is a true value of a parameter in a training sample, and y is a predicted value.

[0022] In the repairing method for saliency detection of uncertain edge points in the application, the output result of the edge detection branch is processed by threshold segmentation using the trained model, and the target class object boundary region is obtained.

[0023] B1, using a 1*1 convolution to map the high-dimensional feature map through the output result of the edge detection branch into a single-channel feature map;

[0024] B2, compressing the value range of the single-channel feature map to 0-1 to obtain an edge intensity;

[0025] B3, according to a preset threshold, performing threshold segmentation on the edge intensity to obtain the target class object boundary region.

[0026] In the repairing method for saliency detection of uncertain edge points in the application, the position generator of the uncertain boundary point is used to obtain a certain number of saliency uncertain points in the target object boundary region.

[0027] C1, based on the target class object boundary region, filtering out non-boundary regions;

[0028] C2, using a 1*1 convolution to map the high-dimensional feature map through the output result of the saliency detection branch into a single-channel feature map;

[0029] C3, compressing the value range of the single-channel feature map to 0-1 to obtain a saliency score of each point in the boundary region;

[0030] C4, using the saliency score of each point, selecting a predetermined number of most uncertain points.

[0031] The repair method for uncertain edge points in the saliency detection in the application includes the following steps:

[0032] D1, the high-dimensional feature map is mapped to obtain the coarse feature by the saliency detection branch output result through 1*1 convolution;

[0033] D2, the high-dimensional strip feature is obtained by mapping the high-dimensional feature map through the edge detection branch output result by using the strip convolution module;

[0034] D3, the coarse feature and the strip feature of each saliency uncertain point are fused.

[0035] In the repair method for uncertain edge points in the saliency detection in the application, the high-dimensional strip feature is obtained by mapping the high-dimensional feature map through the edge detection branch output result by using the strip convolution module, and includes the following steps:

[0036] The strip convolution module is used to capture context information from four different directions of horizontal, vertical, left diagonal and right diagonal. After the input tensor of the strip convolution module is input into four paths through 1*1 convolution, each path includes a strip convolution of a shape, and then the outputs of the four strip convolutions are connected and mapped, and then up-sampling operation and convolution are performed to obtain the output of the strip convolution module.

[0037] The application further includes a repair device for uncertain edge points in the saliency detection, which includes a boundary renderer training module, an edge segmentation module, a position generator of uncertain boundary points, a feature fusion module and a reclassification module, wherein,

[0038] The boundary renderer training module is integrated into a coding and decoding style saliency detection network, and is used for outputting the high-dimensional feature map obtained by the decoding module to the saliency detection branch and the edge detection branch to perform model training and obtain optimal model parameters;

[0039] The edge segmentation module uses the trained model to perform threshold segmentation processing on the edge detection branch output result to obtain a target class object boundary region;

[0040] The position generator of uncertain boundary points is used to obtain a certain number of saliency uncertain points in the target object boundary region;

[0041] The feature fusion module is used to fuse the coarse feature and the strip feature of each saliency uncertain point based on the position of the saliency uncertain point;

[0042] The reclassification module is used to perform reclassification using the multi-layer perception and the fused coarse feature and strip feature.

[0043] The repair device for saliency detection of uncertain edge points in the application comprises a boundary renderer training module, a high-dimensional feature map acquisition unit, a saliency detection branch, an edge detection branch, a multi-layer perception and an optimal model parameter acquisition unit, wherein,

[0044] The high-dimensional feature map acquisition unit is used to acquire the high-dimensional feature map output of the last stage of the decoding module;

[0045] The saliency detection branch is used to take the high-dimensional feature map as input, output a saliency detection mask, and calculate a loss value for model parameter updating;

[0046] The edge detection branch is used to take the high-dimensional feature map as input, output an edge detection mask, and calculate a loss value for model parameter updating;

[0047] The multi-layer perception is used to classify the points obtained in the training stage, output the saliency score of each point, and calculate a loss value for model parameter updating;

[0048] The optimal model parameter acquisition unit is used to obtain the optimal model parameters when the loss values in A2, A3 and A4 converge. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 It is a flowchart of the repair method for saliency detection of uncertain edge points according to the embodiment of the application.

[0050] Figure 2 It is a processing schematic diagram of the repair method for saliency detection of uncertain edge points according to the embodiment of the application.

[0051] Figure 3 It is a strip convolution processing schematic diagram of the repair method for saliency detection of uncertain edge points according to the embodiment of the application.

[0052] Figure 4 It is a selection processing schematic diagram of uncertain boundary points of the repair method for saliency detection of uncertain edge points according to the embodiment of the application.

[0053] Figure 5 It is an application result schematic diagram of the repair method for saliency detection of uncertain edge points according to the embodiment of the application. DETAILED DESCRIPTION

[0054] The application will be described in detail below with reference to the accompanying drawings.

[0055] The following detailed exemplary embodiments are disclosed. However, the specific structural and functional details disclosed herein are merely for the purpose of describing exemplary embodiments.

[0056] It is to be understood, however, that the application is not limited to the particular exemplary implementations disclosed herein, but rather, the full scope of the application will be set forth in the claims appended hereto. In the description of the drawings, like parts are marked with the same reference numerals throughout the several views.

[0057] Refer to the structure, proportion, size, etc. shown in the drawings accompanying the present specification, which are only used to cooperate with the content disclosed in the present specification, to be understood and read by those skilled in the art, and not to define the limited conditions for the implementation of the present application, so it does not have technical significance. Any modification of structure, change of proportion relationship or adjustment of size, which does not affect the effect and purpose that can be achieved by the present application, should still fall within the scope of the technology disclosed by the present application. At the same time, the positional limitation terms referred to in the present specification are only for the purpose of clear description, and not to limit the scope of the present application, and the change or adjustment of the relative relationship is also considered as the implementation of the present application.

[0058] It should also be understood that the term "and / or" as used herein includes any and all combinations of one or more of the associated listed items. In addition, it should be understood that when a component or element is referred to as being "connected" or "coupled" to another component or element, it can be directly connected or coupled to the other component or element, or there can be intervening components or elements. In addition, other words used to describe the relationship between components or elements should be understood in the same way (for example, "between" versus "directly between", "adjacent" versus "directly adjacent", etc.).

[0059] In the prior art, when a target class is segmented by using a saliency detection model, a large number of points with uncertain probability appear in the boundary of the segmented object, which affects the saliency detection accuracy.

[0060] To solve the above problems, the embodiment of the present application provides an algorithm module for repairing uncertain edge points of saliency detection. The method first generates an uncertain point list by using an uncertain boundary point position generator, then fuses the rough features of each point and the strip features with remote context information, and finally reclassifies the fused features by using a multilayer perception machine, and replaces the reclassification results into the initial results.

[0061] The various embodiments of the present application will be described in detail below with reference to the accompanying drawings. Figure 1 A flowchart of a repairing method for uncertain edge points of saliency detection according to the specific embodiment of the present application is shown. Figure 2 A processing diagram of a repairing method for uncertain edge points of saliency detection according to the specific embodiment of the present application is shown.

[0062] Specifically, in one specific embodiment of the present application, a repairing method for saliency detection uncertain edge points is provided, as shown in the flowchart included in Figure 1 The method specifically comprises the following steps.

[0063] S1, integrate the boundary renderer into the codec style saliency detection network, and perform model training to obtain optimal model parameters.

[0064] Further, in the specific embodiment of the present application, as shown in Figure 2 , the optimal parameters of the integrated model mainly include the following steps:

[0065] S11, obtain the high-dimensional feature map output of the last stage of the decoding module.

[0066] S12, take the high-dimensional feature map obtained in S11 as the input of the saliency detection branch, output a saliency detection mask, and calculate a loss value for model parameter updating.

[0067] The saliency coefficient is used to measure the similarity between two groups of different scale feature maps. When the similarity is high, that is, the target inheritance between the feature maps of different scales before and after is good, the saliency coefficient of the corresponding region is large, otherwise, the saliency coefficient is small. The multiplication process of the saliency coefficient map and the feature map can be regarded as the process of re-distribution of the weight of each component of the image, which can highlight the key area and suppress the flat background.

[0068] On the basis of having obtained the saliency coefficient, binaryzation and local field mean determination processing are performed on the saliency coefficient, so as to obtain a saliency detection mask capable of eliminating the flat background.

[0069] Therefore, in the specific embodiment of the present application, the saliency feature extraction method is used to enhance the local features of the image, which is beneficial to improve the target detection performance; and the detection mask generated by the saliency coefficient is used to screen the feature map, so as to exclude the flat background area and only generate a predicted target in the effective area of the image, which can improve the execution efficiency of target detection.

[0070] In the image processing adopted in the specific embodiment of the present application, the convolution kernel refers to a weighted matrix with a predetermined specification, and the weighted matrix is used for convolution filtering: the convolution kernel matrix is rotated first, and then each corresponding region on the high-dimensional feature map detection result is sequentially slid, and the sum of the products of the corresponding sliding regions and the corresponding elements of the convolution kernel is taken as the new value on the corresponding region of the high-dimensional feature map detection result.

[0071] In the specific embodiment, a 3*3 convolution kernel and a sigmoid activation function are used to obtain the saliency mask Ms of the target object:

[0072] M sSigmoid(conv(SODB(F d ))),

[0073] wherein the high-dimensional feature map Fd of the last stage of the decoding module is subjected to a salient object detection block (SODB) and then subjected to a convolution function conv and a Sigmoid function, and the expression of the Sigmoid function is f(x)=1 / (1+e -x When x tends to negative infinity, y tends to 0; when x tends to positive infinity, y tends to 1; when x=0, y=1 / 2.

[0074] S13, taking the high-dimensional feature map obtained in S11 as the input of the edge detection branch, outputting an edge mask, and calculating a loss value for model parameter updating;

[0075] An edge refers to a part with the most significant local intensity change in an image. It mainly exists between targets and targets, targets and backgrounds, and regions and regions (including different colors), and is an important basis for image analysis such as image segmentation, texture features, and shape features. The edge detection is performed on the high-dimensional feature map, and the resulting image has the same band name in the high-dimensional feature map, and a non-zero region indicates an edge region.

[0076] Similarly, in the specific embodiment of the present application, a 3*3 convolution kernel and a sigmoid activation function are used to obtain an edge mask Mb of the target object.

[0077] M b =Sigmoid(conv(BDB(F d ))),

[0078] wherein the high-dimensional feature map Fd of the last stage of the decoding module is subjected to a salient object detection block (SODB) and then subjected to a convolution function conv and a Sigmoid function, and the expression of the Sigmoid function is f(x)=1 / (1+e

[0079] S14, the multilayer perceptron classifies the points obtained in the training stage, outputs the saliency score of each point, and calculates a loss value for model parameter updating.

[0080] Specifically, the loss function involved in S12, S13 and S14 is binary cross-entropy, and the calculation formula is:

[0081]

[0082] wherein y is a true value, and x is a predicted value.

[0083] S15, obtaining the optimal model parameters when the loss value converges.

[0084] S2, performing threshold segmentation processing on the edge detection branch output result to obtain a target class object boundary region. The specific steps are as follows:

[0085] S21, using 1x1 convolution to map the high-dimensional edge detection branch output feature map into a single-channel feature map;

[0086] S22, using a sigmoid function to compress the value range of the above feature map to [0, 1] to obtain edge intensity.

[0087] S23, setting a threshold value of a hyperparameter according to a predetermined target to perform threshold segmentation. For example, in one specific embodiment of the present application, the threshold value is set to 0.5.

[0088] S3, using an uncertain boundary point position generator to obtain a certain number of saliency uncertain points at the target object boundary region. The specific steps are as follows:

[0089] S31, filtering out non-boundary regions based on the boundary region obtained in S2;

[0090] S32, using 1x1 convolution to map the high-dimensional saliency detection branch output feature map into a single-channel feature map;

[0091] S33, using a sigmoid function to compress the value range of the above feature map to [0, 1] to obtain a saliency score;

[0092] S34, selecting N most uncertain points in the saliency detection result according to the score The formula used is as follows:

[0093]

[0094] where S(n i ) represents the probability value of point n i on the saliency mask M s , and B(n i ) represents the probability value of point n i on the edge mask M b .

[0095] Therefore, the uncertain boundary point processing provides K uncertain boundary points and a set of high-dimensional strip feature maps with remote context information generated by the edge detection processing for boundary rendering.

[0096] S4, based on the position of the uncertain points, fusing the rough features and strip features of each point. The specific steps are as follows:

[0097] S41, obtain the high-dimensional feature map of the saliency detection branch, map it using 1x1 convolution, and obtain the rough feature F c .

[0098] S42, obtain the high-dimensional strip feature F output by the strip convolution module in the edge detection branch s .

[0099] Figure 3 This is a schematic diagram of the strip convolution processing for the repair method for uncertain edge points in saliency detection according to an embodiment of the present application. In steps S41 and S42, the high-dimensional strip feature map is captured by the strip convolution module from four different directions: horizontal, vertical, left diagonal, and right diagonal. Let x e R H×W×C represent the input tensor of the strip convolution module, where H, W, and C represent the height, width, and number of channels. In the strip convolution block, X is input into four parallel paths after 1x1 convolution, and each path contains a strip convolution with a shape. Then the output features of the four strip convolutions are connected, and then up-sampling operation and 1x1 convolution are performed to obtain the output of the strip convolution block.

[0100] The definition of the strip convolution used is as follows:

[0101]

[0102] where w e R 2K+1 represents a convolution kernel with a size of 2K+1, D=(D h ,D w ) represents the direction of w, where (1,1), (-1,1), (0,1), and (1,0) represent the left diagonal, right diagonal, horizontal, and vertical, respectively. Z D e R H×W×C′ represents the direction of the strip convolution, and X*w represents the result of the convolution operation. For the filter w, k=4 is set in an embodiment of the present application, so that each strip convolution has 9 parameters, which is the same as a 3*3 convolution filter.

[0103] In the above strip convolution module, each position in the output feature map is allowed to establish a relationship with multiple positions in the input feature map in four directions. The four directions selected are consistent with the distribution of most boundaries in the input image, and are relatively easy to implement.

[0104] The sample points of the edge renderer are generated based on the edge points provided by the uncertain point generator, the feature information of the points is extracted according to the position information of the points, and then a multi-layer perception machine composed of 1*1 convolution is used to predict the foreground probability of the sample points. Finally, the new scores of the sample points are replaced into the initial saliency mask to obtain the refined result.

[0105] Figure 4 The selection of uncertain boundary points for the repairing method of uncertain edge points for saliency detection in the embodiment of the present application is shown in the schematic diagram. In the training and testing phase, the sampling points can be processed respectively by two kinds of sampling point rules. In the training phase, in order to increase the diversity of the sampling points, the sampling points are composed of three parts: (1) random sampling, the embodiment of the present application randomly samples KN (K>1) uncertain candidate points from a uniform distribution; (2) uncertain boundary sampling: βN (N>1) sampling points are generated by the uncertain boundary point generator; (3) non-boundary region sampling: (1-β)N uncertain points are sampled from the non-boundary region.

[0106] In order to balance the accuracy and training complexity, a slightly biased sampling strategy is adopted in one embodiment of the present application, for example, N=18 2 , k=3, β=0.75. In the testing phase, since the gradient does not need to be calculated, more points are used to obtain a denser prediction, and these points are all from the uncertain boundary point generator.

[0107] S43, the features obtained in S41 and S42 are fused, and the fusion formula is as follows:

[0108] F fusion = Concatenate (F s , F c ),

[0109] S5, the fused features are reclassified by using a multi-layer perception, and the reclassified features are replaced into the initial result, and the final comparison result is shown in Figure 5 .

[0110] The boundary renderer optimizes the segmentation result based on the reclassification of the sampling points. Specifically, in the embodiment of the present application, the strip feature map Fs generated by the strip convolution module and the coarse feature map Fc generated by the saliency detection branch are fused and input into a multi-layer perception, the multi-layer perception predicts the scores of the corresponding points and replaces the points generated by the boundary renderer into the initial mask, thereby recovering the high resolution while ensuring the details of the complex object boundary. The multi-layer perception aims to refine the features of the uncertain boundary points.

[0111] S point =Sigmoid (MLP (F fusion )),

[0112] MLP is a multi-layer perception, which introduces one to multiple hidden layers between the input layer and the output layer on the basis of a single-layer neural network.

[0113] Corresponding to the repairing method for uncertain edge points in saliency detection in the embodiments of the present application, the embodiments of the present application also include a repairing device for uncertain edge points in saliency detection, which comprises a boundary renderer training module, an edge segmentation module, an uncertain boundary point position generator, a feature fusion module and a reclassification module, wherein,

[0114] The boundary renderer training module is integrated into a codec style saliency detection network, and is used for outputting a high-dimensional feature map obtained by a decoding module to a saliency detection branch and an edge detection branch to perform model training and obtain optimal model parameters;

[0115] The edge segmentation module uses the trained model to perform threshold segmentation processing on the output result of the edge detection branch to obtain a target class object boundary region;

[0116] The uncertain boundary point position generator uses the uncertain boundary point position generator to obtain a certain number of saliency uncertain points at the target object boundary region;

[0117] The feature fusion module is used for fusing the coarse features and the strip features of each saliency uncertain point based on the position of the saliency uncertain point;

[0118] The reclassification module is used for using a multi-layer perception to perform reclassification using the fused coarse features and strip features.

[0119] In the repairing device for uncertain edge points in saliency detection in the present application, the boundary renderer training module comprises a high-dimensional feature map acquisition unit, a saliency detection branch, an edge detection branch, a multi-layer perception and an optimal model parameter acquisition unit, wherein,

[0120] The high-dimensional feature map acquisition unit is used for acquiring the high-dimensional feature map output of the last stage of the decoding module;

[0121] The saliency detection branch is used for taking the high-dimensional feature map as input, outputting a saliency detection mask and calculating a loss value for model parameter updating;

[0122] The edge detection branch is used for taking the high-dimensional feature map as input, outputting an edge detection mask and calculating a loss value for model parameter updating;

[0123] The multi-layer perception is used for classifying the points obtained in the training stage, outputting the saliency score of each point and calculating a loss value for model parameter updating;

[0124] The optimal model parameter acquisition unit is used for obtaining the optimal model parameters when the loss values in A2, A3 and A4 converge.

[0125] The above description shows and describes several preferred embodiments of the present application, and the description of the embodiments of the present application is only a typical example of the technical process of the present application under the current technical conditions, and there is still a lot of room for optimization and improvement without deviating from the technical principles, steps, functions, applications and implementation framework of the present application. These improvements, optimizations, etc. are also considered to be within the protection scope of the present patent. Therefore, as mentioned above, it should be understood that the present application is not limited to the forms disclosed in the present specification and should not be considered as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the inventive concept described in the present specification by the above teachings or related art or knowledge. Any modification and change made by those skilled in the art without deviating from the spirit and scope of the present application shall be within the protection scope of the claims of the present application.

Claims

1. A repairing method for saliency detection of uncertain edge points, characterized in that, The method comprises the steps of: A, integrating a boundary renderer into a codec style saliency detection network, outputting a high-dimensional feature map obtained by a decoding module to a saliency detection branch and an edge detection branch, performing model training, and obtaining optimal model parameters; B, using the trained model to perform threshold segmentation processing on the output result of the edge detection branch to obtain a target class object boundary region; C, using an uncertain boundary point position generator to obtain a certain number of saliency uncertain points at the target object boundary region; D, based on the position of the saliency uncertain points, fusing the coarse features and strip features of each saliency uncertain point; E, using a multi-layer perception to reclassify the fused coarse features and strip features; wherein the step of integrating the boundary renderer into the codec style saliency detection network, outputting the high-dimensional feature map obtained by the decoding module to the saliency detection branch and the edge detection branch, performing model training, and obtaining the optimal model parameters comprises: A1, obtaining a high-dimensional feature map output of a last stage of the decoding module; A2, taking the high-dimensional feature map as an input of the saliency detection branch, outputting a saliency detection mask, and calculating a loss value for model parameter updating; A3, taking the high-dimensional feature map as an input of the edge detection branch, outputting an edge detection mask, and calculating a loss value for model parameter updating; A4, using a multi-layer perception to classify the points obtained in the training stage, outputting a saliency score of each point, and calculating a loss value for model parameter updating; A5, when the loss values in A2, A3 and A4 converge, obtaining the optimal model parameters; wherein the loss values in steps A2, A3 and A4 are binary cross-entropy, and the calculation method is: ； wherein y is the true value of the parameter in the training sample, is the predicted value.

2. The repairing method for saliency detection of uncertain edge points as claimed in claim 1 wherein, using the trained model to perform threshold segmentation processing on the output result of the edge detection branch to obtain the target class object boundary region comprises: B1, using a 1x1 convolution to map the high-dimensional feature map through the edge detection branch output result into a single-channel feature map; B2, compressing the value range of the single-channel feature map to 0-1 to obtain an edge intensity; B3, according to a preset threshold, performing threshold segmentation on the edge intensity to obtain the target class object boundary region.

3. The repairing method for saliency detection of uncertain edge points as claimed in claim 1 wherein, using the uncertain boundary point position generator to obtain a certain number of saliency uncertain points at the target object boundary region comprises: C1, based on the target class object boundary region, filtering out non-boundary regions; C2, using a 1x1 convolution to map the high-dimensional feature map through the saliency detection branch output result into a single-channel feature map; C3, compressing the value range of the single-channel feature map to 0-1 to obtain a saliency score of each point in the boundary region; C4, using the saliency scores of the points to select a certain number of most uncertain points.

4. The repairing method for saliency detection of uncertain edge points as claimed in claim 1 wherein, based on the position of the saliency uncertain points, fusing the coarse features and strip features of each saliency uncertain point comprises: D1, using a 1x1 convolution to map the high-dimensional feature map through the saliency detection branch output result to obtain coarse features; D2, using a strip convolution module to map the high-dimensional feature map through the edge detection branch output result to obtain high-dimensional strip features; D3, fusing the coarse features and strip features of each saliency uncertain point.

5. The repairing method for saliency detection of uncertain edge points as claimed in claim 4 wherein, The high-dimensional strip features are obtained by mapping the edge detection branch output results of the high-dimensional feature maps using the strip convolution module, including: The strip convolution module is used to capture context information from four different directions, i.e., horizontal, vertical, left diagonal and right diagonal. The input tensor of the strip convolution module is input into four paths after 1*1 convolution, each path includes a strip convolution of a certain shape, then the outputs of the four strip convolutions are connected and mapped, and then up-sampling operation and convolution are performed to obtain the output of the strip convolution module.

6. A repairing device for saliency detection of uncertain edge points, comprising a boundary renderer training module, an edge segmentation module, an uncertain boundary point position generator, a feature fusion module and a reclassification module, wherein, The boundary renderer training module is integrated into a codec style saliency detection network, and is used to output the high-dimensional feature maps obtained by the decoding module to the saliency detection branch and the edge detection branch, and perform model training to obtain optimal model parameters; The edge segmentation module uses the trained model to perform threshold segmentation processing on the edge detection branch output results to obtain a target class object boundary region; The uncertain boundary point position generator uses the uncertain boundary point position generator to obtain a certain number of saliency uncertain points in the target object boundary region; The feature fusion module is used to fuse the coarse features and strip features of each saliency uncertain point based on the position of the saliency uncertain point; The reclassification module is used to use a multi-layer perceptron to reclassify the fused coarse features and strip features; The boundary renderer training module comprises a high-dimensional feature map acquisition unit, a saliency detection branch, an edge detection branch, a multi-layer perceptron and an optimal model parameter acquisition unit, wherein, The high-dimensional feature map acquisition unit is used to acquire the high-dimensional feature map output of the last stage of the decoding module; The saliency detection branch is used to input the high-dimensional feature map as an input, output a saliency detection mask, and calculate a loss value for model parameter updating; The edge detection branch is used to input the high-dimensional feature map as an input, output an edge detection mask, and calculate a loss value for model parameter updating; The multi-layer perceptron classifies the points obtained in the training stage, outputs the saliency score of each point, and calculates a loss value for model parameter updating; The optimal model parameter acquisition unit is used to obtain the optimal model parameters when the loss values in A2, A3 and A4 converge.

Images