Visual priori knowledge-based saliency target sorting method

Abstract

A saliency target sorting method based on visual priori knowledge constructs a saliency target sorting network in an offline stage, and is used for identifying each saliency target in an image after being trained by a data set based on visual priori knowledge and generating a result based on target saliency sorting. The method can automatically generate a generalized significance target sorting result according with a human objective observation rule, and provides a more practical technical support for applications in the fields of intelligent auxiliary tools, controllable image description generation, man-machine interaction, auxiliary decision making, intelligent monitoring, automatic driving, augmented reality and the like of visually impaired patients.

Description

Technical Field

[0001] The present invention relates to a technology in the field of image processing, specifically a method for ranking salient objects based on visual prior knowledge. Background Art

[0002] Existing deep learning-based salient object ranking technologies are highly dependent on datasets and lack the integration of prior knowledge. As a result, the generated results often do not conform to the natural laws of human observation, thus affecting their practicality. Moreover, when faced with new scenarios or new datasets, these technologies usually exhibit poor cross-domain adaptability, resulting in insufficient generalization capabilities and limiting their scope of application. Summary of the Invention

[0003] In response to the above-mentioned deficiencies in the existing technology, the present invention proposes a salient target ranking method based on visual prior knowledge, which can automatically generate generalized salient target ranking results that conform to the objective observation laws of humans, and provide more practical technical support for applications in the fields of intelligent auxiliary tools for visually impaired patients, controllable image description generation, human-computer interaction, decision support, intelligent monitoring, autonomous driving, augmented reality, etc.

[0004] The present invention is achieved through the following technical solutions:

[0005] The present invention relates to a salient object ranking method based on visual prior knowledge. A salient object ranking network is constructed in an offline stage. After being trained on a data set based on visual prior knowledge, the network is used to identify salient objects in an image and generate results based on object saliency ranking.

[0006] The salient object ranking network includes: a backbone feature extraction network, a feature pyramid network (FPN), a region proposal network (RPN), a region of interest dual alignment structure and a salient object ranking head, wherein: the backbone feature extraction network extracts features from the input image, and obtains feature maps at different scales through multiple convolution, pooling and residual operations and inputs them into the FPN; the FPN fuses the multi-scale feature maps from top to bottom paths and lateral connections to obtain an updated feature map that integrates multi-scale information and feeds it to the RPN; the RPN uses a sliding window to extract salient object candidate regions from the updated feature map; the region of interest dual alignment structure realizes the mapping between the salient object candidate regions and the updated feature map through the region of interest alignment operation to obtain a fine-grained feature map; the salient object ranking head identifies each salient object in the fine-grained feature map based on the salient object candidate regions and the fine-grained feature map and generates a result based on object saliency ranking.

[0007] The backbone feature extraction network uses but is not limited to Resnet 101 to extract deep image features through convolution operations.

[0008] The mapping refers to mapping the bounding box of the salient target candidate region output by RPN to the updated feature map output by FPN, specifically converting the bounding box of the salient target candidate region from the original image space to the feature map space of different layers in the feature pyramid, and obtaining the coordinates of the salient target candidate region corresponding to the feature map space, where: the feature map P2 of the second layer and the feature map P of the kth layer in the feature pyramid are k The features of the salient target candidate regions are resampled according to their corresponding coordinates in the feature map space, and then mapped to the same spatial dimension through the region of interest alignment operation. After the splicing operation, they are sent to the salient target sorting head.

[0009] The k-th layer feature map P k The calculation process is: Where: P k Represents the k-th layer feature map in the feature pyramid, and its minimum value is 3; 224 is the standard ImageNet pre-training image size, w, h are the width and height of the region of interest, respectively. Indicates rounding down.

[0010] The salient object ranking head includes: a feature bottleneck layer, a region of interest embedding layer and a multi-label classification layer, wherein: the feature bottleneck layer integrates and downsamples the extracted feature maps through convolution operations to reduce feature dimensions and extract key features; the region of interest embedding layer uses the bounding box coordinate information of the salient object candidate area to embed the contextual relationship between the object and the scene into the features, thereby enhancing the feature representation of the salient object; the multi-label classification layer classifies the object according to multiple attributes based on visual contrast, spatial position and object size, and outputs the saliency ranking result.

[0011] The integration and downsampling are specifically as follows: after de-integrating and downsampling the feature map through [1×1, 3×3] and [1×1, 7×7] convolutions, the output feature map is reduced to only 128 dimensions after passing through a [1×1] convolution layer.

[0012] The region of interest embedding is specifically as follows: after two [3×3] convolutions, two fully connected layers are used to embed the bounding box coordinate information of the salient target candidate region into the feature map.

[0013] The multi-attribute classification is specifically as follows: based on the output features from the feature bottleneck layer and the region of interest embedding layer, based on but not limited to three significant feature clues, three label classification heads are used to obtain multi-label classification results V = {V c ,V l ,V s}, where: Vc ,V l ,V s The classification is carried out according to the visual contrast, spatial position and target size attributes.

[0014] The significance ranking result is specifically: Rank_level = W·V, where: W is the significance ranking weight vector of contrast, spatial position, and target size from high to low, W = [2 2 ,2 1 ,2 0 ], that is, the highest significance ranking level is level 7 (1×2 2 +1×2 1 +1×2 0 ), the minimum is level 0 (0×2 2 +0×2 1 +0×2 0 );· is the inner product of the vectors.

[0015] The loss function of the training is preferably a cross entropy loss function, specifically: Loss = 1 hassaliency ∑ RoI ∑ labels 1 positiveroi CrossEntropy(y true ,y pred ), where: y true represents the true label, y pred Represents the predicted label. 1 hassaliency Is a binary indicator used to indicate whether the target in the image has a saliency level. When the saliency level of the target in the image is marked, 1 hassaliency is equal to 1, the parameters of the entire model are optimized through back propagation. Otherwise, 1 hassaliency Equal to 0, and only optimize the network structure before the saliency ranking head.

[0016] The prior knowledge-based dataset uses an open-source image dataset. Based on the contrast, position, and size attributes of objects in a bottom-up visual attention mechanism, the probabilities of these prior attributes are calculated for each object in the image to obtain a weight vector. This vector is then combined with a hard-coded result thresholded to 0 or 1 to obtain the ground-truth annotation of each object's saliency ranking. The image annotations also include each object's classification category, image ID, a localization box for each object, and a mask for each object. The data files are in JSON format. Technical Effects

[0017] The present invention obtains a more generalized significance ranking result by constructing a multi-label classification layer including multiple visual prior significance attributes including visual contrast, spatial position and target size. At the same time, the present invention uses a double alignment structure of the region of interest to calculate the relationship between the area of ​​the region of interest and the level of the feature map to extract more fine-grained features, thereby further improving the performance of the network. Compared with the existing technology, the present invention can generate a generalized significance target ranking that conforms to the natural observation laws of humans. It can be used for intelligent auxiliary tools, intelligent monitoring, and autonomous driving for visually impaired patients. It also broadens the application boundaries of computer vision technology in image description production, human-computer interaction, decision support, augmented reality and other fields. By integrating visual prior knowledge, this method effectively avoids the problems of traditional significance target algorithms that do not conform to the natural observation laws of humans and have insufficient generalization capabilities, and provides new ideas and solutions for the intelligent and humanized development of computer vision technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Flowchart of the present invention;

[0019] Figure 2 Schematic diagram of the network structure of significant target ranking in the embodiment;

[0020] Figure 3 is a flow chart of an embodiment;

[0021] Figure 4 This is an example diagram of the data set of the embodiment;

[0022] Figure 5 Schematic diagram of the embodiment effect. DETAILED DESCRIPTION

[0023] like Figure 1As shown in the figure, a salient object ranking method based on visual prior knowledge involved in this embodiment includes: a backbone feature extraction network, a feature pyramid network (FPN), a region proposal network (RPN), a region of interest dual alignment structure, and a salient object ranking head. The backbone feature extraction network extracts features from the input image and, after multiple convolution, pooling, residual operations, outputs feature maps at different scales and inputs them to the FPN. The FPN fuses the input multi-scale feature maps through a top-down path and lateral connections, and outputs a new feature map that integrates multi-scale information to the RPN. The RPN extracts possible salient object candidate regions in the image through a sliding window based on the input. The region of interest dual alignment structure simultaneously receives the feature map processed by the FPN and the salient object candidate regions processed by the RPN, and achieves mapping between the salient object candidate regions and the feature map through the region of interest alignment operation. The salient object ranking head simultaneously receives the salient object candidate regions processed by the RPN and the feature map processed by the region of interest dual alignment structure, and outputs the salient object ranking result.

[0024] like Figure 2 As shown in Figure 1, the backbone feature extraction network is a convolutional neural network (taking ResNet101 as an example) as a feature extractor. Through multiple convolutional layers, pooling layers, and residual connections, it gradually extracts high-level semantic features of the image and outputs feature maps at different scales. These feature maps are input into the FPN.

[0025] like Figure 2 As shown in Figure 2-b, a feature pyramid network (FPN) is introduced to fuse feature maps at different levels through a top-down path and a lateral connection mechanism to enhance the extraction and fusion of multi-scale features, and output new feature maps P2 to P6 that fuse multi-scale information. These feature maps are input into the RPN.

[0026] like Figure 2 As shown in Figure 1-c, the region proposal network (RPN) is introduced. It is based on the feature maps P2 to P6 output by FPN, selects regions through a sliding window, and outputs a series of bounding boxes that may contain candidate regions of salient objects, which are used for the subsequent double alignment structure of the region of interest and the salient object sorting head.

[0027] like Figure 2-d, the double alignment structure of the region of interest realizes the mapping between the salient target candidate region and the feature map, which is used to add additional fine-grained features to the network. Specifically, the bounding box of the salient target candidate region output from the RPN network is mapped to the feature map output from the FPN network. This involves converting the bounding box of the salient target candidate region from the original image space to the feature map space of different layers in the feature pyramid, and obtaining the coordinates of the salient target candidate region corresponding to the feature map space. Among them, the feature map P2 from the second layer and the feature map P kth layer in the feature pyramid are k The features of the salient target candidate regions are resampled according to their corresponding coordinates in the feature map space, and then mapped to the same spatial dimension through the region of interest alignment operation. After the splicing operation, they are sent to the salient target sorting head.

[0028] like Figure 2 As shown in -e, the feature bottleneck layer and the ROI embedding layer respectively receive the feature map processed by the ROI double alignment structure and the salient target candidate region processed by RPN. The new feature maps processed by each are spliced ​​and sent to the multi-label classification layer to output the significance ranking result.

[0029] The feature bottleneck layer receives the feature map processed by the double alignment structure of the region of interest, integrates and downsamples the feature map output by the FPN through [1×1, 3×3] and [1×1, 7×7] convolutions, and then passes through a [1×1] convolution layer to reduce the feature map to 128 dimensions.

[0030] The region of interest embedding layer receives the salient target candidate region after RPN processing, and after two [3×3] convolutions, uses two fully connected layers to embed the bounding box coordinate information of the salient target candidate region into the feature map.

[0031] The multi-label classification layer receives the feature map spliced ​​after processing by the feature bottleneck layer and the region of interest embedding layer, and obtains the multi-label classification results V = {V c ,V l ,V s}, where: V c ,V l ,V s The classification is carried out according to the visual contrast, spatial position and target size attributes.

[0032] The significance ranking result is specifically: Rank_level = W·V, where: W is the significance ranking weight vector of contrast, spatial position, and target size from high to low, W = [2 2 ,2 1 ,20 ], that is, the highest significance ranking level is level 7 (1×2 2 +1×2 1 +1×2 0 ), the minimum is level 0 (0×2 2 +0×2 1 +0×2 0 );· is the inner product of the vectors.

[0033] like Figure 3 As shown in FIG, the salient object ranking method based on visual prior knowledge of this embodiment includes: Step 1: Construct a data set, including:

[0034] 1.1 Using the contrast, position, and size priors of the target in the bottom-up visual attention mechanism, we construct a corresponding probability calculation model for the three image attributes to accurately quantify the salience of each target on different attributes. These probability values ​​are then thresholded (hard-coded) to 0 or 1 to indicate whether the current target should have the corresponding image attribute.

[0035] The contrast attribute is calculated by comparing the color difference between the current marked object and the surrounding area to calculate the probability or confidence of the contrast attribute. The larger the difference ratio, the higher the probability or confidence (Prob contrast ) is higher. Specifically: Where: O and C represent the number of pixels of the object and the object contour respectively. pix represents pixels, pix∈object represents pixels belonging to the object, and pix∈contour represents pixels belonging to the edge contour. and The distribution histogram of the three channels representing the target and the local area around the target, where c represents the color R, G, and B channels to be taken. pixvalue Represents the pixel value of the edge contour of the target, * represents the pixel point of the four neighborhoods, that is, *∈4-neighbor, G pix,* Represents the four neighborhoods of the target contour pixel. ||·||2 is the L2 norm, and |·| represents the absolute value calculation. The first half of the formula is used to quantify the probability of contrast. In the formula, To represent the color vector of the target in R, G, and B channels. Then, use the distribution histogram in an area with the same aspect ratio and twice the area of ​​the original target area The color vector of the local area surrounding the target is then used. The inner product and L2 norm of the two vectors are then calculated to measure their similarity. A higher similarity indicates lower contrast between the target and its surrounding area. Furthermore, the second half of the formula quantifies the difference in value between each pixel in the target's outline and its surroundings; a greater difference indicates higher contrast. The sum of these two components represents the total probability of the contrast attribute.

[0036] The position attribute is used to measure the spatial position of the labeled target compared to the center area of ​​the image. The closer the target is to the center area of ​​the image, the higher the probability or confidence level of the target with the position attribute (Prob loc ) is higher. Specifically: Where: O, E represent the total number of pixels in the target area and the elliptical area respectively. object represents a significant target. center w , center h , center x , center y They represent the horizontal axis length, vertical axis length, horizontal coordinates of pixels in the area, and vertical coordinates of pixels in the area, respectively. x and pix y Represent the horizontal and vertical coordinates of the pixels within the annotated object, respectively. The specific calculation method is shown in the formula. An elliptical region coinciding with the image center represents the image center. The final location attribute confidence of the object is determined by the average contribution of all pixels in the object.

[0037] The size attribute is used to measure the proportion of the labeled object in the entire image. Generally, the higher the proportion, the higher the probability or confidence of the size attribute. size ), specifically: Where: image h and image w Represent the height and width of the image respectively. pix represents pixels, and object represents a salient object.

[0038] 1.2 Based on the original MS-COCO dataset annotation, the probability values ​​corresponding to the three attributes of the annotated target are calculated through step 1.1, and then each attribute is converted into 0 or 1 by hard coding according to the preset threshold (usually 0.5) to obtain the training dataset. Specifically, the saliency level of the target is sorted in the absolute order of contrast, spatial position, and size from high to low. The sorting weight vector of the three image attributes is [2 2 ,2 1,2 0 Based on the hard-coded results of the three attributes and their corresponding weight vectors, the dataset annotation accuracy was ensured through manual verification, resulting in the ground-truth annotation of the object saliency ranking. The image annotation also includes the classification category of each object, the image ID, the positioning box of each object, and the mask of each object. The data file is in JSON format. The final dataset includes 5150 training images of 30 object classes.

[0039] 1.3 Divide the training set, validation set, and test set into a ratio of 8:1:1 for training and evaluating the performance of the model. Figure 4 This is an example of a dataset constructed for the present invention.

[0040] Step 2: Model training, specifically including: selecting cross entropy as the loss function, setting hyperparameters (such as learning rate, batch size, and optimizer), using the training set obtained in step 1 to train the model, and evaluating the model performance on the validation set. Based on the validation results, adjust the model structure or hyperparameters to finally obtain the best performing model.

[0041] After specific practical experiments, the network deployment was completed on a computer device with an Intel 9700k CPU, an NVIDIA RTX 2080Ti graphics card, and 32GB of memory. The learning rate was set to 0.02, and the training process was iterated 18,000 times to complete the model training.

[0042] Model performance evaluation includes: model test set performance evaluation, model generalization performance evaluation, and test evaluation of key links in model performance improvement.

[0043] The model test set performance evaluation is to test the trained model on the test set obtained in step 1, using the accuracy of significant target ranking, namely the Spearman correlation rank (SOR), as the evaluation criterion.

[0044] The model generalization performance evaluation is to test the trained model on a public but untrained dataset and use SOR as the evaluation criterion to examine the generalization performance of the model.

[0045] The test evaluation of the key links of model performance improvement is to examine which links improve the performance of the model, and the average accuracy of saliency prediction under different IoU mask thresholds (i.e. ) as the evaluation criteria.

[0046] The specific model test set performance evaluation test results: SOR reached 0.906. The ranking results generated by the significant target ranking model are as follows Figure 5 shown.

[0047] The trained salient object ranking model was further tested on the ASSR and PASCAL-S datasets using zero-shot testing. The specific generalization performance evaluation results of the model are shown in Table 1.

[0048] Table 1. Model generalization test

[0049] As shown in Table 1, the trained salient object ranking model achieved a salient object ranking accuracy (SOR) of 0.823 on the ASSR dataset. This performance not only surpasses some existing models trained on this dataset, but is also comparable to the current leading model on this dataset. Similarly, on the PASCAL-S dataset, the trained salient object ranking model also achieved an SOR of 0.923, further demonstrating its ability to accurately capture and rank salient objects in images across datasets.

[0050] The test evaluation of the key aspects of model performance improvement is shown in Table 2. Represents the average accuracy of saliency prediction at different IoU mask thresholds. When the ROI embedding layer structure and ROI dual alignment structure described in this invention are added to the backbone feature extraction network and the multi-label classification layer structure described in this invention, the performance index is further improved.

[0051] Table 2. Test results of key aspects of model performance improvement

[0052] Compared with the existing technology, the salient target ranking network of the present invention can automatically generate generalized salient target ranking results that conform to the natural observation rules of humans. This improvement is mainly due to the design of the salient target ranking head in the model construction, which uses salient ranking clues based on prior knowledge to guide the model to learn how to sort salient targets more accurately. This design not only improves the salient target ranking accuracy of the model, but also makes the ranking results it generates more consistent with the natural observation habits of humans, thereby achieving generalized salient target ranking. At the same time, the network can also be regarded as a multi-label classification task, such as providing visual clues with target priority for visually impaired patients and other application scenarios that require detailed information. The classification accuracy mAP commonly used in standard target classification tasks can be used to perform saliency prediction evaluation, and the effect of saliency prediction is positively correlated with the effect of salient target ranking.

[0053] The above-mentioned specific implementation can be partially adjusted in different ways by those skilled in the art without departing from the principles and purpose of the present invention. The scope of protection of the present invention shall be based on the claims and shall not be limited by the above-mentioned specific implementation. All implementation schemes within its scope shall be subject to the constraints of the present invention.

Claims

1. A method for ranking salient objects based on visual prior knowledge, characterized in that: By constructing a salient object ranking network in the offline stage, it is used to identify each salient object in the image after being trained with a data set based on visual prior knowledge and generate results based on object saliency ranking; The salient object sorting network comprises: a backbone feature extraction network, a feature pyramid network (FPN), a region proposal network (RPN), a region of interest dual alignment structure and a salient object sorting head, wherein: the backbone feature extraction network extracts features from an input image, and obtains feature maps at different scales through multiple convolution, pooling and residual operations and inputs the feature maps into the FPN; the FPN performs top-down path and lateral connection fusion on the multi-scale feature map to obtain an updated feature map that fuses multi-scale information to the RPN; the RPN uses a sliding window to extract salient object candidate regions from the updated feature map; the region of interest dual alignment structure realizes the mapping between the salient object candidate regions and the updated feature map through the region of interest alignment operation to obtain a fine-grained feature map; the salient object sorting head identifies each salient object in the fine-grained feature map according to the salient object candidate regions and the fine-grained feature map and generates a result based on the object saliency sorting.

2. The method for ranking salient objects based on visual prior knowledge according to claim 1, characterized in that: The mapping refers to mapping the bounding box of the salient target candidate region output by the RPN to the updated feature map output by the FPN, specifically, converting the bounding box of the salient target candidate region from the original image space to the feature map space of different layers in the feature pyramid, and obtaining the coordinates of the salient target candidate region corresponding to the feature map space, wherein: the feature map P2 from the second layer and the feature map P k from the kth layer in the feature pyramid k The features are resampled according to the coordinates of the candidate regions of the salient objects in the feature map space, and then mapped to the same spatial dimension through the region of interest alignment operation, and then sent to the salient object sorting head after the splicing operation; The k-th layer feature map P k The calculation process is: Where: P k represents the k-th layer feature map in the feature pyramid, and its minimum value is 3; 224 is the standard ImageNet pre-trained image size, w and h are the width and height of the region of interest, respectively. Indicates rounding down.

3. The method for ranking salient objects based on visual prior knowledge according to claim 1, characterized in that: The salient target sorting head comprises: a feature bottleneck layer, an interest region embedding layer and a multi-label classification layer, wherein: the feature bottleneck layer integrates and downsamples the extracted feature maps through convolution operations to reduce feature dimensions and extract key features; the interest region embedding layer uses the bounding box coordinate information of the salient target candidate area to embed the contextual relationship between the target and the scene into the feature, thereby enhancing the feature representation of the salient target; the multi-label classification layer classifies the target by multiple attributes according to visual contrast, spatial position and target size, and outputs the salient sorting result.

4. The method for ranking salient objects based on visual prior knowledge according to claim 1, characterized in that: The multi-attribute classification is specifically as follows: based on the output features from the feature bottleneck layer and the region of interest embedding layer, based on but not limited to three significant feature clues, three label classification heads are used to obtain multi-label classification results V = {V c ,V l ,V s }, where: V c ,V l ,V s The classification is carried out according to the attributes of visual contrast, spatial position and target size.

5. The method for ranking salient objects based on visual prior knowledge according to claim 1, characterized in that: The significance ranking result is specifically: Rank_level = W·V, where: W is the significance ranking weight vector from high to low in terms of contrast, spatial position, and target size, W = [2 2 ,2 1 ,2 0 ], that is, the highest significance ranking level is level 7 (1×2 2 +1×2 1 +1×2 0 ), the minimum is level 0 (0×2 2 +0×2 1 +0×2 0 );·is the inner product of the vectors.

6. The method for ranking salient objects based on visual prior knowledge according to claim 1, characterized in that: The loss function of the training is preferably a cross entropy loss function, specifically: Loss = 1 hassaliency ∑ RoI ∑ labels 1 positiveroi CrossEntropy(y true ,y pred ), where: y true represents the true label, y pred Represents the predicted label, 1 hassaliency Is a binary indicator used to indicate whether the target in the image has a saliency level. When the saliency level of the target in the image is labeled, 1 hassaliency is equal to 1, the parameters of the entire model are optimized by back propagation, otherwise, 1 hassaliency is equal to 0, and only the network structure before the saliency sorting head is optimized.

7. The method for ranking salient objects based on visual prior knowledge according to any one of claims 1 to 6, characterized in that: include: Step 1: Construct a data set, including: 1.1 Using the contrast, position and size prior attributes of the target in the bottom-up visual attention mechanism, we construct a corresponding probability calculation model for the three image attributes to accurately quantify the saliency of each target on different attributes; 1.2 Based on the original MS-COCO dataset annotation, the probability values ​​corresponding to the three attributes of the annotated target are calculated through step 1.1, and then each attribute is converted into 0 or 1 by hard coding according to the preset threshold to obtain the training dataset. Specifically, the saliency level of the target is sorted in absolute order from high to low in contrast, spatial position, and size. The sorting weight vector of the three image attributes is [2 2 ,2 1 ,2 0 ], based on the hard-coded results of the three attributes and their corresponding weight vector combinations, the accuracy of the dataset annotation is ensured through manual verification, and then the true value annotation of the target significance ranking is obtained; 1.3 Divide the training set, validation set, and test set in a ratio of 8:1:1 to train and evaluate the performance of the model; Step 2: Model training, specifically including: selecting cross entropy as the loss function, setting hyperparameters, using the training set obtained in step 1 to train the model, and evaluating the model performance on the validation set, adjusting the model structure or hyperparameters based on the validation results, and finally obtaining the best performing model.

8. The method for ranking salient objects based on visual prior knowledge according to claim 7, characterized in that: The contrast attribute is to calculate the probability or confidence of the contrast attribute by comparing the color difference between the current marked target and the surrounding area, specifically: Where: O, C represent the number of pixels of the target and target contour respectively, pix represents pixels, pix∈object represents pixels belonging to the target, pix∈contour represents pixels belonging to the edge contour, and The distribution histogram of the three channels representing the target and the local area around the target, where c represents the color R, G, B channels to be taken, G pixvalue Represents the pixel value of the edge contour of the target, * represents the pixel point of the four neighborhoods, that is, *∈4-neighbor, G pix,* represents the four neighborhoods of the target contour pixel, ‖·‖2 is the L2 norm, and |·| represents the absolute value calculation.

9. The method for ranking salient objects based on visual prior knowledge according to claim 7, characterized in that: The position is used to measure the spatial position of the labeled object compared to the center area of ​​the image. The closer the object is to the center area of ​​the image, the probability or confidence (Prob loc ), specifically: Among them: O, E represent the total number of pixels in the target area and the ellipse area respectively, object represents a significant target, center w , center h , center x , center y Represents the horizontal axis length, vertical axis length, horizontal coordinate of the pixel in the area, and vertical coordinate of the pixel in the area, respectively. x and pix y Represents the horizontal and vertical coordinates of the pixels within the labeled object, respectively.

10. The method for ranking significant objects based on visual prior knowledge according to claim 7, characterized in that: The size is used to measure the proportion of the labeled object in the entire image, the probability or confidence of the size attribute (Prob size ), specifically: Where: image h and image w Represent the height and width of the image respectively, pix represents pixels, and object represents a significant target.

Images