Method for ranking salient objects based on visual prior knowledge

Abstract

The application discloses a salient object ranking method based on visual prior knowledge, constructs a salient object ranking network in an offline stage, and is used for identifying each salient object in an image and generating a result based on target salient object ranking after training based on a data set of visual prior knowledge. The application can automatically generate a salient object ranking result conforming to human objective observation rules and generalization, and provides more practical technical support for the application of intelligent auxiliary tools for visually impaired patients, controllable image description generation, human-computer interaction, auxiliary decision, intelligent monitoring, automatic driving, augmented reality and the like.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of image processing, and particularly relates to a salient object ranking method based on visual prior knowledge. BACKGROUND

[0002] Existing salient object ranking technologies based on deep learning highly depend on data sets, lack the integration of prior knowledge, and thus the generated results often do not conform to the natural observation law of human beings, thereby affecting the practicability; and when facing new scenes or new data sets, these technologies usually show poor cross-domain adaptability, resulting in insufficient generalization ability and limiting the application range. SUMMARY

[0003] The salient object ranking method based on visual prior knowledge can automatically generate salient object ranking results conforming to the objective observation law of human beings and generalization, and provides more practical technical support for the application of intelligent auxiliary tools for visually impaired patients, controllable image description generation, human-computer interaction, auxiliary decision-making, intelligent monitoring, automatic driving, augmented reality and the like.

[0004] The application is implemented by the following technical scheme:

[0005] The application relates to a salient object ranking method based on visual prior knowledge, which constructs a salient object ranking network in an offline stage, and generates a result based on target saliency ranking after training based on a data set of visual prior knowledge.

[0006] The salient object ranking network comprises a backbone feature extraction network, a feature pyramid network (FPN), a region proposal network (RPN), a region of interest double alignment structure and a salient object ranking head. The backbone feature extraction network extracts features of an input image, and obtains feature maps at different scales through a plurality of convolution, pooling and residual operations and inputs the feature maps into the FPN. The FPN fuses the multi-scale feature maps from top to bottom and horizontally, obtains updated feature maps with fused multi-scale information to the RPN. The RPN extracts salient object candidate regions from the updated feature maps by using a sliding window. The region of interest double alignment structure realizes the mapping between the salient object candidate regions and the updated feature maps through a region of interest alignment operation, and obtains a fine-grained feature map. The salient object ranking head identifies each salient object in the fine-grained feature map according to the salient object candidate regions and the fine-grained feature map, and generates a result based on target saliency ranking.

[0007] The backbone feature extraction network adopts, but is not limited to, Resnet 101 to extract deep image features through convolution operations.

[0008] The mapping refers to mapping the boundary box of the salient target candidate region output from the RPN to the updated feature map output in the FPN, specifically: converting the boundary box of the salient target candidate region from the original image space to the feature map space of different layers in the feature pyramid, to obtain the coordinates of the salient target candidate region in the feature map space, wherein: the second layer feature map P2 and the kth layer feature map Pk in the feature pyramid come from the first layer feature map P1 of the feature pyramid. k According to the coordinates of the salient target candidate region in the feature map space, the feature is resampled respectively, and then is mapped to the same spatial dimension through the region of interest alignment operation, and after the splicing operation, is sent to the salient target ranking head.

[0009] The kth layer feature map P k The calculation process is as follows: Wherein: P k represents the kth layer feature map in the feature pyramid, and the minimum value is 3; 224 is the standard ImageNet pre-trained image size, and w and h are the width and height of the region of interest, represents the floor.

[0010] The salient target ranking head includes a feature bottleneck layer, a region of interest embedding layer and a multi-label classification layer, wherein: the feature bottleneck layer integrates and down-samples the extracted feature map through convolution operation, so as to reduce the feature dimension and extract key features; the region of interest embedding layer embeds the context relationship between the target and the scene into the feature by using the boundary box coordinate information of the salient target candidate region, so as to enhance the feature representation of the salient target; and the multi-label classification layer classifies the target according to visual contrast, spatial position and target size, and outputs the salient ranking result.

[0011] The integration and down-sampling specifically refers to: after the de-integrated and down-sampled feature map is subjected to [1x1, 3x3] and [1x1, 7x7] convolution, the feature map is output after a [1x1] convolution layer, and the output feature map is reduced to 128 dimensions.

[0012] The region of interest embedding specifically refers to: after two [3x3] convolutions, the boundary box coordinate information of the salient target candidate region is embedded into the feature map by using two fully connected layers.

[0013] The multi-attribute classification specifically refers to: according to the output features from the feature bottleneck layer and the region of interest embedding layer, based on but not limited to three salient feature clues, three label classification heads are used to obtain multi-label classification results V={V c ,V l ,V s} for visual contrast, spatial position and target size attributes, wherein: V c ,Vl ,V s The classification of the visual contrast, the spatial position and the target size attribute in turn.

[0014] The significant ranking result is Rank_level=W·V, wherein W is a significant ranking weight vector from high to low of the contrast, the spatial position and the target size, W=[2 2 ,2 1 ,2 0 ], that is, the highest significant ranking level is 7 (1x2 2 +1x2 1 +1x2 0 ), and the minimum is 0 (0x2 2 +0x2 1 +0x2 0 ); and · is the inner product of vectors.

[0015] The loss function of the training is preferably a cross-entropy loss function, and specifically: Loss=

[0016] 1 hassaliency ∑ RoI ∑ labels 1 positiveroi CrossEntropy(y true ,y pred ), wherein y true represents a real label, y pred represents a predicted label, 1 hassaliency is a binary indicator for indicating whether the target in the image has a significant level, when the significant level of the target in the image is labeled, 1 hassaliency is equal to 1, and the parameters of the whole model are optimized through back propagation. Otherwise, 1 hassaliency is equal to 0, and only the network structure before the significant ranking head is optimized.

[0017] The dataset based on prior knowledge adopts an open-source image dataset, obtains a weight vector by calculating the probabilities of the contrast, the position and the size attributes of the target in the image based on the bottom-up visual attention mechanism, and combines the hard-coded results thresholded to 0 or 1 to obtain the significant ranking true value label of each target. The label of the image also contains the classification category of each target, the picture ID, the positioning frame of each target, and the mask of each target, and the data file is in json format.

[0018] Technical effects

[0019] The present invention obtains a more generalized significance ranking result by constructing a multi-label classification layer including multiple visual prior significance attributes including visual contrast, spatial position and target size. At the same time, the present invention uses a double alignment structure of the region of interest to calculate the relationship between the area of ​​the region of interest and the level of the feature map to extract more fine-grained features, thereby further improving the performance of the network. Compared with the existing technology, the present invention can generate a generalized significance target ranking that conforms to the natural observation laws of humans. It can be used for intelligent auxiliary tools, intelligent monitoring, and autonomous driving for visually impaired patients. It also broadens the application boundaries of computer vision technology in image description production, human-computer interaction, decision support, augmented reality and other fields. By integrating visual prior knowledge, this method effectively avoids the problems of traditional significance target algorithms that do not conform to the natural observation laws of humans and have insufficient generalization capabilities, and provides new ideas and solutions for the intelligent and humanized development of computer vision technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Flowchart of the present invention;

[0021] Figure 2 Schematic diagram of the network structure of significant target ranking in the embodiment;

[0022] Figure 3 is a flow chart of an embodiment;

[0023] Figure 4 This is an example diagram of the data set of the embodiment;

[0024] Figure 5 Schematic diagram of the embodiment effect. DETAILED DESCRIPTION

[0025] like Figure 1As shown, a significant target ranking method based on visual prior knowledge is related to the present embodiment, which includes a backbone feature extraction network, a feature pyramid network (FPN), a region proposal network (RPN), a region of interest double alignment structure, and a significant target ranking head. Among them, the backbone feature extraction network extracts the features of the input image, and outputs the feature maps at different scales through a plurality of convolution, pooling, residual and other operations, and inputs the feature maps into the FPN; the FPN fuses the input multi-scale feature maps through the top-down path and horizontal connection, and outputs new feature maps with fused multi-scale information to the RPN; the RPN extracts possible significant target candidate regions in the image through a sliding window according to the input; the region of interest double alignment structure receives the feature maps through the FPN and the significant target candidate regions processed by the RPN at the same time, and realizes the mapping between the significant target candidate regions and the feature maps through the region of interest alignment operation; the significant target ranking head receives the significant target candidate regions processed by the RPN and the feature maps processed by the region of interest double alignment structure at the same time, and outputs the ranking result of the significant target.

[0026] As shown in Figure 2 The backbone feature extraction network is a convolutional neural network (taking ResNet101 as an example) as a feature extractor, which gradually extracts high-level semantic features of the image through a plurality of convolution layers, pooling layers and residual connections, and outputs feature maps at different scales, which are input into the FPN.

[0027] As shown in Figure 2 The feature pyramid network (FPN) is introduced, which fuses feature maps at different levels through a top-down path and a horizontal connection mechanism to enhance the extraction and fusion of multi-scale features, and outputs new feature maps P2-P6 with fused multi-scale information, which are input into the RPN.

[0028] As shown in Figure 2 The region proposal network (RPN) is introduced, which selects regions through a sliding window based on the feature maps P2-P6 output by the FPN, and outputs a series of boundary boxes that may contain significant target candidate regions, which are used for the subsequent region of interest double alignment structure and significant target ranking head.

[0029] As shown in Figure 2-d, the double alignment structure of the region of interest realizes the mapping between the salient target candidate region and the feature map, which is used to add additional fine-grained features to the network. Specifically, the bounding box of the salient target candidate region output from the RPN network is mapped to the feature map output from the FPN network. This involves converting the bounding box of the salient target candidate region from the original image space to the feature map space of different layers in the feature pyramid, and obtaining the coordinates of the salient target candidate region corresponding to the feature map space. Among them, the feature map P2 from the second layer and the feature map P kth layer in the feature pyramid are k The features of the salient target candidate regions are resampled according to their corresponding coordinates in the feature map space, and then mapped to the same spatial dimension through the region of interest alignment operation. After the splicing operation, they are sent to the salient target sorting head.

[0030] like Figure 2 As shown in -e, the feature bottleneck layer and the ROI embedding layer respectively receive the feature map processed by the ROI double alignment structure and the salient target candidate region processed by RPN. The new feature maps processed by each are spliced ​​and sent to the multi-label classification layer to output the significance ranking result.

[0031] The feature bottleneck layer receives the feature map processed by the double alignment structure of the region of interest, integrates and downsamples the feature map output by the FPN through [1×1, 3×3] and [1×1, 7×7] convolutions, and then passes through a [1×1] convolution layer to reduce the feature map to 128 dimensions.

[0032] The region of interest embedding layer receives the salient target candidate region after RPN processing, and after two [3×3] convolutions, uses two fully connected layers to embed the bounding box coordinate information of the salient target candidate region into the feature map.

[0033] The multi-label classification layer receives the feature map spliced ​​after processing by the feature bottleneck layer and the region of interest embedding layer, and obtains the multi-label classification results V = {V c ,V l ,V s}, where: V c ,V l ,V s The classification is carried out according to the visual contrast, spatial position and target size attributes.

[0034] The significance ranking result is specifically: Rank_level = W·V, where: W is the significance ranking weight vector of contrast, spatial position, and target size from high to low, W = [2 2 ,2 1 ,20 ], that is, the highest significance ranking level is level 7 (1×2 2 +1×2 1 +1×2 0 ), the minimum is level 0 (0×2 2 +0×2 1 +0×2 0 );· is the inner product of the vectors.

[0035] like Figure 3 As shown in FIG, the salient object ranking method based on visual prior knowledge of this embodiment includes:

[0036] Step 1: Construct a data set, including:

[0037] 1.1 Using the contrast, position, and size priors of the target in the bottom-up visual attention mechanism, we construct a corresponding probability calculation model for the three image attributes to accurately quantify the salience of each target on different attributes. These probability values ​​are then thresholded (hard-coded) to 0 or 1 to indicate whether the current target should have the corresponding image attribute.

[0038] The contrast attribute is calculated by comparing the color difference between the current marked object and the surrounding area to calculate the probability or confidence of the contrast attribute. The larger the difference ratio, the higher the probability or confidence (Prob contrast ) is higher. Specifically:

[0039] Where: O and C represent the number of pixels of the object and the object contour respectively. pix represents pixels, pix∈object represents pixels belonging to the object, and pix∈contour represents pixels belonging to the edge contour. and The distribution histogram of the three channels representing the target and the local area around the target, where c represents the color R, G, and B channels to be taken. pixvalue Represents the pixel value of the edge contour of the target, * represents the pixel point of the four neighborhoods, that is, *∈4-neighbor, G pix,* Represents the four neighborhoods of the target contour pixel. ||·||2 is the L2 norm, and |·| represents the absolute value calculation. The first half of the formula is used to quantify the probability of contrast. In the formula, To represent the color vector of the target in R, G, and B channels. Then, use the distribution histogram in an area with the same aspect ratio and twice the area of ​​the original target area The color vector of the local area surrounding the target is then used. The inner product and L2 norm of the two vectors are then calculated to measure their similarity. A higher similarity indicates lower contrast between the target and its surrounding area. Furthermore, the second half of the formula quantifies the difference in value between each pixel in the target's outline and its surroundings; a greater difference indicates higher contrast. The sum of these two components represents the total probability of the contrast attribute.

[0040] The position attribute is used to measure the spatial position of the labeled target compared to the center area of ​​the image. The closer the target is to the center area of ​​the image, the higher the probability or confidence level of the target with the position attribute (Prob loc ) is higher. Specifically:

[0041] Where: O, E represent the total number of pixels in the target area and the elliptical area respectively. object represents a significant target. center w , center h , center x , center y They represent the horizontal axis length, vertical axis length, horizontal coordinates of pixels in the area, and vertical coordinates of pixels in the area, respectively. x and pix y Represent the horizontal and vertical coordinates of the pixels within the annotated object, respectively. The specific calculation method is shown in the formula. An elliptical region coinciding with the image center represents the image center. The final location attribute confidence of the object is determined by the average contribution of all pixels in the object.

[0042] The size attribute is used to measure the proportion of the labeled object in the entire image. Generally, the higher the proportion, the higher the probability or confidence of the size attribute. size ), specifically: Where: image h and image w Represent the height and width of the image respectively. pix represents pixels, and object represents a salient object.

[0043] 1.2 Based on the original MS-COCO dataset annotation, the probability values ​​corresponding to the three attributes of the annotated target are calculated through step 1.1, and then each attribute is converted into 0 or 1 by hard coding according to the preset threshold (usually 0.5) to obtain the training dataset. Specifically, the saliency level of the target is sorted in the absolute order of contrast, spatial position, and size from high to low. The sorting weight vector of the three image attributes is [2 2 ,2 1 ,2 0]According to the hard-coded results of the three attributes and the corresponding weight vector combination, the accuracy of the data set annotation is ensured through manual verification, and then the significance ranking true value annotation of the target is obtained. The annotation of the image also includes the classification category of each target, the picture ID, the positioning box of each target, and the mask of each target, and the data file is in json format. The finally obtained data set includes: 5150 training images of 30 target classes.

[0044] 1.3 Divide the training set, validation set and test set in the ratio of 8:1:1 for training and evaluating the performance of the model. As Figure 4 The example of the data set constructed by the present application.

[0045] Step 2, model training, specifically including: selecting cross-entropy as the loss function, setting the hyperparameters (such as learning rate, batch size, optimizer), using the training set obtained in step 1 to train the model, and evaluating the model performance on the validation set, adjusting the model structure or hyperparameters according to the validation results, and finally obtaining the model with the best performance.

[0046] Through specific actual experiments, the network deployment is completed on a computer device with a CPU model of Intel 9700k, a graphics card of Nvidia RTX 2080Ti, and 32GB of memory, the learning rate is set to 0.02, the training process is iterated for a total of 18000 times, and the model training is completed.

[0047] The model performance evaluation includes: model test set performance evaluation, model generalization performance evaluation, and model performance improvement key link test evaluation.

[0048] The model test set performance evaluation is to test the trained model on the test set obtained in step 1, and the significance target ranking accuracy, i.e. Spearman correlation rank (SOR), is used as the evaluation standard.

[0049] The model generalization performance evaluation is to test the trained model on the published but untrained data set, and use SOR as the evaluation standard to evaluate the generalization performance of the model.

[0050] The model performance improvement key link test evaluation is to investigate which link improves the performance of the model, and the average accuracy of the significance prediction under different IoU mask thresholds (i.e. ) is used as the evaluation standard.

[0051] The specific model test set performance evaluation test result: SOR reaches 0.906. The ranking result generated by the significance target ranking model is as shown in Figure 5 .

[0052] Further, the trained salient object ranking model is tested in zero-shot manner on the ASSR and PASCAL-S datasets, and the generalization performance evaluation results of the specific model are shown in Table 1.

[0053] Table 1. Model generalization test

[0054]

[0055] As shown in Table 1, on the ASSR dataset, the salient object ranking accuracy (SOR) of the trained salient object ranking model reaches 0.823, which not only surpasses the existing models that are partially dependent on the dataset for training, but also is comparable to the model with the most leading performance on the dataset. Similarly, on the PASCAL-S dataset, the trained salient object ranking model also stands out with an SOR accuracy of 0.923, further verifying its ability to accurately capture and rank salient objects in images across different dataset boundaries.

[0056] The test evaluation of the key links of the performance improvement of the specific model is shown in Table 2, and the performance indicators in the table represent the average accuracy of salient prediction under different IoU mask thresholds. When the region of interest embedding layer structure and the region of interest double alignment structure of the application are added to the backbone feature extraction network and the multi-label classification layer structure of the application, the performance indicators are further improved.

[0057] Table 2. Test results of key links of performance improvement of the specific model

[0058]

[0059] Compared with the prior art, the salient object ranking network of the application can automatically generate salient object ranking results that conform to the natural observation rules of humans and have generalization. This improvement is mainly due to the design of the salient object ranking head in the model construction, which uses salient ranking clues based on prior knowledge to guide the model to learn how to more accurately rank salient objects. This design not only improves the salient object ranking accuracy of the model, but also makes the ranking results generated by the model more consistent with the natural observation habits of humans, thereby achieving salient object ranking with generalization. At the same time, the network can also be regarded as a multi-label classification task, such as providing visual clues with object priority for visually impaired patients and other application scenarios that require detailed information. The classification accuracy mAP commonly used in standard object classification tasks can be used to evaluate the salient prediction, and the effect of salient prediction is positively correlated with the effect of salient object ranking.

[0060] The above specific embodiments can be partially adjusted in different ways by those skilled in the art without departing from the principles and purposes of the present application, the protection scope of the present application is subject to the claims and is not limited by the above specific embodiments, each implementation within the scope is subject to the present application.

Claims

1. A method for ranking salient objects based on visual prior knowledge, characterized in that: By constructing a salient object ranking network in the offline stage, it is used to identify each salient object in the image after training with a dataset based on visual prior knowledge and generate results based on object saliency ranking; The salient object ranking network includes: a backbone feature extraction network, a feature pyramid network FPN, a region proposal network RPN, a region of interest dual alignment structure and a salient object ranking head, wherein: the backbone feature extraction network extracts features from the input image, and obtains feature maps at different scales through multiple convolution, pooling and residual operations and inputs them into the FPN; the FPN performs a top-down path and lateral connection fusion on the multi-scale feature map to obtain an updated feature map that integrates multi-scale information and inputs it to the RPN; the RPN uses a sliding window to extract salient object candidate regions from the updated feature map; the region of interest dual alignment structure realizes the mapping between the salient object candidate regions and the updated feature map through the region of interest alignment operation to obtain a fine-grained feature map; the salient object ranking head identifies each salient object in the fine-grained feature map based on the salient object candidate regions and the fine-grained feature map and generates a result based on the object saliency ranking; The mapping refers to mapping the bounding box of the salient target candidate region output by RPN to the updated feature map output by FPN, specifically converting the bounding box of the salient target candidate region from the original image space to the feature map space of different layers in the feature pyramid, and obtaining the coordinates of the salient target candidate region corresponding to the feature map space, where: the feature map P2 of the second layer and the feature map P of the kth layer in the feature pyramid are k The features of the candidate regions of the salient objects are resampled according to their corresponding coordinates in the feature map space, and then mapped to the same spatial dimension through the region of interest alignment operation. After the splicing operation, they are sent to the salient object sorting head; The k-th layer feature map P k The calculation process is: Where: P k Represents the k-th layer feature map in the feature pyramid, and its minimum value is 3; 224 is the standard ImageNet pre-training image size, w, h are the width and height of the region of interest, respectively. Indicates rounding down; The salient object ranking head includes: a feature bottleneck layer, a region of interest embedding layer, and a multi-label classification layer, wherein: the feature bottleneck layer integrates and downsamples the extracted feature maps through convolution operations to reduce feature dimensions and extract key features; the region of interest embedding layer uses the bounding box coordinate information of the salient object candidate region to embed the contextual relationship between the object and the scene into the features, thereby enhancing the feature representation of the salient object; the multi-label classification layer classifies the object according to multiple attributes based on visual contrast, spatial position, and object size, and outputs the saliency ranking result; The aforementioned significant target ranking specifically includes: Step 1: Construct a data set, including: 1.1 Using the contrast, position, and size priors of the target in the bottom-up visual attention mechanism, we construct corresponding probability calculation models for the three image attributes to accurately quantify the salience of each target in different attributes; 1.2 Based on the original MS-COCO dataset annotation, the three attributes of the annotated target are calculated through step 1.1 The corresponding probability values ​​are then converted into 0 or 1 by hard coding according to the preset threshold to obtain the training data set. Specifically, the significance level of the target is sorted in the absolute order of contrast, spatial position, and size from high to low. The sorting weight vector of the three image attributes is [2 2 ,2 1 ,2 0 ], based on the hard-coded results of the three attributes and their corresponding weight vector combinations, the accuracy of the dataset annotation is ensured through manual verification, and the true value annotation of the target significance ranking is obtained; 1.3 Divide the training set, validation set, and test set into a ratio of 8:1:1 for training and evaluating the performance of the model; Step 2: Model training. This includes selecting cross entropy as the loss function, setting hyperparameters, training the model using the training set obtained in step 1, evaluating the model performance on the validation set, and adjusting the model structure or hyperparameters based on the validation results to ultimately obtain the optimal performing model.

2. The method for ranking significant objects based on visual prior knowledge according to claim 1, wherein: The multi-attribute classification is specifically as follows: based on the output features from the feature bottleneck layer and the region of interest embedding layer, based on but not limited to three significant feature clues, three label classification heads are used to obtain multi-label classification results V = {V c ,V l ,V s }, where: V c ,V l ,V s The classification is carried out according to the visual contrast, spatial position and target size attributes.

3. The method for ranking significant objects based on visual prior knowledge according to claim 1, wherein: The significance ranking result is specifically: Rank_level = W·V, where: W is the significance ranking weight vector of contrast, spatial position, and target size from high to low, W = [2 2 ,2 1 ,2 0 ], that is, the highest significance ranking level is level 7 (1×2 2 +1×2 1 +1×2 0 ), the minimum is level 0 (0×2 2 +0×2 1 +0×2 0 );· is the inner product of the vectors.

4. The method for ranking significant objects based on visual prior knowledge according to claim 1, wherein: The loss function of the training is the cross entropy loss function, specifically: Loss = 1 hassaliency ∑ RoI ∑ labels 1 positiveroi CrossEntropy(y true ,y pred ), where: y true represents the true label, y pred Represents the predicted label, 1 hassaliency Is a binary indicator used to indicate whether the target in the image has a saliency level. When the saliency level of the target in the image is marked, 1 hassaliency is equal to 1, the parameters of the entire model are optimized by back propagation, otherwise, 1 hassaliency Equal to 0, and only optimize the network structure before the saliency ranking head.

5. The method for ranking significant objects based on visual prior knowledge according to claim 1, wherein: The contrast attribute is calculated by comparing the color difference between the current marked object and the surrounding area to determine the probability or confidence of the contrast attribute. Specifically, Where: O, C represent the number of pixels of the target and target contour respectively, pix represents pixels, pix∈object represents pixels belonging to the target, pix∈contour represents pixels belonging to the edge contour, and The distribution histogram of the three channels representing the target and the local area around the target, where c represents the color R, G, B channel to be valued, G pixvalue Represents the pixel value of the edge contour of the target, * represents the pixel point of the four neighborhoods, that is, *∈4-neighbor, G pix,* represents the four neighborhoods of the target contour pixel, ||·||2 is the L2 norm, and |·| represents the absolute value calculation.

6. The method for ranking significant objects based on visual prior knowledge according to claim 1, wherein: The position is used to measure the spatial position of the labeled target compared to the center of the image. The closer the target is to the center of the image, the higher the probability or confidence of the position attribute. loc The higher the value, the more: Among them: O, E represent the total number of pixels in the target area and the elliptical area respectively, object represents the salient target, center w , center h , center x , center y Represents the horizontal axis length, vertical axis length, horizontal coordinate of the pixel in the area, and vertical coordinate of the pixel in the area, respectively. x and pix y Represents the horizontal and vertical coordinates of the pixel within the labeled object, respectively.

7. The method for ranking significant objects based on visual prior knowledge according to claim 1, wherein: The size is used to measure the proportion of the labeled target in the entire image, and the probability or confidence Prob of the size attribute size The higher the value, the more: Where: image h and image w Represent the height and width of the image respectively, pix represents pixels, and object represents a salient target.

Images