Saliency target sorting method based on reinforcement learning

Abstract

The invention discloses a saliency target sorting method based on reinforcement learning. An environment and an agent are constructed in an offline stage, and interaction training of the agent and the environment is carried out through a saliency target sorting data set to obtain a saliency target sorting result in a scene. According to the method, the optimal sorting strategy is learned by utilizing dynamic interaction with the environment, so that the model learning can be flexibly adjusted according to factors such as context information, mutual relations among targets and reward signals, the dynamic transfer process of human visual attention is simulated, a more accurate and practical saliency target sorting result is generated, and the sorting efficiency is improved. And a more practical technical support is provided for applications in the fields of intelligent auxiliary tools, image description generation, man-machine interaction, auxiliary decision making, intelligent monitoring, automatic driving, augmented reality and the like of the visually impaired patient.

Description

Technical Field

[0001] The present invention relates to a technology in the field of image processing, and specifically to a method for ranking salient objects based on reinforcement learning. Background Art

[0002] Existing deep learning-based salient object ranking techniques are usually transformed into classification problems or regression problems. They achieve prediction through static feature extraction and lack dynamic adaptability. They find it difficult to capture the dynamic shift of human visual attention mechanism according to the relationship between context and objects, which results in the inability to accurately reflect the natural flow of human vision in the ranking task, affecting the accuracy and practicality of the results. Summary of the invention

[0003] In view of the deficiency of the prior art that the dynamic movement order between salient targets cannot be observed, the present invention proposes a salient target sorting method based on reinforcement learning, which utilizes dynamic interaction with the environment to learn the optimal sorting strategy, so that the model learns to flexibly adjust according to factors such as context information, the relationship between targets and reward signals, thereby simulating the dynamic transfer process of human visual attention and generating more accurate and practical salient target sorting results, providing more practical technical support for the application of intelligent auxiliary tools for visually impaired patients, image description generation, human-computer interaction, decision support, intelligent monitoring, autonomous driving, augmented reality and other fields.

[0004] The present invention is achieved through the following technical solutions:

[0005] The present invention relates to a method for ranking significant objects based on reinforcement learning. In an offline stage, an environment and an agent are constructed, and interactive training between the agent and the environment is performed through a significant object ranking data set to obtain the result of ranking significant objects in a scene.

[0006] The environment includes: an environment candidate generator (ECG), an action executor and a memory unit, wherein: after the ECG segments all target instances in the original image and extracts the corresponding target instance features, significant candidate target instances and their feature sets are obtained through screening; the memory unit uses the feature set as the initial state for recording and state maintenance, and the intelligent agent performs spatial context information feature fusion and temporal context information fusion in turn according to the initial state, and predicts the cumulative score of the executed action; the action executor judges the overall performance of the currently selected target instance according to the cumulative score, and then rewards or punishes the action according to the set reward strategy to update the parameters of the intelligent agent, and at the same time, records the selected target instances and their order according to the relationship between the results of the intelligent agent's selection and the processed target instances, and updates the state in the memory unit, that is, the remaining candidate target instances and their feature sets.

[0007] The environment candidate generator (ECG) extracts and screens out significant candidate target instances and their feature sets from the original image, including: a target instance segmentation module, a feature extraction module and a candidate target screening module, wherein: the target instance segmentation module generates masks (Mask) and target instance bounding boxes (BoundingBox) of all target instances according to the original image and outputs them to the feature extraction module together with the original image to obtain the features of all target instances; the candidate target screening module screens out significant candidate target instances and their feature sets from the set of all target instance features according to the target instance bounding box (BoundingBox).

[0008] The intelligent agent includes: a spatial attention module (SAM), a temporal attention module (TAM) and a scoring head (ScoreHead), wherein: the spatial attention module integrates information of surrounding target instances, that is, spatial context information into target instance features; the temporal attention module calculates the temporal interaction between significant targets, that is, the degree of significance of the target instance noticed in the previous time step affecting the target instance noticed in the next time step; the scoring head predicts the value of selecting each action (Action), that is, the cumulative score predicted by the scoring head.

[0009] The integration mentioned above refers to: through the structure of self-attention, is a feature set of candidate target instances that incorporates spatial context information. is the feature of target instance i containing spatial context information, i∈[1,2,....n], n is the number of target instances to be selected. Global feature o g is the P5 feature map of the original image, the dimension of the feature map is d, and the set of significant target instance features is O candidate The feature set O of the candidate target instance fused with spatial context information is calculated as follows: s =ReLU(W a Attn s V), where: Q=W q O candidate , K=W k [O candidate ,o g ],V=W v [O candidate ,o g ], Q, K, V represent the query vector (Query), key vector (Key) and value vector (Value) in the attention mechanism respectively; W q , W k and W vIt is a trainable parameter, which is used to convert the input vector into Query, Key and Value vectors; Attn s is the spatial attention matrix, which is the attention distribution in space; T is the transposition operation; W a Is the effect on Attn s The trainable parameters.

[0010] The calculation of the temporal interaction between the salient targets refers to: calculating the feature set of the candidate target instance that integrates the temporal context information in: is the feature of the target instance i noticed in the current time step, i∈[1,2,....n], and n is the number of target instances to be selected. Let o t-1 is the feature of the target instance noticed in the previous time step, is the feature of target instance i containing spatial context information, W is a trainable parameter.

[0011] The predicted value of each action is: candidate The action of selecting a specific target instance and candidate The number of target instances n in state s t Next, select action a and continue to use the current strategy π to interact with the environment multiple times until the cumulative score at the end of the round W g and W h is a trainable parameter. It is a global feature o g Features after linear layer mapping.

[0012] The updating of the parameters of the agent refers to: updating the parameters of each module network of the agent through the temporal difference algorithm, based on the simultaneous participation of two agents with the same structure but different parameters in the training and calculation of the loss. Specifically, the parameter update frequency of the agent as the target model is much lower than that of the learning model. The strategy of the target model is π * , then the target model predicts that at time t+1, the cumulative score of the interaction with the environment according to the strategy of the target model until the end of the entire round is At time t, the cumulative score of the agent interacting with the environment according to the strategy of the learning model agent until the end of the entire round is Q π (s t ,a). The environment’s score for the action a selected by the learning model at time t is r t , then the goal of model training is: to make Q π (s t ,a) and The value of is as close as possible. Technical Effects

[0013] The present invention constructs a dynamic adjustment module containing time-varying features, combined with real-time monitoring of data distribution and feedback optimization mechanism, which can capture the changes of significant targets under the influence of spatial context and temporal context, extract more representative and refined features, and improve the performance of the model. Compared with the prior art, the present invention solves the problem of static sorting of traditional methods and difficulty in capturing the relationship between context and targets. The generated results are more in line with the changing laws in practical applications, and can be widely used in intelligent decision-making, real-time monitoring and other fields, providing a new idea for solving the problem of limited performance of traditional modeling methods in dynamic environments. By dynamically adjusting the modeling strategy and using the interaction between the environment and the intelligent body to simulate the dynamic transfer process of human visual attention, the present invention not only improves the sorting accuracy, but also broadens the scope of application of the model, providing a solution for intelligent modeling and efficient analysis of complex systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a flow chart of the present invention;

[0015] Figure 2 is a schematic diagram of an embodiment network;

[0016] Figure 3 (a) is the memory unit and Figure 3 (b) Schematic diagram of the action executor flow;

[0017] Figure 4 is a flow chart of an embodiment;

[0018] Figure 5 It is a schematic diagram of the effect of the embodiment. DETAILED DESCRIPTION

[0019] like Figure 1 and Figure 2 As shown, this embodiment relates to a method for ranking significant targets based on reinforcement learning. In the offline stage, an environment including an environment candidate generator, an action executor and a memory unit and an agent including a spatial attention module, a temporal attention module and a scoring head are constructed, and the interaction training between the agent and the environment is performed through a significant target ranking data set to obtain the result of ranking significant targets in the scene.

[0020] like Figure 2As shown, the environment candidate generator includes: a target instance segmentation module, a feature extraction module and a candidate target screening module, wherein: the target instance segmentation module adopts the CenterMask model, and generates the target instance mask and the target instance bounding box through the mask head and the bounding box head of the model; the feature extraction module uses the result of the bitwise AND calculation of the target instance Mask and the original image as input, and then uses the backbone network of the CenterMask model and combines it with the feature pyramid network to extract the multi-layer and multi-scale feature maps of all target instances and use the 5th layer feature map (P5) in the feature pyramid as the output target instance features and scene features, and finally forms a set of all target instances and their features; the candidate target screening module screens out the significant candidate target instances and their feature sets from the set of all target instances and their features, specifically: firstly, the intersection over union (IoU) is calculated according to the true value and the predicted target instance bounding box, and then the real significant targets are screened out according to the intersection over union.

[0021] The screening means that if the intersection-over-union ratio of the predicted target instance outer bounding box and the true annotation is greater than 0.5, it is set as a significant target, and the significance level is the same as the true value of the annotation. If it is less than 0.5, the significance level is set to the lowest, that is, not significant.

[0022] like Figure 2 As shown, the memory unit records and maintains the state in the environment, transmits the current state to the agent, and outputs the significant target ranking result.

[0023] like Figure 3 As shown in (a), the memory unit constructs a mapping from a target instance identifier set to a target instance related information set, that is, a dictionary data structure M = {(id i ,info i )}|i={1,2,…,n}, where: id i is the unique identifier of the i-th target instance, specifically a digital number, info i is a set of relevant information corresponding to the target instance, n is the total number of target instances, and the info corresponding to the i-th target instance is i ={s i ,r i ,f i},s i is the case where the i-th target instance is selected, s i ∈{pending state, selected state}; r i When the i-th target instance has been selected, it records the order in which it is selected. It is a non-negative integer. Let its value be k, satisfying where k = {1, 2, ..., n-1}, and for all selected target instances in each round, their r i The value is unique and is assigned in ascending order according to the order in which it is selected. i Store the feature set of the target instance to be selected. Secondly, during the operation of the entire salient target sorting network, as the interaction proceeds, the relevant information info of each target instance in the dictionary data structure M i The order in which the target instances are selected will be recorded according to the selection of the target instances, and the remaining target instances and their feature sets will be sent to the agent as new states for the next round of interaction until the end of the round and the final result of the ranking of significant targets is output.

[0024] like Figure 2 As shown, the action executor updates the state of the memory unit according to the result of the agent's selection and rewards or punishes the agent's action according to the true significance of the target instance.

[0025] like Figure 3 As shown in (b), when the agent selects the jth target instance as the most significant target among the remaining target instances, the action executor updates the dictionary data structure M of the memory unit corresponding to the selected target instance, from s j =Change the pending status to s j = Selected state and record the order in which the target is selected: r j =max{r i |s i = selected state}+1, and at the same time, the remaining target instances and their feature sets are updated to the new state of the memory unit M: NewState=UpdateState({f i |s i = candidate state, i = {1,2,…,n}}), UpdateState(·) is the update operation. In addition, the action executor will reward or penalize the score of the agent executing the current action according to the real saliency of the salient target by setting a reward strategy and feedback the score to the agent. The adopted reward strategy is: if the agent selects the most salient of the remaining target instances, it will be rewarded, otherwise it will be penalized (that is, no cumulative bonus points are added in this round), specifically: let the real saliency of target instance i be r true_i , with a value in the interval [0,1]; when the agent selects the jth target instance as the most significant target among the remaining target instances, its initial score is set to score initial The score after being processed by the action executor is score final : The reward function R(·) is set to R(x) = αx, where α is the reward coefficient and x is the true significance value.

[0026] like Figure 2 As shown, the interaction is specifically as follows: the environment currently provides a significant target instance and its feature set (state), and the agent selects the target with the highest degree of significance from the set (action). The environment records the agent's choice and removes the target from the set (new state). This is an interaction, and the process is repeated for multiple interactions until all the target instances are removed from the set. In the training phase, the round ends when the agent completes the sorting of all target instances. In the reasoning phase, the agent's action space adds active stop actions to provide the agent with the opportunity to terminate the interaction early. When the agent determines that no matter which target is selected, it can only get a negative score, the agent can choose to end the sorting, that is, end the round.

[0027] After specific practical experiments, network deployment and model training were completed on a computer device with an Intel 9700k CPU, an Nvidia RTX 3090Ti graphics card, and 64GB of memory. Figure 4 As shown in FIG. 1 , the method for ranking significant targets based on reinforcement learning in this embodiment includes:

[0028] Step 1: Prepare the data set, including:

[0029] 1.1 Download the open-source ASSR dataset. Each original RGB scene image in the dataset has a corresponding saliency ranking truth map. This truth map is a grayscale image. The annotations of this dataset contain two types of files. One type of file is the saliency target annotation file, which contains the location, category, mask and other information of the salient targets in each image; the other type of file is the saliency level file of the saliency target, which contains the saliency level of each saliency target in the scene. The saliency level here is a floating point value in the range [0, 1].

[0030] 1.2 Divide the training set, validation set, and test set into a ratio of 7:1:2.

[0031] 1.3 Before starting training, the dataset needs to be preprocessed. In order to speed up the training process, the target instances and their features in the original image are extracted in advance and stored locally, thereby reducing redundant feature extraction steps. After using the CenterMask target instance segmentation model to predict the target location in the image and its corresponding mask, these masks are applied to the original image to obtain the target instance. The backbone network and feature pyramid network in CenterMask are then used to extract multi-layer, multi-scale features of the target instance. Subsequently, the set of these target locations, all target instances and their features are stored locally and used as input to the execution component of the model.

[0032] Step 2: Model training, including:

[0033] The loss function used in 2.1 is the mean square error (MSE), which is:

[0034] 2.2 Setting of hyperparameters, specifically: The threshold of the candidate target screening module in the environment is set to 0.5. The reward strategy of the action executor in the environment is set to: if the agent selects the most significant target, it will be scored, if it selects a non-significant target, it will be deducted, and no points will be deducted or added in other cases. The reward coefficient is set to 1. The number of update iterations is set to 20,000. In the training phase, the probability of randomly selecting an object is set to 0.9, the learning rate is set to 1e-6, and the batch size is set to 4. The agent is trained for 11,000 epochs using the training set obtained in step 1 and the model performance is evaluated on the validation set. The model structure or hyperparameters are adjusted according to the validation results to finally obtain the best performing model.

[0035] Step 3: Use the mean absolute error (MAE) indicator and f1-Sor indicator to evaluate the model performance.

[0036] The MAE index is specifically calculated as follows: first, a saliency map is drawn according to the saliency of the target. The higher the saliency of the target, the higher its brightness in the map. Then, the pixel-by-pixel difference between the normalized predicted saliency map and the actual saliency map is calculated, and then the average value is calculated. MAE measures the pixel-level difference between the predicted saliency ranking map and the true value. MAE not only considers the accuracy of ranking the top N most important objects, but also the recall rate and error rate of the model.

[0037] The f1-Sor index is specifically calculated as follows: first, the f1 value and the Sor value are calculated separately, and then the two are multiplied to obtain the f1-Sor score. Where Sor is the Pearson correlation coefficient. The f1 value is calculated as follows: first, the recall rate (Recall) and the precision (precision) of the significant target are calculated, and then:

[0038] As shown in Table 1, the performance evaluation test results of the model test set are as follows: Figure 5 shown.

[0039] Table 1. Model test set performance evaluation method f1-SOR of ASSR dataset (higher is better) MAE of ASSR dataset (lower is better) SORPPA 0.746 0.081 PSR 0.782 0.075 OCOR 0.717 0.086 ASSR 0.774 0.101 IRSR - 0.105 SORCSR - 0.081 hyperSor - 0.101 The present invention 0.788 0.074

[0040] As shown in Table 1, on the ASSR dataset, the salient object ranking accuracy (f1-SOR) of the trained model reaches 0.788, reaching the current best level (the short dash - represents the lack of public data and the inability to calculate the indicator). Similarly, on the MAE indicator, the trained salient object ranking model also stands out with a performance of 0.074, further verifying its ability to dynamically calculate the salient ranking process and accurately capture and sort the salient objects in the image.

[0041] As shown in Table 2, the test evaluation of the key links of model performance improvement is shown. The baseline model involves directly inputting the target features into the linear layer, which is directly trained using reinforcement learning. On this basis, integrating the SAM or TAM module can improve the performance of the model in terms of F1-Sor and MAE indicators. When the SAM and TAM modules are combined at the same time, the best overall performance can be achieved, indicating that the simultaneous use of these two modules has a synergistic effect.

[0042] Table 2. Test results of key aspects of model performance improvement method f1-SOR of ASSR dataset (higher is better) MAE of ASSR dataset (lower is better) Benchmarks 0.732 0.075 Benchmark + TAM 0.748 0.074 Benchmark+SAM 0.787 0.074 Benchmark + SAM + TAM 0.788 0.074

[0043] Compared with the prior art, the core advantage of the present invention is that it makes the modeling process dynamic, breaks through the limitations of traditional static modeling, and effectively improves the performance of significant target sorting. By constructing the model as a dynamic and sustainable optimization process, the present method can adapt to changes in the data environment in real time, thereby significantly improving the flexibility and applicability of the model. In actual application scenarios, data often presents non-static characteristics. Traditional methods usually achieve prediction through static feature extraction, lack dynamic adaptability, and are difficult to capture the dynamic transfer of human visual attention mechanism according to the relationship between context and target, resulting in the inability to accurately reflect the natural flow of human vision in the sorting task, affecting the accuracy and practicality of the results. However, the present method dynamically adjusts the modeling strategy, uses the interaction between the environment and the agent to simulate the dynamic transfer process of human visual attention, and enhances the model's context understanding and target relationship capture capabilities through the TAM and SAM modules in the agent, so as to more accurately capture the temporal characteristics and potential change laws of the data, and effectively solves this problem. In general, the present method breaks through the limitations of static modeling through a dynamic modeling process, provides a flexible, efficient and reliable solution for complex data modeling, and has broad application prospects and practical value.

[0044] The above-mentioned specific implementation can be partially adjusted in different ways by those skilled in the art without departing from the principle and purpose of the present invention. The protection scope of the present invention shall be based on the claims and shall not be limited by the above-mentioned specific implementation. Each implementation scheme within its scope shall be subject to the constraints of the present invention.

Claims

1. A method for ranking salient objects based on reinforcement learning, characterized in that: In the offline stage, the environment and the agent are constructed, and the interaction between the agent and the environment is trained through the salient object ranking dataset to obtain the result of the salient object ranking in the scene; The environment includes: an environment candidate generator (ECG), an action executor and a memory unit, wherein: after the ECG segments all target instances in the original image and extracts the corresponding target instance features, a significant candidate target instance and its feature set are obtained through screening; the memory unit uses the feature set as the initial state for recording and state maintenance, and the intelligent agent performs spatial context information feature fusion and temporal context information fusion in turn according to the initial state, and predicts the cumulative score of the executed action; the action executor judges the overall performance of the currently selected target instance according to the cumulative score, and then rewards or punishes the action according to the set reward strategy to update the parameters of the intelligent agent, and at the same time, records the selected target instances and their order according to the relationship between the results of the intelligent agent selection and the processed target instances, and updates the state in the memory unit, that is, the remaining candidate target instances and their feature sets; The intelligent agent includes: a spatial attention module (SAM), a temporal attention module (TAM) and a scoring head (ScoreHead), wherein: the spatial attention module integrates information of surrounding target instances, that is, spatial context information into target instance features; the temporal attention module calculates the temporal interaction between significant targets, that is, the degree of significance of the target instance noticed in the previous time step affecting the target instance noticed in the next time step; the scoring head predicts the value of selecting each action (Action), that is, the cumulative score predicted by the scoring head.

2. The method for ranking significant objects based on reinforcement learning according to claim 1, characterized in that: The environment candidate generator (ECG) extracts and screens out significant candidate target instances and their feature sets from the original image, including: a target instance segmentation module, a feature extraction module and a candidate target screening module, wherein: the target instance segmentation module generates masks (Mask) and target instance bounding boxes (BoundingBox) of all target instances according to the original image and outputs them to the feature extraction module together with the original image to obtain the features of all target instances; the candidate target screening module screens out significant candidate target instances and their feature sets from the set of all target instance features according to the target instance bounding box (BoundingBox).

3. The method for ranking significant objects based on reinforcement learning according to claim 1, characterized in that: The integration mentioned above refers to: through the structure of self-attention, is a feature set of candidate target instances that incorporates spatial context information. is the feature of target instance i containing spatial context information, i∈[1,2,....n], n is the number of target instances to be selected, and the global feature o g is the P5 feature map of the original image, the dimension of the feature map is d, and the set of significant target instance features is O candidate The feature set O of the candidate target instance fused with spatial context information is calculated as follows: s =ReLU(W a Attn s V), where: Q=W q O candidate , K=W k [O candidate ,o g ],V=W v [O candidate ,o g ], Q, K, V represent the query vector (Query), key vector (Key) and value vector (Value) in the attention mechanism respectively; W q , W k and W v It is a trainable parameter, which is used to convert the input vector into Query, Key and Value vectors; Attn s is the spatial attention matrix, which is the attention distribution in space; T is the transposition operation; W a It acts on Attn s The trainable parameters.

4. The method for ranking significant objects based on reinforcement learning according to claim 1, characterized in that: The calculation of the temporal interaction between the salient targets refers to: calculating the feature set of the candidate target instance that integrates the temporal context information in: is the feature of the target instance i noticed in the current time step, i∈[1,2,....n], n is the number of target instances to be selected, let o t-1 is the feature of the target instance noticed in the previous time step, is the feature of target instance i containing spatial context information, W is a trainable parameter.

5. The method for ranking significant objects based on reinforcement learning according to claim 1, characterized in that: The predicted value of each action is: candidate The action of selecting a specific target instance and candidate The number of target instances n in state s t Next, select action a and continue to use the current strategy π to interact with the environment multiple times until the cumulative score at the end of the round W g and W h is a trainable parameter, It is a global feature o g Features after linear layer mapping.

6. The method for ranking significant objects based on reinforcement learning according to claim 1, characterized in that: The environment candidate generator comprises: a target instance segmentation module, a feature extraction module and a candidate target screening module, wherein: the target instance segmentation module adopts a CenterMask model, and generates a target instance mask and a target instance bounding box through the mask head and the bounding box head of the model; the feature extraction module uses the result of bitwise AND calculation of the target instance mask and the original image as input, then uses the backbone network of the CenterMask model and combines with the feature pyramid network to extract multi-layer and multi-scale feature maps of all target instances and uses the fifth layer feature map (P5) in the feature pyramid as the output target instance features and scene features, and finally forms a set of all target instances and their features; the candidate target screening module screens out significant candidate target instances and their feature sets from the set of all target instances and their features, specifically: firstly, the intersection over union (IoU) is calculated according to the true value and the predicted target instance bounding box, and then the real significant target is screened out according to the intersection over union.

7. The method for ranking significant objects based on reinforcement learning according to claim 1, characterized in that: The memory unit records and maintains the state in the environment and transmits the current state to the agent while outputting the final salient target sorting result, that is, recording the pending and selected states of the target instance. Specifically, a mapping from the target instance identifier set to the target instance related information set is constructed, that is, a dictionary data structure M = {(id i ,info i )}|i={1,2,…,n}, where: id i is the unique identifier of the i-th target instance, specifically a digital number, info i is a set of relevant information corresponding to the target instance, n is the total number of target instances, and the info corresponding to the i-th target instance is i ={s i ,r i ,f i },s i is the case where the i-th target instance is selected, s i ∈{pending state, selected state}; r i When the i-th target instance has been selected, it records the order in which it is selected. It is a non-negative integer. Let its value be k, satisfying where k = {1, 2, ..., n-1}, and for all selected target instances in each round, their r i The value is unique and is assigned in ascending order according to the order of selection. i Store the feature set of the target instance to be selected. Secondly, during the operation of the entire salient target sorting network, as the interaction proceeds, the relevant information info of each target instance in the memory unit M is i The order in which the target instances are selected will be recorded according to the selection of the target instances, and the remaining target instances and their feature sets will be sent to the agent as new states for the next round of interaction until the end of the round and the final result of the ranking of significant targets is output.

8. The method for ranking significant objects based on reinforcement learning according to claim 1, characterized in that: The action executor updates the state of the memory unit according to the result of the agent's selection and rewards or punishes the agent's action according to the real prominence of the target instance. Specifically, when the agent selects the jth target instance as the most significant target among the remaining target instances, the action executor updates the selection of the corresponding target instance in the memory unit M, and j =Change the pending status to s j = Selected state and record the order in which the target is selected: r j =max{r i |s i = selected state}+1, and at the same time, the remaining target instances and their feature sets are updated to the new state of the memory unit M: NewState=UpdateState({f i |S i = Waiting state, i = { 1,2,…,n}}), UpdateState(·) is the update operation. In addition, the action executor will reward or penalize the score of the agent executing the current action according to the real saliency of the salient target by setting a reward strategy and feedback the score to the agent. The adopted reward strategy is: if the agent selects the most salient of the remaining target instances, it will be rewarded, otherwise it will be penalized (that is, no cumulative bonus points are added in this round). Specifically, let the real saliency of target instance i be r true_i , with a value in the interval [0,1]; when the agent selects the jth target instance as the most significant target among the remaining target instances, its initial score is set to score initial , the score after being processed by the action executor is score final : The reward function R(·) is set to R(x) = αx, where α is the reward coefficient and x is the true significance value.

9. The method for ranking significant objects based on reinforcement learning according to claim 7, characterized in that: The interaction is specifically as follows: the environment currently provides a candidate target instance with significant features and a feature set (state), the agent selects a target (action) with the highest degree of significant features from the set, the environment records the agent's choice and removes the target from the set (new state), and an interaction is completed; the interaction is repeated multiple times until all candidate target instances are removed from the set, which is a round; In the training phase, the round ends when the agent completes the sorting of all candidate target instances. In the reasoning phase, the agent's action space adds active stop actions to provide the agent with the opportunity to terminate the interaction early. When the agent determines that no matter which target is selected, it can only get a negative score, the agent chooses to end the sorting, that is, end the round.

Images